Wednesday December 22, 2010

[Time] NameMessage
[07:14] ytang hi, anyone online?
[07:33] sustrik hi
[07:38] Steve-o good morning
[08:05] mindful anybody in
[08:08] sustrik yup
[10:29] mikko good morning
[10:29] mikko i made it
[10:39] Skaag made what?
[10:39] Skaag to work? :)
[12:29] mikko Skaag: made it home for christmas
[12:29] mikko Skaag: i was flying from london to finland
[12:30] mikko took five days
[12:31] Skaag wow!
[12:31] Skaag why 5 days?!
[12:31] Skaag I thought 5 days is with a ferry
[12:31] Skaag London to Stockholm, then Stockholm to Helsinki
[12:32] Skaag brb
[12:34] mikko Skaag: tons of canceled flights
[12:34] mikko i was supposed to fly last friday and finally managed to fly tuesday evening
[12:35] Skaag sounds like a nightmare
[12:40] Skaag back.
[12:40] Skaag I really should bring my bouncer into freenode as well...
[15:55] dos000 howdy
[15:55] dos000 can someone tell me if concurrent lock free circular buffers are used in zmq ? i read something recently that was talking about the perf improvements
[16:43] cremes dos000: 0mq definitely uses lock-free algos; i don't know if they protect a circular buffer though
[16:43] cremes the code answers all!
[16:43] Skaag lol
[16:43] Skaag *snicker* typical developer answer ;)
[16:44] Skaag what documentation? look at the comments in the code... ;)
[16:44] cremes yep ;)
[16:45] cremes docs usually don't cover those kinds of implementation details anyway
[16:45] cremes they are intended to remain hidden under the covers; anyone using the api shouldn't care what the algo is
[16:46] cremes since dos000 was asking specifically about it, the only way to get the real answer is to 1) ask sustrik
[16:46] cremes or 2) check the source
[16:46] dos000 ah!
[16:48] dos000 i was also looking for some experience in other libs offering lock-free circular buffers
[16:48] dos000 this stuff seems like black magic
[16:48] cremes yes, very dark :)
[16:49] dos000 no one is doing it or talking about their implementation
[16:49] dos000 and no central de facto to go fo this .. evryone rolls their own
[16:49] cremes true
[16:49] cremes the main idea behind lock-free is to leverage a platform's atomic compare-and-swap operation
[16:50] cremes if your hw supports that, a lock-free impl is fairly "easy" to create
[16:50] dos000 yes .. and i am hoping (please) support it
[16:50] cremes the hard part then becomes dealing with cache coherency and its perf implications
[16:50] dos000 i am hoping 0mq has it for sparc and x86 and/or x86_64
[16:51] cremes it definitely has it for x86 and x86_64; not sure on sparc
[16:51] dos000 cremes: zmq does not deal with threads correct ? i was thinking a way of "forcing" a thread to run on a particular core
[16:52] cremes dos000: not correct; you can force cpu affinity for sockets
[16:52] cremes check the setsockopt docs on the web site
[16:52] dos000 ok
[16:52] cremes (i haven't used it myself otherwise i'd be more helpful...)
[16:52] dos000 here is the (interesting) talk
[16:53] cremes ah, i just saw that this morning! it's in my queue to watch a bit later
[16:53] dos000 ah!
[16:53] cremes i do trading stuff but i'm not in the super high-freq space
[16:54] dos000 they mention someway of sending a packet straight to user space bypassing (virtually) the kernel ... but they dont explain how or which os supports it
[16:55] dos000 very interestng talk indeed
[16:55] dos000 cremes, which blog/sites do you follow for this types of talks ?
[16:56] dos000 how did you stumble on the LMAX presentation ?
[16:56] cremes i subscribe to reddit rss feeds as well as the infoq rss stuff
[16:58] dos000 the solution they propose starts from 36:00 if you want to skip through .. a picture is worth a thousand words
[16:58] cremes cool, thanks for the heads up
[16:58] cremes are you involved with trading systems or similar?
[16:58] dos000 yes very similar .. payment systems
[16:59] Skaag dos000: don't the boost libraries do this type of locking?
[17:00] dos000 i asked already in #C++ it might be only circular buffers .. not lock free implementations
[17:01] Skaag weren't all such kernel modules that broke the core -> user space membrane been kicked from the kernel project by Linus?
[17:02] dos000 no idea!
[17:02] Skaag they are doing this without a custom kernel module?
[17:02] dos000 not sure how .. its just a blimp in the video
[17:40] dos000 ok ... now i found this
[17:52] mikko dos000:
[17:52] mikko this might be interesting read as well
[17:52] mikko not directly related
[17:58] dos000 fastflow seems to be up there with 0mq .. lots of similarities
[18:03] cremes dos000: they diss ruby in that blog so now i need to go on a rant! ;)
[18:08] Skaag lol!!!
[18:08] dos000 lol
[20:11] Skaag the developer I work with tells me he managed to publish and consume a message on a topic with rabbitmq, but it's still unclear to him how to cluster the thing ;)
[21:41] mikko Skaag: you can publish and consume a message with zeromq within minutes
[21:41] mikko :)
[21:42] Skaag I know
[21:42] Skaag anyway, what I am thinking to answer him, is that Zookeeper will be in charge of managing a list of IP addresses of available hosts
[21:42] Skaag and he will just connect to all of them and subscribe
[21:43] mikko did you test zookeeper
[21:43] mikko ?
[21:43] Skaag not yet
[21:43] Skaag I'm installing it as we speak
[21:44] mikko the only annoyance i found was that zookeeper didnt have rhel packaging
[21:44] mikko i think i made spec-files back then
[21:45] Skaag well I am running on Ubuntu myself. I can't stand rhel.
[21:45] Skaag I used to be very redhat oriented
[21:45] Skaag but once I was exposed to debian, and then to ubuntu, I knew my destiny was sealed ;)
[21:46] mikko i use debian on my own stuff
[21:46] Skaag I know it may sound stupid, and rhel is considered a kind of 'standard'.
[21:46] mikko but many companies i work with prefer rhel
[21:46] dos000 i hate rhel .. anything non debian (based) for that matter
[21:47] dos000 apt-get is sweet
[21:47] mikko rhel packaging is far more pleasant to make imho
[21:47] mikko compared to debs
[21:47] Skaag i disagree
[21:47] dos000 to use however isa different thing
[21:47] Skaag I make a deb in 3 minutes.
[21:47] Skaag less...
[21:48] Skaag well I have a ready skeleton, from the first 5 minutes I spent trying to make a deb
[21:48] dos000 there are some utilities that will create the pkgs for you .. based on make && make installl
[21:48] Skaag so now to make a deb is a matter of copying the files to the right places, specifying dependencies, and perhaps adding some install / uninstall scripts...
[21:48] Skaag not to mention how easy it is to create a repository
[21:49] mikko Skaag: it's easy for yum as well
[21:49] mikko createrepo /path/to
[21:49] dos000 i did not find companies that do support for debian so far
[21:50] dos000 something that passes the CEO test
[21:51] Skaag the CEO test is different in every company...
[21:51] Skaag in some, the money talks
[21:51] Skaag in some others, it's how large the support company is
[21:51] Skaag and in some, it's how technically knowledgable :-)
[21:51] dos000 in some companies the CEO (and evryone around him) is dumb !
[21:51] dos000 thats the problem
[21:51] Skaag and here in Russia! It's about how much money you slip into the CEO's pocket, behind the back of the board of directors...! ;-)
[21:52] dos000 heh!
[21:53] mikko im not far from russia atm
[21:53] mikko a few hundred km if even that
[21:54] mikko Skaag:
[21:54] mikko it's not very well tested but should give some ideas for zk integration
[21:54] Skaag mikko: I read your page already ;)
[21:54] Skaag ah, that's another page
[21:54] Skaag sorry
[21:54] Skaag Where in Finland are you?
[21:55] Skaag I took the bus from Helsinki to St. Petersburg, just 6 month ago :-)
[21:55] Skaag 17 euros and I was in Russia!
[21:56] mikko they got a new fast train as well
[21:56] mikko three hours to st petersburg from helsinki
[21:56] Skaag amazing
[21:56] dos000 lets say i had 3 packets coming from 3 different sources ... and all packets are sequentially numbered.
[21:56] Skaag but I doubt in such a weather condition?
[21:57] dos000 what would you do to wait for three packets with the same seq numbers ?
[21:57] mikko Skaag: even in such conditions
[21:57] Skaag amazing!
[21:57] mikko things dont stop in .fi even when its cold
[21:57] Skaag I know
[21:57] Skaag but on the russian side...?
[21:57] Skaag :-)
[21:57] Skaag I lived in Helsinki for 5 month
[21:57] Skaag I absolutely love Finland
[21:57] Skaag awesome country.
[21:57] Skaag I loved it since childhood.
[21:58] Skaag and recently, after living there, I understood why.
[21:58] mikko i dislike the winter. thats why i moved to london
[21:59] Skaag maybe because you grew up with it.
[21:59] Skaag this is my second REAL winter here in Moscow, first time in Helsinki, where it was almost constantly under -10˚c
[21:59] Skaag and I don't mind it much.
[22:01] mikko im in middle .fi now
[22:01] mikko -30 outside
[22:02] mikko does your company operate in russia?
[22:03] Skaag yes
[22:03] Skaag among other places
[22:03] Skaag middle = near Sodankylä?
[22:03] mikko thats the northest place in finland
[22:04] mikko close to the sea up north
[22:04] mikko pretty close to Jyväskylä
[22:05] Skaag cool
[22:05] Skaag I have a good friend in Mikkeli
[22:07] Skaag I actually passed through Jyväskylä one time
[22:09] Skaag they just prepared Olutglögi
[22:09] Skaagögi/14.jpg
[22:10] Skaag some part of me is glad I can not taste it.
[22:11] mikko warm beer and spices?
[22:11] Skaag actually it looks interesting
[22:13] Skaag yes
[22:59] ryandmonk hello... curious if someone can help me out with at pyzmq issue
[22:59] ryandmonk i followed the instructions on site
[22:59] ryandmonk and ran python install
[23:00] ryandmonk its spits out some stuff.. but eventually gives an error saying "zmq/core/constants.c:1: error: #error Do not use this file, it is the result of a failed Cython compilation."
[23:01] ryandmonk im on ubuntu 10.10.. using zeromq-2.0.10
[23:10] cremes ryandmonk: try searching the ML archives; i have seen discussions on python binding problems there before
[23:11] cremes perhaps the solution is buried in the archives
[23:12] ryandmonk where are these archives?
[23:13] Skaag cool - I have a working 16 node zookeeper cluster :)
[23:18] ryandmonk found em.. thanks cremes!