Tuesday June 22, 2010

[Time] NameMessage
[02:04] stuarthalloway if I run either the latency/throughput tests from Java, but increase the roundtrip count by a factor of 10, I get an exception:
[02:05] stuarthalloway "Context was terminated" at org.zeromq.ZMQ$Socket.recv
[02:05] stuarthalloway running with verbose:gc suggests that this happens immediately after the first garbage collection
[02:06] stuarthalloway ...which suggests a resource management issue, or maybe I have a bad build
[02:06] stuarthalloway I am using 2.0.7 plus the latest jzmq from git
[10:52] travlr sustrik: Hi Martin.. are you available?
[10:55] sustrik hi
[10:55] sustrik yeah, i'm here
[10:58] travlr Hi. I first want to appologize for the the email I (peter alexander) posted to the list last night. I was half asleep when I wrote it and didn't mean to suggest that I could know more about things than you do. I most certainly do not and I think thats the way my email may have come across.
[11:09] sustrik travlr: you very well may know more
[11:09] sustrik it's the pipelining
[11:09] sustrik i haven't been involved with that much
[11:09] sustrik the existing behaviour is based on few cases we've seen
[11:09] sustrik no theoretical beckground there
[11:10] travlr ok, very well. but I am self taught and no where near your caliber of expertise. enough said.
[11:11] travlr would you like to have a doxygen api representation for the zmq sources?
[11:12] sustrik what would that require?
[11:12] sustrik adding formatted comments to the source?
[11:13] travlr changing the comments from /* .. */ to /** .. */ and // to /// and then deciding on a suitable style sheet.
[11:15] travlr i could do a patch set or clone on github for you to decide.
[11:16] sustrik that would be great
[11:18] travlr ok I'll start with that today and while doing so I'll familiarize myself with the sources so I can hopefully help the community to formulate more flexible/powerful load balancing/distribution and possibly resource management aspects of zmq.
[11:23] sustrik great
[11:23] sustrik good luck :)
[11:24] travlr k. thanks.
[11:41] sjampoo I've written an introduction to ZeroMQ for my blog
[11:41] sjampoo It is currently in draft mode here:
[11:41] sjampoo I'd appreciate any feedback, especially if i am telling bullshit ;)
[11:43] sjampoo i'll be off for now, but would really like to hear if you think stuff is missing or wrong by email (to
[11:46] sustrik sjampoo: let me see
[11:47] sustrik the graphic design is awesome!
[11:52] sustrik sjampoo: what tool do you use for drawing diagrams?
[11:52] sustrik they are really neat
[11:57] sjampoo Thanks, its OmniGaffle for OSX, really one of my favorite tools
[11:57] sjampoo Have to catch a train now!
[11:58] travlr they are sweet. anything similar to omnigraffe on linux i wonder.
[12:00] sustrik doesn't look like :(
[12:02] travlr yeah not as sweet from what i have found but here' s a nice site for alternatives anyway.
[12:06] sustrik I would switch for OmniGaffe if I had a Mac box
[12:06] sustrik but looks like I'll stay with Dia for now
[12:07] travlr Dia is nice, i haven't used it in quite a while though. Aris Express looks like it might be cool.
[12:08] travlr but its trial ware :(
[12:09] travlr oh my mistake.. its not trial ware.
[12:18] sustrik maybe worth of trying - but it seems to be tied to a particular design methodology rather than being a generic drawing tool
[12:24] travlr yup. i almost jumped on the koffice team a year ago to modernize kivio, but as usual time is always a problem
[12:32] travlr sustrik: would you consider including an alternative cmake build system in zmq master if i told you i would be around to help maintain it?
[12:41] sustrik travlr: there's already a cmake build system here: git://
[12:41] sustrik if you want to help with it try speaking to the guy maintaining it
[12:44] travlr yes, i was aware of his fork. not a problem but merely a question. i'm not big on autotools so i thought i'd simply ask you.
[12:46] sustrik well, the state of affairs is that the autotools build works on all supported platforms
[12:46] sustrik if cmake build wants to become an alternative
[12:46] sustrik there's a lot of testing to do on different platforms
[12:47] travlr i see. cmake is portable to all platforms though i believe. right?
[12:49] sustrik i think so, but you have to deal with petty details by hand
[12:50] sustrik have a look here:
[12:50] sustrik different things done on different platforms...
[12:50] travlr yes. no prob. something for me to think about in the future maybe. i'm not yet schooled on all platform intricacies so i'll just use it privately for now.
[15:40] bapt Hi alk
[15:41] bapt is it possible with using pubsub that only one client (randomly) get the published messages
[15:41] bapt ?
[15:42] bapt I mean the publisher send a message N clients subscribed to it but I want only one client to really do the treatment
[15:56] cremes bapt: so you want all clients to receive the message but only 1 client to actually process the data?
[15:57] bapt yes
[15:58] cremes i would recommend looking at the butterfly example; it doesn't use pub/sub
[15:58] bapt in fact what I want is butterfly (I'm looking at the tutorail right now)
[15:58] bapt :)
[15:58] cremes it uses downstream/upstream sockets
[15:58] cremes oh, then you are on the right track!
[15:59] cremes do you have a specific question on that example?
[15:59] bapt not currently
[15:59] bapt thanks
[16:00] cremes ok
[16:11] bapt it's incredibly easy and powerfull
[16:12] bapt thanks a lot for 0mq, you saved a lot of my coding time :)
[19:23] jldupont hi - is there an official apt repository? i.e. apt-get
[19:25] jldupont anyone??
[19:28] mivert jldupont: 2.0.6 is in sid and maverick iMatix does not produce debs though
[19:29] jldupont mivert: thanks.
[19:34] sustrik jldupont: there are deb packaging scripts in debian subdirectory, so in theory you can build the package yourself
[19:34] jldupont sustrik: thanks. Is there a problem if I package and publish on Launchpad?
[19:35] sustrik no, no problem at all
[19:35] jldupont just one doubt: can the Python bindings do non-blocking?
[19:36] sustrik you mean async, right?
[19:37] sustrik the messages being sent on the background
[19:37] jldupont I guess. I want to "recv" with timeout.
[19:37] sustrik ah, there's no timeout in recv
[19:37] jldupont i.e. no blocking in receive loop
[19:37] sustrik so you have to poll
[19:37] sustrik (that one has timeout)
[19:37] sustrik and then recv
[19:37] sustrik either blocking or non-blocking
[19:38] sustrik all these features should be available in pyzmq
[19:38] jldupont great.
[19:38] jldupont so, if I want a "message fabric", I have to build one myself i.e. there isn't a "broker/switch" in the system as in AMQP. Right?
[19:39] sustrik there are so called "devices"
[19:40] sustrik they are like micro-brokers
[19:40] sustrik say an executable that is in fact a queue
[19:40] sustrik single one
[19:40] sustrik (zmq_queue)
[19:40] sustrik but you can build the fabric by hand if you wish so
[19:41] jldupont hmmm... ... sort of empty...
[19:42] sustrik have a look here
[19:43] jldupont ok thanks. I'll check it out later. Ciao.
[19:43] sustrik cyl
[19:49] soren I'm wondering about a few things wrt to pub/sub using multicast.
[19:50] soren I haven't completely wrapped my head around multicast yet, so please bear with me.
[19:50] soren :)
[19:51] soren Ok, so say I have a bunch of publishers (let's say 100-ish) and a /lot/ of subscribers (let's say 100000-ish).
[19:51] soren They all "connect" (is that even the correct term for multicast) to some multicast IP and port.
[19:51] soren The publishers have no idea how many subscribers there are, right?
[19:52] soren The distribution is handled by... what? The networking infrastructure?
[19:52] soren Like routers/switches or whatnot, I mean?
[19:54] soren Furthermore, if the subscribers only set a message filter using ZMQ_SUBSRIBE, they still receive all the messages, they're just filtered by zmq and never reach my application, correct?
[19:57] soren sustrik: Can I pick your brain on this ^^ for a few minutes? :)
[19:58] sustrik the right term is "to join a multicast group"
[19:58] sustrik publishers have no idea of subscribers and vice versa
[19:59] sustrik data are distributed by your network switch
[19:59] sustrik in theory multicast traffic can pass via router to another subnet but don't try that at home
[19:59] soren Right.
[19:59] soren Ok, so it requires a reasonably capable switch.
[20:00] sustrik look for switches with "IGMP snooping"
[20:00] soren I mean... a $20 gigabit switch probably won't give me much multicast love.
[20:00] sustrik have a look at the specs
[20:00] soren Sure. Ok.
[20:01] sustrik for high-perf scenarios you should always keep in mind that the actual throughput is limited by switches backplane throughput
[20:01] soren Sure.
[20:01] sustrik ~100Gb/sec for real switches
[20:01] sustrik (not toy switches)
[20:01] soren :)
[20:02] soren I assume a multicast group is defined by the IP alone?
[20:03] sustrik yes
[20:03] soren Ok.
[20:03] sustrik it's a range of IP addresses you can choose from
[20:03] soren Sure, sure, I'm just thinkgin..
[20:04] soren my connection string also includes a port number, but if the multicast group is defined by IP (and not the port), I want to make sure not to just use one IP with multiple ports.
[20:05] soren ...Since I might as well let the switch help me filter messages.
[20:06] soren Am I making sense? :)
[20:09] soren brb
[22:50] jldupont is there more information on how to configure the "zmq_forwarder" application?
[22:58] jldupont or there isn't much to configure else than what's up on ? I.e. can the forwarder discriminate against the "message type" and forward only to subscribers interested by a given "message type"?