ZeroMq IRC Log

Tuesday June 22, 2010

[Time] Name	Message
[02:04] stuarthalloway	if I run either the latency/throughput tests from Java, but increase the roundtrip count by a factor of 10, I get an exception:
[02:05] stuarthalloway	"Context was terminated" at org.zeromq.ZMQ$Socket.recv
[02:05] stuarthalloway	running with verbose:gc suggests that this happens immediately after the first garbage collection
[02:06] stuarthalloway	...which suggests a resource management issue, or maybe I have a bad build
[02:06] stuarthalloway	I am using 2.0.7 plus the latest jzmq from git
[10:52] travlr	sustrik: Hi Martin.. are you available?
[10:55] sustrik	hi
[10:55] sustrik	yeah, i'm here
[10:58] travlr	Hi. I first want to appologize for the the email I (peter alexander) posted to the list last night. I was half asleep when I wrote it and didn't mean to suggest that I could know more about things than you do. I most certainly do not and I think thats the way my email may have come across.
[11:09] sustrik	travlr: you very well may know more
[11:09] sustrik	it's the pipelining
[11:09] sustrik	i haven't been involved with that much
[11:09] sustrik	the existing behaviour is based on few cases we've seen
[11:09] sustrik	no theoretical beckground there
[11:10] travlr	ok, very well. but I am self taught and no where near your caliber of expertise. enough said.
[11:11] travlr	would you like to have a doxygen api representation for the zmq sources?
[11:12] sustrik	what would that require?
[11:12] sustrik	adding formatted comments to the source?
[11:13] travlr	changing the comments from /* .. / to /* .. */ and // to /// and then deciding on a suitable style sheet.
[11:15] travlr	i could do a patch set or clone on github for you to decide.
[11:16] sustrik	that would be great
[11:18] travlr	ok I'll start with that today and while doing so I'll familiarize myself with the sources so I can hopefully help the community to formulate more flexible/powerful load balancing/distribution and possibly resource management aspects of zmq.
[11:23] sustrik	great
[11:23] sustrik	good luck :)
[11:24] travlr	k. thanks.
[11:41] sjampoo	I've written an introduction to ZeroMQ for my blog
[11:41] sjampoo	It is currently in draft mode here: http://nichol.as/?p=606&shareadraft=baba606_4c20a0df10298
[11:41] sjampoo	I'd appreciate any feedback, especially if i am telling bullshit ;)
[11:43] sjampoo	i'll be off for now, but would really like to hear if you think stuff is missing or wrong by email (to nicholas@nichol.as)
[11:46] sustrik	sjampoo: let me see
[11:47] sustrik	the graphic design is awesome!
[11:52] sustrik	sjampoo: what tool do you use for drawing diagrams?
[11:52] sustrik	they are really neat
[11:57] sjampoo	Thanks, its OmniGaffle for OSX, really one of my favorite tools
[11:57] sjampoo	Have to catch a train now!
[11:58] travlr	they are sweet. anything similar to omnigraffe on linux i wonder.
[12:00] sustrik	doesn't look like :(
[12:02] travlr	yeah not as sweet from what i have found but here' s a nice site for alternatives anyway. http://alternativeto.net/desktop/omnigraffle/?platform=linux
[12:06] sustrik	I would switch for OmniGaffe if I had a Mac box
[12:06] sustrik	but looks like I'll stay with Dia for now
[12:07] travlr	Dia is nice, i haven't used it in quite a while though. Aris Express looks like it might be cool. http://alternativeto.net/desktop/aris-express/about
[12:08] travlr	but its trial ware :(
[12:09] travlr	oh my mistake.. its not trial ware.
[12:18] sustrik	maybe worth of trying - but it seems to be tied to a particular design methodology rather than being a generic drawing tool
[12:24] travlr	yup. i almost jumped on the koffice team a year ago to modernize kivio, but as usual time is always a problem
[12:32] travlr	sustrik: would you consider including an alternative cmake build system in zmq master if i told you i would be around to help maintain it?
[12:41] sustrik	travlr: there's already a cmake build system here: git://github.com/PatrickCheng/zeromq2.git
[12:41] sustrik	if you want to help with it try speaking to the guy maintaining it
[12:44] travlr	yes, i was aware of his fork. not a problem but merely a question. i'm not big on autotools so i thought i'd simply ask you.
[12:46] sustrik	well, the state of affairs is that the autotools build works on all supported platforms
[12:46] sustrik	if cmake build wants to become an alternative
[12:46] sustrik	there's a lot of testing to do on different platforms
[12:47] travlr	i see. cmake is portable to all platforms though i believe. right?
[12:49] sustrik	i think so, but you have to deal with petty details by hand
[12:50] sustrik	have a look here: http://github.com/zeromq/zeromq2/blob/master/configure.in
[12:50] sustrik	different things done on different platforms...
[12:50] travlr	yes. no prob. something for me to think about in the future maybe. i'm not yet schooled on all platform intricacies so i'll just use it privately for now.
[15:40] bapt	Hi alk
[15:41] bapt	is it possible with using pubsub that only one client (randomly) get the published messages
[15:41] bapt	?
[15:42] bapt	I mean the publisher send a message N clients subscribed to it but I want only one client to really do the treatment
[15:56] cremes	bapt: so you want all clients to receive the message but only 1 client to actually process the data?
[15:57] bapt	yes
[15:58] cremes	i would recommend looking at the butterfly example; it doesn't use pub/sub
[15:58] bapt	in fact what I want is butterfly (I'm looking at the tutorail right now)
[15:58] bapt	:)
[15:58] cremes	it uses downstream/upstream sockets
[15:58] cremes	oh, then you are on the right track!
[15:59] cremes	do you have a specific question on that example?
[15:59] bapt	not currently
[15:59] bapt	thanks
[16:00] cremes	ok
[16:11] bapt	it's incredibly easy and powerfull
[16:12] bapt	thanks a lot for 0mq, you saved a lot of my coding time :)
[19:23] jldupont	hi - is there an official apt repository? i.e. apt-get
[19:25] jldupont	anyone??
[19:28] mivert	jldupont: 2.0.6 is in sid and maverick http://packages.debian.org/source/sid/zeromq https://launchpad.net/ubuntu/maverick/+source/zeromq iMatix does not produce debs though
[19:29] jldupont	mivert: thanks.
[19:34] sustrik	jldupont: there are deb packaging scripts in debian subdirectory, so in theory you can build the package yourself
[19:34] jldupont	sustrik: thanks. Is there a problem if I package and publish on Launchpad?
[19:35] sustrik	no, no problem at all
[19:35] jldupont	just one doubt: can the Python bindings do non-blocking?
[19:36] sustrik	you mean async, right?
[19:37] sustrik	the messages being sent on the background
[19:37] jldupont	I guess. I want to "recv" with timeout.
[19:37] sustrik	ah, there's no timeout in recv
[19:37] jldupont	i.e. no blocking in receive loop
[19:37] sustrik	so you have to poll
[19:37] sustrik	(that one has timeout)
[19:37] sustrik	and then recv
[19:37] sustrik	either blocking or non-blocking
[19:38] sustrik	all these features should be available in pyzmq
[19:38] jldupont	great.
[19:38] jldupont	so, if I want a "message fabric", I have to build one myself i.e. there isn't a "broker/switch" in the system as in AMQP. Right?
[19:39] sustrik	there are so called "devices"
[19:40] sustrik	they are like micro-brokers
[19:40] sustrik	say an executable that is in fact a queue
[19:40] sustrik	single one
[19:40] sustrik	(zmq_queue)
[19:40] sustrik	but you can build the fabric by hand if you wish so
[19:41] jldupont	hmmm... http://api.zeromq.org/zmq_forwarder.html ... sort of empty...
[19:42] sustrik	have a look here http://github.com/zeromq/zeromq-chat/blob/master/README
[19:43] jldupont	ok thanks. I'll check it out later. Ciao.
[19:43] sustrik	cyl
[19:49] soren	I'm wondering about a few things wrt to pub/sub using multicast.
[19:50] soren	I haven't completely wrapped my head around multicast yet, so please bear with me.
[19:50] soren	:)
[19:51] soren	Ok, so say I have a bunch of publishers (let's say 100-ish) and a /lot/ of subscribers (let's say 100000-ish).
[19:51] soren	They all "connect" (is that even the correct term for multicast) to some multicast IP and port.
[19:51] soren	The publishers have no idea how many subscribers there are, right?
[19:52] soren	The distribution is handled by... what? The networking infrastructure?
[19:52] soren	Like routers/switches or whatnot, I mean?
[19:54] soren	Furthermore, if the subscribers only set a message filter using ZMQ_SUBSRIBE, they still receive all the messages, they're just filtered by zmq and never reach my application, correct?
[19:57] soren	sustrik: Can I pick your brain on this ^^ for a few minutes? :)
[19:58] sustrik	the right term is "to join a multicast group"
[19:58] sustrik	publishers have no idea of subscribers and vice versa
[19:59] sustrik	data are distributed by your network switch
[19:59] sustrik	in theory multicast traffic can pass via router to another subnet but don't try that at home
[19:59] soren	Right.
[19:59] soren	Ok, so it requires a reasonably capable switch.
[20:00] sustrik	look for switches with "IGMP snooping"
[20:00] soren	I mean... a $20 gigabit switch probably won't give me much multicast love.
[20:00] sustrik	have a look at the specs
[20:00] soren	Sure. Ok.
[20:01] sustrik	for high-perf scenarios you should always keep in mind that the actual throughput is limited by switches backplane throughput
[20:01] soren	Sure.
[20:01] sustrik	~100Gb/sec for real switches
[20:01] sustrik	(not toy switches)
[20:01] soren	:)
[20:02] soren	I assume a multicast group is defined by the IP alone?
[20:03] sustrik	yes
[20:03] soren	Ok.
[20:03] sustrik	it's a range of IP addresses you can choose from
[20:03] soren	Sure, sure, I'm just thinkgin..
[20:04] soren	my connection string also includes a port number, but if the multicast group is defined by IP (and not the port), I want to make sure not to just use one IP with multiple ports.
[20:05] soren	...Since I might as well let the switch help me filter messages.
[20:06] soren	Am I making sense? :)
[20:09] soren	brb
[22:50] jldupont	is there more information on how to configure the "zmq_forwarder" application?
[22:58] jldupont	or there isn't much to configure else than what's up on http://www.zeromq.org/docs:cookbook#toc8 ? I.e. can the forwarder discriminate against the "message type" and forward only to subscribers interested by a given "message type"?