ZeroMq IRC Log

Thursday January 27, 2011

[Time] Name	Message
[00:19] stodge	I just started reading about zeromq, so my quick question is, can it be used to create a jgroups-like technology?
[00:19] stodge	I'm reading about publish/suscribe
[00:20] stodge	Wondering if it could work as a heartbeat mechanism between an unknown number of peers (talking less than 100)
[00:20] stodge	Reading to see if zeromq could be used for peer discovery
[00:40] lt_schmidt_jr	why not use zookeeper for something like that?
[00:45] lt_schmidt_jr	comment aimed at stodge
[00:45] stodge	I haven't read about zookeeper yet - it's on my list. My system is primarily (95%) Python so I jumped at zeromq when I saw the Python API.
[00:45] stodge	Thanks
[00:47] lt_schmidt_jr	stodge, NP, I believe there is a python binding. not that you should not use zeromq, but I think zookeeper is probably better suited for this particular task (and simpler)
[00:48] stodge	ok Thanks, I'll definitely read up on zookeeper
[00:48] lt_schmidt_jr	simpler for the task - not in general
[01:09] stockMQ	Hi.. Can anyone confirm that pgm multicast will work on Windows and not restricted to LINUX
[04:48] stockMQ	Anyone doing multicast on windows?
[04:51] jugg	http://www.google.com/search?hl=en&q=site:lists.zeromq.org+multicast+windows
[04:59] stockMQ	I see there were some issues in the past for support of pgm for windows
[05:00] stockMQ	hence just would like to know if it is supported now before i get into using it
[05:06] stockMQ	http://lists.zeromq.org/pipermail/zeromq-dev/2010-September/006274.html
[05:07] stockMQ	I am on windows and have a similar requirement to do multicast using PUB-SUB
[05:07] stockMQ	does this problem still persist??
[05:43] stockMQ	anyone?
[07:26] sustrik	mikko: morning
[07:53] mikko	sustrik: hi
[07:54] mikko	sustrik: what is the policy on constructor errors ? i noticed that there arent really throws
[07:57] sustrik	mikko: the exceptions are not used at all
[07:57] sustrik	when you need an error in constructor
[07:57] sustrik	you separate the construction into 2 phases
[07:57] sustrik	1. the part that cannot fail (constructor)
[07:57] sustrik	2. the part that can fail (init function)
[07:58] sustrik	the latter can return error
[08:08] stockMQ	http://lists.zeromq.org/pipermail/zeromq-dev/2010-September/006274.html
[08:08] stockMQ	does this problem still persist??
[08:08] stockMQ	I am on windows and have a similar requirement to do multicast using PUB-SUB
[08:12] sustrik	stockMQ: i have no idea what's the status quo wrt OpenPGM on Windows
[08:12] sustrik	there's definitely no MSVC intrgrated build
[08:12] sustrik	but you may try MinGW
[08:13] stockMQ	ok..Please confirm my understanding
[08:13] stockMQ	to multicast two options available are pgm,epgm
[08:14] stockMQ	But there does not seem to be a support for windows and maybe a minGW port could only be the option
[08:14] stockMQ	Is that right?
[08:18] stockMQ	??
[08:47] sustrik	stockMQ: it should work on Windows
[08:47] sustrik	the problem is that the build system is not at all obvious
[08:47] sustrik	so, it's hard to build, but once you build it, it should work :)
[09:01] stockMQ	okk.:).. Any ideas if i can test the pgm multicast on the same host.. meaning both pub and sub being on the same machine
[09:03] sustrik	yes, you can
[09:03] sustrik	but it's actually a simulation rather than real multicast then
[09:03] sustrik	different code is used
[09:03] sustrik	so if you want to test the real multicast code, use it over the network
[09:08] stockMQ	yes.. i will be doing the real testing but i dont have access to the network full time..so was thinking of testing it on the simulation and then go ahead
[09:09] stockMQ	actually the loopback would do it right
[09:12] guido_g	hmm... i don't think that the loopback device is multicast capable, wouldn't make sense
[09:12] guido_g	you need to use the external ip of the box
[09:12] guido_g	adn this leads to a lot of free floating traffic in the network
[09:13] mikko	sustrik: hmmm, thats not exception safe (?)
[09:15] stockMQ	publisher->bind(" pgm://161.144.100.162;234.1.2.10:5555")
[09:15] stockMQ	the first IP is the NIC i want to bind to
[09:15] stockMQ	and the second one is the multicast group IP
[09:15] stockMQ	now how would this change if i wanted to test it on localhost
[09:15] stockMQ	Or will this not work on Windows altogether
[09:29] stockMQ	guido?
[09:31] guido_g	hm?
[09:31] stockMQ	any insights/
[09:32] stockMQ	?
[09:32] guido_g	into what?
[09:35] stockMQ	publisher->bind(" pgm://161.144.10.162;234.1.20.10:5555")
[09:35] stockMQ	Now how would this change for for localhost
[09:35] stockMQ	I believe the first IP is the NIC i want to bind to
[09:35] stockMQ	and second one is the multicast group ip
[09:35] guido_g	did you ever read the Ã¸mq docs?
[09:36] stockMQ	I read the guide
[09:36] guido_g	there is more
[09:37] stockMQ	http://zguide.zeromq.org/chapter:all
[09:39] stockMQ	http://api.zeromq.org/zmq_pgm.html
[09:39] stockMQ	this says first IP is the network interface
[09:40] stockMQ	that is what i think i mentioned..
[09:57] sustrik	mikko: what do you mean by exception-safe?
[09:58] sustrik	there are no exceptions used in the codebase
[10:38] mikko	sustrik: ok
[10:38] mikko	sustrik: i was thinking about situations where the initialisation method might throw
[10:38] mikko	but i guess if there is no exceptions that is not so much an issue
[10:39] sustrik	exactly
[10:39] sustrik	exceptions are good for programs with low reliabity requirement
[10:40] sustrik	say: "if something bad happens show a dialog box"
[10:41] sustrik	for systems that should run 24/7 without user intervention the errors should not be handled in generic manner (the practice encouraged by exceptions)
[10:41] sustrik	rather on one-by-one basis
[11:36] stimpie	Is it possible to acknowledge a message after it has been processed (not when received)?
[11:44] mikko	stimpie: what do you mean?
[11:44] mikko	stimpie: you would probably want to use REQ/REP pattern and reply only after the message has been processed
[11:44] stimpie	I would like a message to stay in the buffer of the send until the receiver has finished processing (stored) the message
[11:45] mikko	stimpie: that sounds more like application level thing
[11:45] mikko	stimpie: i think request-reply would work well for that
[11:47] stimpie	req/rep will be sufficient indeed but I was wondering if it could be done more efficient
[12:33] sustrik	stimpie: you are after guaranteed delivery, right?
[12:33] stimpie	yes
[12:37] sustrik	the standard way to do that in 0mq
[12:37] sustrik	is the requester sends a request then waits for a reply for some time
[12:38] sustrik	if the reply is not delivered untill the time runs out
[12:38] sustrik	it resends the request
[12:39] sustrik	that way you can guarantee the request is processed no matter what SW/HW/network failures occur
[12:59] stimpie	sustrik, I guess the standard way is usually the best way ;-)
[12:59] sustrik	i think so :)
[14:44] Steve-o	lol, someone still signed in with the name stockMQ?
[14:44] mikko	Steve-o: yep
[14:45] Steve-o	Got into NY last night and all snowed in.
[14:46] mikko	i've never been to that coast
[14:46] mikko	only west coast
[14:46] Steve-o	not much snow in san francisco
[14:46] mikko	no, but i understand that seattle is colder?
[14:47] mikko	i've only been to san diego and once driven from LA to SF
[14:48] Steve-o	ok, NY is a lot further south than North Europe for sure
[14:49] Steve-o	it certainly isn't many feet of snow that I used to see
[14:50] sustrik	ah, speaking of SF, I'm going to Bay Area in Feb or so
[14:50] sustrik	maybe it's time for another meetup
[14:50] mikko	hmm i'm going to be in San Diego around end of May
[14:51] sustrik	well, you can do a meetup of your own
[14:51] sustrik	if there's interest from folks down there
[14:52] sustrik	drbobbeaty: are you there?
[14:52] mikko	would be interesting to see where the users are from
[14:52] drbobbeaty	sustrik: yup
[14:52] sustrik	the problem you had with OpenPGM
[14:52] sustrik	Steve-o is online
[14:52] sustrik	he's author of OpenPGM
[14:52] mikko	sustrik: are you interested in the additional tests?
[14:52] mikko	the HWM and identity tests are sitting somewhere in github
[14:53] drbobbeaty	Very nice. Sure, if he's interested.
[14:53] sustrik	mikko: definitely the HWM tests
[14:53] sustrik	the identity is kind of leaky
[14:54] mikko	https://github.com/mkoppanen/zeromq2/blob/build-test/tests/test_hwm.cpp
[14:54] mikko	does that look correct?
[14:54] sustrik	not sure we can have a good tests until we understand how the identity is supposed to work exactly
[14:54] sustrik	let me see
[14:55] sustrik	looks good
[14:55] Steve-o	who made the previous set of Windows MSIs for 0mq?
[14:55] sustrik	would you send that as a patch?
[14:55] mikko	sustrik: will do this evening
[14:55] sustrik	Steve-o: You mean the MSI two years ago?
[14:55] mikko	i'm fairly busy with work stuff for this and next week
[14:55] sustrik	0MQ/0.4 or somesuch
[14:55] sustrik	mikko: sure, no haste
[14:56] Steve-o	I think there is a directory full of MSIs somewhere
[14:56] mikko	Steve-o: on the topic of windows there are win snapshots available now
[14:56] mikko	mingw and msvc http://snapshot.valokuva.org/
[14:56] sustrik	afai am aware there was just one MSI created ~2 years ago by Tamara, an intern we used to have at FastMQ
[14:57] Steve-o	ok, all the old ones here: http://download.zeromq.org/historic/
[14:59] Steve-o	Wondered about making a quick CPack script to get 0MQ + PGM packages wrapped
[14:59] sustrik	mikko: the location of users...
[15:00] sustrik	for 2010 google analytics says: 246,087 visits came from 9,211 cities
[15:00] sustrik	let me upload the map...
[15:01] sustrik	quarter of million visits, not bad...
[15:01] sustrik	here you go:
[15:01] sustrik	http://zeromq.wdfiles.com/local--files/community/map.png
[15:01] Steve-o	I think I messed up the licensing notes for the Windows binary distribution, need to find a nice source to c&p something for the installer confirmation window
[15:02] sustrik	what about LGPL?
[15:02] Steve-o	The Regents of California
[15:02] Steve-o	a bsd advertising clause
[15:03] Steve-o	Yup, for IP header details, http://code.google.com/p/openpgm/source/browse/trunk/openpgm/pgm/include/impl/ip.h
[15:03] Steve-o	"Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution."
[15:04] sustrik	i see
[15:10] Steve-o	mikko: If I give you WIndows PGM installers can you make Hudson use that to build 0MQ+PGM builds?
[15:10] mikko	Steve-o: you mean the msvc ones?
[15:11] Steve-o	yes
[15:11] mikko	do i need to modify the build files ?
[15:12] Steve-o	yes, you need ZMQ_HAVE_OPENPGM defined alongside the additional include & libpath
[15:12] mikko	Steve-o: ok, i think that should be possible
[15:15] Steve-o	I have Windows on this laptop, let me setup a build environment and check that using the static PGM library isn't too problematic for 0MQ first
[15:15] Steve-o	then I can send you the links and try out Hudson on it
[15:17] Steve-o	CPack might also be handy for OSX, the other targets not too sure about, http://www.cmake.org/Wiki/CMake:CPackPackageGenerators
[15:19] sustrik	Steve-o: still in NY?
[15:19] Steve-o	first day
[15:19] sustrik	there must be lots of 0MQ/OpenPGM users there as well
[15:19] sustrik	especially on wall street i suppose
[15:20] sustrik	why not meet them?
[15:21] Steve-o	Honestly no idea who is using it
[15:22] sustrik	try sending an email to the mailing list
[15:22] sustrik	there are ~600 subscribers there
[15:22] Steve-o	Saw an interesting job position at Goldmans looking after their RV, EMS, & MQ infrastructure though
[15:22] sustrik	heh
[15:23] sustrik	i bet couple of the guys on mailing list won't mind meeting for a beer
[15:24] Steve-o	Will give a shout then
[16:50] pieterhintjens	sustrik: ping
[16:50] sustrik	pong
[16:51] pieterh	I stuck that (nice) map on the community page
[16:51] pieterh	changed the colors a little to have more contrast
[16:51] sustrik	great
[16:51] sustrik	where it is linked from?
[16:51] pieterh	is that what you were intending?
[16:52] pieterh	http://www.zeromq.org/community
[16:52] sustrik	not really, i just don't have another place for uploads :)
[16:52] pieterh	ah
[16:53] sustrik	not bad
[16:53] sustrik	what about adding a caption?
[16:53] sustrik	246,087 visits came from 9,211 cities during 2010
[16:53] pieterh	ah, I like the notion that this is the "0MQ Community"
[16:53] pieterh	so page title = caption
[16:54] mikko	do we have "Upcoming Talks" page?
[16:54] mikko	i think there are several talks about 0MQ in different conferences during the spring
[16:55] pieterh	mikko: good idea
[16:55] pieterh	we can IMO just add them to the community page
[16:55] pieterh	top right
[16:56] pieterh	I'll do that and ask folk to add their own talks...
[16:57] mikko	ianbarber is having talk next month
[16:57] mikko	i think there was a couple of others in the list as well
[17:00] mikko	pieterh: what days are you going to be at FOSDEM?
[17:00] ianbarber	i'm speaking at PHP Tek in may in chicago about 0MQ as well
[17:01] pieterh	mikko: 5th Feb
[17:01] pieterh	ianbarber: would you like to edit the community page?
[17:01] pieterh	I've made a section for upcoming events
[17:01] pieterh	order by date, I'd suggest
[17:02] ianbarber	sure!
[17:02] mikko	hmmm
[17:02] mikko	might be possible to take a train from london to brussels on the 5th
[17:04] mikko	Â£34.50 by train
[17:04] pieterh	mikko: sure, if you book in advance it's quite reasonable
[17:05] pieterh	allow time to get from the station to FOSDEM, that is quite far from the center
[17:05] pieterh	I suspect it's 34.50 each way, though
[17:05] mikko	yeah
[17:07] ianbarber	irritatingly i'm helping my brother move on the fifth, otherwise i'd have been keen on going too
[17:07] ianbarber	pieterh: not to be clueless, but how do you edit the page? i'm signed in but am missing the edit link
[17:07] pieterh	it might be at the bottom of the page
[17:08] pieterh	otherwise, press Ctrl+E
[17:08] mikko	not enough rights?
[17:08] sustrik	maybe press the "join wiki" button first
[17:08] ianbarber	ah, yeah , looks like it
[17:08] ianbarber	Sorry, you can not edit this page. Only owner (creator) of this page, site administrators and perhaps selected moderators are allowed to do it.
[17:08] pieterh	Let me fix permissions to show the page actions to all members
[17:08] pieterh	hang on...
[17:09] pieterh	hmm
[17:09] pieterh	ianbarber: try reloading, you will have to have joined the site
[17:10] ianbarber	yep, that's it, they're there now
[17:10] pieterh	it's because this page is '/community' and thus in the default category, which had strict permissions... normally the wiki pages are in categories like doc: and binding: which had open permissions
[18:25] stockMQ	Hi..anyone tested pgm/epgm on localhost..?
[21:31] mikko	sustrik: interesting, i increased socket buffers and recv/send buffers on freebsd and shutdown stress still fails
[22:20] drbobbeaty	Steve-o: I have a question about the OpenPGM UDP receive buffer sizes. One of my users is looking at the UDP receive buffers and they are seeing a size of 128kB and they say it appears to never empty. Having not seen this myself, I'm asking you if this is normal behavior for OpenPGM? Specifically, they are worried that the buffer is full and we're dropping messages. I'm guessing if we suffer unrecoverable data loss in ZMQ, we'd get an exception or somet
[22:20] drbobbeaty	on the recv().
[22:21] drbobbeaty	Steve-o: I'll send this question to the mailing list in case you aren't online.