Thursday January 27, 2011

[Time] NameMessage
[00:19] stodge I just started reading about zeromq, so my quick question is, can it be used to create a jgroups-like technology?
[00:19] stodge I'm reading about publish/suscribe
[00:20] stodge Wondering if it could work as a heartbeat mechanism between an unknown number of peers (talking less than 100)
[00:20] stodge Reading to see if zeromq could be used for peer discovery
[00:40] lt_schmidt_jr why not use zookeeper for something like that?
[00:45] lt_schmidt_jr comment aimed at stodge
[00:45] stodge I haven't read about zookeeper yet - it's on my list. My system is primarily (95%) Python so I jumped at zeromq when I saw the Python API.
[00:45] stodge Thanks
[00:47] lt_schmidt_jr stodge, NP, I believe there is a python binding. not that you should not use zeromq, but I think zookeeper is probably better suited for this particular task (and simpler)
[00:48] stodge ok Thanks, I'll definitely read up on zookeeper
[00:48] lt_schmidt_jr simpler for the task - not in general
[01:09] stockMQ Hi.. Can anyone confirm that pgm multicast will work on Windows and not restricted to LINUX
[04:48] stockMQ Anyone doing multicast on windows?
[04:51] jugg
[04:59] stockMQ I see there were some issues in the past for support of pgm for windows
[05:00] stockMQ hence just would like to know if it is supported now before i get into using it
[05:06] stockMQ
[05:07] stockMQ I am on windows and have a similar requirement to do multicast using PUB-SUB
[05:07] stockMQ does this problem still persist??
[05:43] stockMQ anyone?
[07:26] sustrik mikko: morning
[07:53] mikko sustrik: hi
[07:54] mikko sustrik: what is the policy on constructor errors ? i noticed that there arent really throws
[07:57] sustrik mikko: the exceptions are not used at all
[07:57] sustrik when you need an error in constructor
[07:57] sustrik you separate the construction into 2 phases
[07:57] sustrik 1. the part that cannot fail (constructor)
[07:57] sustrik 2. the part that can fail (init function)
[07:58] sustrik the latter can return error
[08:08] stockMQ
[08:08] stockMQ does this problem still persist??
[08:08] stockMQ I am on windows and have a similar requirement to do multicast using PUB-SUB
[08:12] sustrik stockMQ: i have no idea what's the status quo wrt OpenPGM on Windows
[08:12] sustrik there's definitely no MSVC intrgrated build
[08:12] sustrik but you may try MinGW
[08:13] stockMQ ok..Please confirm my understanding
[08:13] stockMQ to multicast two options available are pgm,epgm
[08:14] stockMQ But there does not seem to be a support for windows and maybe a minGW port could only be the option
[08:14] stockMQ Is that right?
[08:18] stockMQ ??
[08:47] sustrik stockMQ: it should work on Windows
[08:47] sustrik the problem is that the build system is not at all obvious
[08:47] sustrik so, it's hard to build, but once you build it, it should work :)
[09:01] stockMQ okk.:).. Any ideas if i can test the pgm multicast on the same host.. meaning both pub and sub being on the same machine
[09:03] sustrik yes, you can
[09:03] sustrik but it's actually a simulation rather than real multicast then
[09:03] sustrik different code is used
[09:03] sustrik so if you want to test the real multicast code, use it over the network
[09:08] stockMQ yes.. i will be doing the real testing but i dont have access to the network full was thinking of testing it on the simulation and then go ahead
[09:09] stockMQ actually the loopback would do it right
[09:12] guido_g hmm... i don't think that the loopback device is multicast capable, wouldn't make sense
[09:12] guido_g you need to use the external ip of the box
[09:12] guido_g adn this leads to a lot of free floating traffic in the network
[09:13] mikko sustrik: hmmm, thats not exception safe (?)
[09:15] stockMQ publisher->bind(" pgm://;")
[09:15] stockMQ the first IP is the NIC i want to bind to
[09:15] stockMQ and the second one is the multicast group IP
[09:15] stockMQ now how would this change if i wanted to test it on localhost
[09:15] stockMQ Or will this not work on Windows altogether
[09:29] stockMQ guido?
[09:31] guido_g hm?
[09:31] stockMQ any insights/
[09:32] stockMQ ?
[09:32] guido_g into what?
[09:35] stockMQ publisher->bind(" pgm://;")
[09:35] stockMQ Now how would this change for for localhost
[09:35] stockMQ I believe the first IP is the NIC i want to bind to
[09:35] stockMQ and second one is the multicast group ip
[09:35] guido_g did you ever read the ømq docs?
[09:36] stockMQ I read the guide
[09:36] guido_g there is more
[09:37] stockMQ
[09:39] stockMQ
[09:39] stockMQ this says first IP is the network interface
[09:40] stockMQ that is what i think i mentioned..
[09:57] sustrik mikko: what do you mean by exception-safe?
[09:58] sustrik there are no exceptions used in the codebase
[10:38] mikko sustrik: ok
[10:38] mikko sustrik: i was thinking about situations where the initialisation method might throw
[10:38] mikko but i guess if there is no exceptions that is not so much an issue
[10:39] sustrik exactly
[10:39] sustrik exceptions are good for programs with low reliabity requirement
[10:40] sustrik say: "if something bad happens show a dialog box"
[10:41] sustrik for systems that should run 24/7 without user intervention the errors should not be handled in generic manner (the practice encouraged by exceptions)
[10:41] sustrik rather on one-by-one basis
[11:36] stimpie Is it possible to acknowledge a message after it has been processed (not when received)?
[11:44] mikko stimpie: what do you mean?
[11:44] mikko stimpie: you would probably want to use REQ/REP pattern and reply only after the message has been processed
[11:44] stimpie I would like a message to stay in the buffer of the send until the receiver has finished processing (stored) the message
[11:45] mikko stimpie: that sounds more like application level thing
[11:45] mikko stimpie: i think request-reply would work well for that
[11:47] stimpie req/rep will be sufficient indeed but I was wondering if it could be done more efficient
[12:33] sustrik stimpie: you are after guaranteed delivery, right?
[12:33] stimpie yes
[12:37] sustrik the standard way to do that in 0mq
[12:37] sustrik is the requester sends a request then waits for a reply for some time
[12:38] sustrik if the reply is not delivered untill the time runs out
[12:38] sustrik it resends the request
[12:39] sustrik that way you can guarantee the request is processed no matter what SW/HW/network failures occur
[12:59] stimpie sustrik, I guess the standard way is usually the best way ;-)
[12:59] sustrik i think so :)
[14:44] Steve-o lol, someone still signed in with the name stockMQ?
[14:44] mikko Steve-o: yep
[14:45] Steve-o Got into NY last night and all snowed in.
[14:46] mikko i've never been to that coast
[14:46] mikko only west coast
[14:46] Steve-o not much snow in san francisco
[14:46] mikko no, but i understand that seattle is colder?
[14:47] mikko i've only been to san diego and once driven from LA to SF
[14:48] Steve-o ok, NY is a lot further south than North Europe for sure
[14:49] Steve-o it certainly isn't many feet of snow that I used to see
[14:50] sustrik ah, speaking of SF, I'm going to Bay Area in Feb or so
[14:50] sustrik maybe it's time for another meetup
[14:50] mikko hmm i'm going to be in San Diego around end of May
[14:51] sustrik well, you can do a meetup of your own
[14:51] sustrik if there's interest from folks down there
[14:52] sustrik drbobbeaty: are you there?
[14:52] mikko would be interesting to see where the users are from
[14:52] drbobbeaty sustrik: yup
[14:52] sustrik the problem you had with OpenPGM
[14:52] sustrik Steve-o is online
[14:52] sustrik he's author of OpenPGM
[14:52] mikko sustrik: are you interested in the additional tests?
[14:52] mikko the HWM and identity tests are sitting somewhere in github
[14:53] drbobbeaty Very nice. Sure, if he's interested.
[14:53] sustrik mikko: definitely the HWM tests
[14:53] sustrik the identity is kind of leaky
[14:54] mikko
[14:54] mikko does that look correct?
[14:54] sustrik not sure we can have a good tests until we understand how the identity is supposed to work exactly
[14:54] sustrik let me see
[14:55] sustrik looks good
[14:55] Steve-o who made the previous set of Windows MSIs for 0mq?
[14:55] sustrik would you send that as a patch?
[14:55] mikko sustrik: will do this evening
[14:55] sustrik Steve-o: You mean the MSI two years ago?
[14:55] mikko i'm fairly busy with work stuff for this and next week
[14:55] sustrik 0MQ/0.4 or somesuch
[14:55] sustrik mikko: sure, no haste
[14:56] Steve-o I think there is a directory full of MSIs somewhere
[14:56] mikko Steve-o: on the topic of windows there are win snapshots available now
[14:56] mikko mingw and msvc
[14:56] sustrik afai am aware there was just one MSI created ~2 years ago by Tamara, an intern we used to have at FastMQ
[14:57] Steve-o ok, all the old ones here:
[14:59] Steve-o Wondered about making a quick CPack script to get 0MQ + PGM packages wrapped
[14:59] sustrik mikko: the location of users...
[15:00] sustrik for 2010 google analytics says: 246,087 visits came from 9,211 cities
[15:00] sustrik let me upload the map...
[15:01] sustrik quarter of million visits, not bad...
[15:01] sustrik here you go:
[15:01] sustrik
[15:01] Steve-o I think I messed up the licensing notes for the Windows binary distribution, need to find a nice source to c&p something for the installer confirmation window
[15:02] sustrik what about LGPL?
[15:02] Steve-o The Regents of California
[15:02] Steve-o a bsd advertising clause
[15:03] Steve-o Yup, for IP header details,
[15:03] Steve-o "Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution."
[15:04] sustrik i see
[15:10] Steve-o mikko: If I give you WIndows PGM installers can you make Hudson use that to build 0MQ+PGM builds?
[15:10] mikko Steve-o: you mean the msvc ones?
[15:11] Steve-o yes
[15:11] mikko do i need to modify the build files ?
[15:12] Steve-o yes, you need ZMQ_HAVE_OPENPGM defined alongside the additional include & libpath
[15:12] mikko Steve-o: ok, i think that should be possible
[15:15] Steve-o I have Windows on this laptop, let me setup a build environment and check that using the static PGM library isn't too problematic for 0MQ first
[15:15] Steve-o then I can send you the links and try out Hudson on it
[15:17] Steve-o CPack might also be handy for OSX, the other targets not too sure about,
[15:19] sustrik Steve-o: still in NY?
[15:19] Steve-o first day
[15:19] sustrik there must be lots of 0MQ/OpenPGM users there as well
[15:19] sustrik especially on wall street i suppose
[15:20] sustrik why not meet them?
[15:21] Steve-o Honestly no idea who is using it
[15:22] sustrik try sending an email to the mailing list
[15:22] sustrik there are ~600 subscribers there
[15:22] Steve-o Saw an interesting job position at Goldmans looking after their RV, EMS, & MQ infrastructure though
[15:22] sustrik heh
[15:23] sustrik i bet couple of the guys on mailing list won't mind meeting for a beer
[15:24] Steve-o Will give a shout then
[16:50] pieterhintjens sustrik: ping
[16:50] sustrik pong
[16:51] pieterh I stuck that (nice) map on the community page
[16:51] pieterh changed the colors a little to have more contrast
[16:51] sustrik great
[16:51] sustrik where it is linked from?
[16:51] pieterh is that what you were intending?
[16:52] pieterh
[16:52] sustrik not really, i just don't have another place for uploads :)
[16:52] pieterh ah
[16:53] sustrik not bad
[16:53] sustrik what about adding a caption?
[16:53] sustrik 246,087 visits came from 9,211 cities during 2010
[16:53] pieterh ah, I like the notion that this is the "0MQ Community"
[16:53] pieterh so page title = caption
[16:54] mikko do we have "Upcoming Talks" page?
[16:54] mikko i think there are several talks about 0MQ in different conferences during the spring
[16:55] pieterh mikko: good idea
[16:55] pieterh we can IMO just add them to the community page
[16:55] pieterh top right
[16:56] pieterh I'll do that and ask folk to add their own talks...
[16:57] mikko ianbarber is having talk next month
[16:57] mikko i think there was a couple of others in the list as well
[17:00] mikko pieterh: what days are you going to be at FOSDEM?
[17:00] ianbarber i'm speaking at PHP Tek in may in chicago about 0MQ as well
[17:01] pieterh mikko: 5th Feb
[17:01] pieterh ianbarber: would you like to edit the community page?
[17:01] pieterh I've made a section for upcoming events
[17:01] pieterh order by date, I'd suggest
[17:02] ianbarber sure!
[17:02] mikko hmmm
[17:02] mikko might be possible to take a train from london to brussels on the 5th
[17:04] mikko £34.50 by train
[17:04] pieterh mikko: sure, if you book in advance it's quite reasonable
[17:05] pieterh allow time to get from the station to FOSDEM, that is quite far from the center
[17:05] pieterh I suspect it's 34.50 each way, though
[17:05] mikko yeah
[17:07] ianbarber irritatingly i'm helping my brother move on the fifth, otherwise i'd have been keen on going too
[17:07] ianbarber pieterh: not to be clueless, but how do you edit the page? i'm signed in but am missing the edit link
[17:07] pieterh it might be at the bottom of the page
[17:08] pieterh otherwise, press Ctrl+E
[17:08] mikko not enough rights?
[17:08] sustrik maybe press the "join wiki" button first
[17:08] ianbarber ah, yeah , looks like it
[17:08] ianbarber Sorry, you can not edit this page. Only owner (creator) of this page, site administrators and perhaps selected moderators are allowed to do it.
[17:08] pieterh Let me fix permissions to show the page actions to all members
[17:08] pieterh hang on...
[17:09] pieterh hmm
[17:09] pieterh ianbarber: try reloading, you will have to have joined the site
[17:10] ianbarber yep, that's it, they're there now
[17:10] pieterh it's because this page is '/community' and thus in the default category, which had strict permissions... normally the wiki pages are in categories like doc: and binding: which had open permissions
[18:25] stockMQ Hi..anyone tested pgm/epgm on localhost..?
[21:31] mikko sustrik: interesting, i increased socket buffers and recv/send buffers on freebsd and shutdown stress still fails
[22:20] drbobbeaty Steve-o: I have a question about the OpenPGM UDP receive buffer sizes. One of my users is looking at the UDP receive buffers and they are seeing a size of 128kB and they say it appears to never empty. Having not seen this myself, I'm asking you if this is normal behavior for OpenPGM? Specifically, they are worried that the buffer is full and we're dropping messages. I'm guessing if we suffer unrecoverable data loss in ZMQ, we'd get an exception or somet
[22:20] drbobbeaty on the recv().
[22:21] drbobbeaty Steve-o: I'll send this question to the mailing list in case you aren't online.