Wednesday July 13, 2011

[Time] NameMessage
[00:00] CIA-32 pyzmq: 03Stefan van der Walt 07master * ra6d9846 10/ zmq/core/context.pyx : Do not attempt to acquire the GIL when it is already held. -
[02:01] gdan if you are using the push pull fan-out, does only one of the pull sockets receive a message per message pushed
[02:03] whack gdan: I think that's the idea, yeah
[02:05] gdan i wonder with a lot of nodes to push to... if you use tcp, the push socket will choose one path to send to... thus there is no bandwidth saving using tcp over pgm
[02:07] whack well with push only one receiver gets each message; so multicast vs tcp shouldn't matter
[02:07] whack if you have 10 recievers, 1 message, 1 receiver gets it
[02:10] gdan thanks, I appreciate. i thought it was something like that reading the guide but it wasn't spelled out as that
[02:10] gdan as clearly as that...
[06:01] c_nick Can someone help me with this
[10:31] omarkj Is exposing zeromq to the internet safe?
[10:32] omarkj I remember there being an issue with that?
[11:54] guido_g not using pair sockets in the first place
[11:56] guido_g there're a lot of things "wrong" with them
[11:56] guido_g not a usage pattern, no re-connect and so on
[11:57] guido_g <- there is even a big warning
[11:58] guido_g alternatives depend on what you want to achieve
[12:00] guido_g case closed :)
[12:00] guido_g don't miss the guide! explains a lot, really
[12:08] reuben omg
[12:09] reuben theY4Kman`lab!
[12:10] reuben didn't know you were using 0MQ :)
[12:10] reuben haha
[12:10] reuben yes it is :)
[12:11] reuben I'm gonna do a talk about it on PyCon Brasil later this year
[12:13] reuben I'm in the process of convincing the people at work to use 0MQ
[12:13] reuben writing a groovy binding, showing cool examples, etc
[12:14] reuben
[12:14] reuben just mapping technology they're already familiar with
[12:16] reuben theY4Kman`lab, is that for the kinect minecraft generation thing?
[12:18] reuben cool
[12:25] reuben yeah that's groovy
[12:26] reuben whoa, cool stuff
[12:26] reuben I didn't know about SIGGRAPH
[12:36] kontinuity hey folks
[12:37] kontinuity we were looking for a xrep/xreq client server example
[12:37] kontinuity without a dealer router
[12:37] kontinuity is that pattern possible?
[12:48] reuben kontinuity, yes
[12:48] reuben
[12:49] reuben the weird messages are the metadata needed for XREQ/XREP stuff
[12:49] reuben AFAIK
[12:50] kontinuity reuben: probably the routing data
[12:50] kontinuity :)
[12:50] guido_g it's the socket id as assumed by the receiver
[12:50] reuben yeah
[12:50] guido_g yes, for routing purposes
[12:50] kontinuity guido_g: how do know the valid messages from them?
[12:51] guido_g pardon?
[12:51] reuben if it doesn't start with 0x00?
[12:51] kontinuity guido_g: in reuben's example he did a standard recv
[12:51] kontinuity guido_g: how do we know its the socket or real data
[12:52] guido_g it's written in the xrep/xreq spec and also discussed in detail in the guide
[12:53] kontinuity guido_g: thanks
[12:56] kontinuity reuben: the example looks more like PUSH-PULL
[12:56] guido_g btw, i'm somwhat shocked that the example above works
[12:56] kontinuity guido_g: precisely :)
[12:56] reuben why?
[12:56] kontinuity guido_g: but not that it works or not but rather where is the async rep in the example?
[12:56] guido_g should be one multipart message
[12:57] guido_g ah forget it, my fault
[13:04] kontinuity guido_g: how do we use timeouts with rep/req?
[13:05] guido_g you don't
[13:05] guido_g you've to use poll for that
[13:15] kontinuity guido_g: using poll now thanks. In case of a req for which we may have no rep how do skip and go to next
[13:15] kontinuity ?
[13:15] reuben guido_g, it *is* a multipart message
[13:15] reuben :)
[13:16] guido_g yep
[13:16] reuben kontinuity, I don't think you can, but can't you just send an empty message?
[13:16] guido_g forgot that the plain recv fetches whatever is there
[13:16] guido_g damn wrappers! :)
[13:17] kontinuity reuben: in our case the server may have bounced and the client will wait for something that the server has lost reference to
[13:17] reuben hm
[13:18] kontinuity reuben: should the client reinit the connection?
[13:19] reuben I don't know, that's probably what I would do in this case
[13:20] guido_g <- there might be some more info
[13:22] kontinuity guido_g: saw it superficially but we don't want to introduce a dealer in between
[13:22] kontinuity guido_g: just use req-rep or xreq/xrep with recovery from failed reps
[13:23] kontinuity guido_g: its a fairly simple client - server with ack problem we have ;)
[13:23] kontinuity guido_g: and missing acks as I mentioned above
[13:33] stodge I'm trying to get epgm working with pyzmq. I installed zeromq 3.0 from source using ran ./configure --with-pgm. This built and installed ok. I then did easy_install pyzmq and that installed. But my simple test app tells me that epgm isn't supported. Did I miss a step?
[13:38] guido_g might be a problem w/ ømq3 vs. pyzmq
[13:38] guido_g if you don't need the 3.0 features, try to use the latest stable
[13:49] stodge I did - it didn't work but let me try the stable release again
[13:51] guido_g you should try the latest pyzmq from github and build from source
[13:51] stodge I tried that but it didnt build for me
[13:51] stodge I cloned the repo, instead of using a download
[13:52] stodge This is on Fedora 14 64bit
[13:52] stodge With zeromq stable and latest pyzqm through easy_install I get "protocol not supported"
[13:52] stodge Let me try to build pyzqm from source again
[13:52] stodge Bah - my typing/spelling is off
[13:55] stodge Sorry - pyzqm did build from source, but I get a runtime error undefined symbol zmq_sendmsg
[13:55] reuben stodge, grab 0MQ 3.0, build it with PGM enabled, grab pyzmq from github, build it with PGM enabled too
[13:56] reuben or 0MQ stable and pyzmq stable if you need PGM on 2.1.7
[13:56] reuben 0MQ builds with PGM disabled by default
[13:57] stodge Building pyzmq is done with python build ?
[13:58] reuben just do python install
[13:58] reuben it'll build and install
[13:59] stodge How do I specify "with pgm"?
[13:59] reuben --with-openpgm IIRC
[13:59] reuben let me check
[13:59] reuben --with-pgm
[13:59] reuben stodge, ^
[14:00] reuben or --with-system-pgm if you want to use openpgm from the package management
[14:00] stodge Right when building zeromq, but python install doesn't accept --with-pgm
[14:00] stodge Or am I just being incredibly dense today?
[14:00] reuben you shouldn't need anything special to have PGM on pyzmq
[14:00] reuben as long as 0MQ has it enabled
[14:01] stodge Ok let me build stable zeromq and pyzmq from source
[14:05] stodge Still get protocol not supported
[14:05] guido_g please show the code you're using
[14:06] stodge I'll have to add it to my issue on the pyzmq github site - I'm using different machines for IRC and dev
[14:06] guido_g jsut use a paste bin
[14:07] stodge ok
[14:07] guido_g ahh, read my comment on your issue
[14:07] reuben stodge, I just built it here and it worked fine
[14:07] guido_g you're using a wrong address for multicast
[14:07] reuben I didn't even have to rebuild pyzmq
[14:08] guido_g ack
[14:08] stodge I put the multicast address in instead of localhost. I used to have the mc address but for some reason took it out
[14:08] guido_g it's nevertheless wrong
[14:08] stodge
[14:08] guido_g and try to avoid interface names, that's also somewhat flaky
[14:09] reuben
[14:09] guido_g copy&paste works here
[14:09] reuben I had pyzmq installed with |sudo pip install pyzmq| before
[14:10] stodge hmmm
[14:10] stodge zeromq installs to /usr/local/lib? I better make sure I don't have multiple copies of zeromq lying around
[14:10] guido_g omg
[14:10] reuben everything defaults to /usr/local
[14:10] reuben lol
[14:11] stodge I don't thikn I have multiple copies - I can only find the one I built
[14:14] reuben <- make sure you do it like that
[14:14] reuben uninstall the versions you already have before doing that
[14:15] stodge ok
[14:20] stodge I don't have pip installed but easy_install should work
[14:21] stodge Oh - when I do easy_install pyzmq I see a message that says that ZMQ 2.1.4 was detected. I don't have that version installed anywhere
[14:23] stodge Crap - I yum removed all zmq stuff except I forgot to remove the zeromq package itself... :P
[14:24] guido_g *sigh*
[14:24] reuben haha
[14:24] stodge I need more caffeine
[14:24] reuben stodge, if you don't have pip, here's another useful command for you:
[14:24] reuben sudo easy_install pip
[14:24] kontinuity guido_g: is there any documentation on what each socket type does? I am seeing examples like REQ - XREP | REP - REP | REQ - REQ
[14:24] kontinuity totally confused
[14:25] kontinuity XREQ - ZREQ*
[14:25] kontinuity XREQ - XREQ**
[14:25] reuben XREP and XREQ are routers and dealers, they can interoperate with most socket types
[14:25] stodge Of course it works now *hides in corner in shame*
[14:25] stodge Thanks
[14:25] guido_g there is the reference and the guide
[14:25] guido_g that should do
[14:25] reuben yes
[14:25] guido_g but you have to read it
[14:26] reuben you should *really* read ze guide
[14:26] stodge Now to get the consumer to read correctly...
[14:27] reuben it's kinda hard to do that, pieter did an amazing job :)
[14:34] stodge Are there any example of pyzmq & pgm?
[14:36] guido_g not really
[14:37] guido_g but there are close to no examples for (e)pgm anyway
[14:37] stodge :)
[14:37] stodge receiver isn't receiving anything
[14:38] guido_g check multicast routing and mc loopback flag
[14:38] guido_g the later only if sender and receiver are on the same machine
[14:39] stodge Ah it is the same machine
[14:41] reuben it worked just fine for me, on the same machine
[14:42] reuben
[14:42] reuben :P
[14:42] guido_g yeah, default is "do loopback"
[14:43] guido_g i guess the subscribe is missing or the the subscriber was activated to late
[14:45] stodge I'm still trying to find out how to change that flag - you guys are fast :)
[14:45] guido_g setsockopt
[14:52] stodge That pastie code doesn't work for me - the code sits there doing nothing. I'll have to check my network config
[15:12] stodge membership join requests are being sent but no data is received. Ah well; need to move onto something else for the day. Thanks
[15:19] pieter_hintjens sustrik: ping
[15:20] pieter_hintjens any chance of backporting to 2.1?
[15:32] pieter_hintjens ok, done it myself
[16:29] ssi is there a significant performance difference between sending a 2 part x byte per part message vs a 1 part 2x byte message?
[16:37] mikko ssi: the latter might be slightly faster
[16:55] cremes ssi: one less iteration of the loop to read in a message part
[17:01] rev does zmq 3.0 support inproc for windows? i googled and got nothing and there is no mention in the change log. i noticed new msvc project files that has me curious.
[17:10] ssi ugh these patterns get out of control in a damned hurry
[17:44] rev it's awfully quiet in here.
[17:44] ssi it usually is :)
[17:45] rev perhaps you can answer my question then
[17:45] ssi I wasn't aware there was a 3.0, tbh
[17:45] ssi and I haven't touched zmq on windows at all
[17:45] rev oh hehe
[17:45] rev that explains the silence
[17:45] rev hehe
[17:45] ssi 3.0 happened yesterday :D
[17:46] rev yeah
[17:46] ssi if you're that new, you probably should run with 2.1.7 for now
[17:46] rev thats why i thought it would be more chatty in here
[17:46] rev i am running 2.1.7
[17:46] rev i'm trying to decide if the upgrade is worth the effort
[17:46] ssi beats me! :D
[17:46] rev normally i'd let the dust settle
[17:47] rev it looks like they sneaked inproc in for windows ... surprised me tho -
[17:47] ssi 2.x doesn't support inproc in windows?
[17:47] rev nope
[17:48] ssi hooboy
[17:48] rev something about "it's complicated"
[17:48] rev hehe
[17:48] ssi that's gonna cause me some issues :D
[17:48] rev bazinga
[17:48] ssi the first thingy that I wrote relies entirely on inproc
[17:48] ssi and I dev on OSX
[17:48] ssi but I'm gonna have to make it so that our worthless windows developers can use it
[17:48] ssi that's gonna make my life hard
[17:49] rev you're going to have to use sockets on localhost
[17:49] rev it works really well
[17:49] rev but its not inproc
[17:49] ssi I wonder if I can sense and switch internally
[17:49] ssi cause it'll all be deployed on linux, and I want the performance
[17:49] rev probably
[17:50] rev well, hehe - i came in looking for help and helped someone else. go figure
[17:51] harris hi guys!
[17:51] harris is there any way to do fail over in zmq ?
[17:51] ssi rev: that's how it goes!
[17:51] ssi harris: it's all in the guide! (just warning you cause that's what martin and pieter will say)
[17:52] harris ssi: thanks I will dig in then
[17:59] panu Hi. Anybody knows any know reason why zmq_poll might not alter the poll item lista at all (I am using 0mq version 2.1.7)?
[18:24] pieter_hintjens panu: either the timeout expired, or there was an error, or there was an interrupt
[18:24] pieter_hintjens check for rc != 0 and then check errno
[18:50] panu pieter_hintjens: poll returns positive value..
[18:51] panu I am writing bindings from scratch so this is most likely problem in my code
[18:52] pieter_hintjens panu: interesting, and revents is zero on all items?
[18:52] panu No... revents and events seems to be unchanged.
[18:58] ssi argh... assertions D:
[18:58] ssi Assertion failed: (exception_class), function raise_exception, file util.cpp, line 33.
[18:58] ssi this one's in jzmq it seems
[18:58] ssi jclass exception_class = env->FindClass ("org/zeromq/ZMQException");
[18:58] ssi assert (exception_class);
[19:19] panu If the socket pointer passed to poll in polled item structure is rubbish should 0mq detect this or will it just behave odd?
[21:34] pauldix hi all. wondering if anyone can help. I'm working with a XREQ and REP socket combo in Ruby and the REP isn't reading correctly
[21:34] pauldix code and output here:
[21:35] pauldix please to help
[21:35] pauldix <3