Wednesday September 28, 2011

[Time] NameMessage
[00:06] dcolish is it an ability of the ROUTER/XREP socket type to be able to respond directly to REQ socket messages?
[00:10] dcolish i'm not seeing that happen :/
[00:11] dcolish
[03:41] dublisk Hi, I'm trying to improve my knowledge and gain some experience with low latency socket programming. Do you think learning about 0MQ and possibly trying to contribute code would be a good direction?
[05:50] sustrik dublisk: yes
[05:50] sustrik there are no much other open source options that do this
[05:51] sustrik you can have a look at OpenPGM too, but multicast is order of magnitude more complex than TCP
[05:52] sustrik dcolish: with REQ socket you have to send a request first, then get a reply
[05:52] sustrik you are doing it other way round
[06:04] CIA-79 libzmq: 03Martin Sustrik 07master * r8485a5e 10/ (src/router.cpp src/xrep.cpp): Assert fixed in XREP & ROUTER when sending to terminating pipe (issue 258) ...
[13:34] pieterh hi folks
[13:35] alfred1 hi piet
[13:35] pieterh hi alfred1
[14:03] shales I'm writing some unittests. How can I setup a socket so that a send blocks or returns EAGAIN? Right now I have two connected XREQ sockets, A and B, and have set low HWM and SNDBUF on skt A. I send 1000 small messages on A and don't recv on B, but it doesn't block. Should it?
[14:06] dcolish sustrik: what do you mean?
[14:07] dcolish in that pasted example, i combined two distinct files
[14:12] guido_g howdy guys
[14:13] guido_g pieterh: you wanted something a few days ago?
[14:41] dcolish ah i see, the python bindings use send_mulitpart/recv_mulitpart when using a ROUTER socket
[17:38] sundbp cremes: fyi I forked ffi-rzmq and made a few changes. EINTR was marked as not being used in the v2 api. pretty sure it is.
[17:38] cremes sundbp: cool
[17:38] cremes btw, i am probably going to rip out *all* of the exceptions stuff and just go back
[17:38] cremes to pure return codes
[17:39] cremes it's much simpler
[17:39] cremes though it will definitely break the existing ruby api
[17:39] cremes thoughts?
[17:40] sundbp cremes: fine by me. i just started plauying and from reading the api doc on then needed to read through how the exceptions were created etc to know what to catch to handle things etc
[17:41] cremes i fear it may piss off a bunch of folks who have a lot of code tied to the old api... oh well,
[17:41] cremes they don't *have to* upgrade :)
[17:41] sundbp cremes: always the case
[17:42] sundbp cremes: i think it's good to have a simple wrapper around the C api as a base. may not be the most ruby idiomatic but makes it simple to translate from the guide etc. then one can wrap that up nicely in a layer on top to be more idiomatic
[17:42] cremes agreed
[17:43] sundbp cremes: can be part of the same gem, just two ways of viewing the world
[17:43] cremes yes
[17:46] whack sundbp: +1
[17:48] sundbp cremes: for example i'm implementing the majordomo part of the zguide and on using socket.recvmsgs i need to figure out what exceptions to catch by looking at the code in socket.rb and util.rb (for error_check)
[17:49] cremes right... it's just too confusing
[17:49] cremes just using the rc along with zmq_errno() would be simpler
[17:49] sundbp cremes: would be great if i could use return codes and errno in my first pass just as C api, and on 2nd pass update to a high level api, or wrap up my own low level stuff into a high level api suiting the domain
[17:50] cremes sundbp: you are inspiring me to work on that today...
[17:50] cremes i need to grab lunch but i may poke at this afterwards
[17:51] sundbp cremes: you're welcome :) so my wishlist is a "carbon" copy C api in one namespace, and then later a more high level api in a different namespace
[17:56] mikko sustrik: there?
[18:07] pieterh guido_g: hi...
[18:07] guido_g hi pieterh_
[18:08] guido_g you tried to contact me a few days ago
[18:08] pieterh I did... it was a random question
[18:15] mikko finally ==20108== ERROR SUMMARY: 0 errors from 0 contexts
[18:15] mikko it is very hard to get shutdown order right with C++ and shared ptrs
[18:16] pieterh mikko: there's a persistent issue on Jenkins with czmq binding
[18:17] mikko looks like a lot og builds stuck
[18:17] pieterh hmm
[18:18] mikko tests getting stuck
[19:02] guido_g wb pieterh
[19:09] shales I'm trying to force a zmq_send to block for a unit test. I have two connected XREQ sockets. I set HWM and SNDBUF to low numbers on one socket and send 10000 small messages through it without calling recv on the other socket. Should this block?
[19:10] minrk no
[19:10] guido_g depends on the size of the os-level buffers
[19:10] guido_g on *both* sides
[19:12] shales so is setting RCVBUF on the receiving socket enough?
[19:13] michelp shales, do you want to test zmq_send, or test the thing that calls it? if the latter, then I would suggest mocking the zmq_send function instead of trying to force it to block by setting up an unrealistic configuration
[19:21] shales michelp: I'm testing an integration of zeromq with another eventloop, so I'd like to have zmq_send actually return non-zero and set errno to EAGAIN. Trying to ensure I don't miss events on the sockets events FD. I don't think a mock zmq_send would work
[19:24] guido_g then you have to set the os-level buffers to a very small size
[19:24] guido_g which might be prevented by the os
[19:54] staticfloat Hey guys, I'm having trouble using a ROUTER socket; I'm trying to use custom routing messages at the beginning of the my frame to route packets, and I think the router is dropping them for some reason
[19:55] staticfloat I'm using zmq 3.0
[19:55] staticfloat Does anyone know if I need to use LABELs instead of just normal message parts to differentiate the address of the peer I want to talk to from the data I want to send to it?
[19:55] minrk ROUTERs do not use LABELs in 3.0
[19:56] staticfloat Alright. Let me run some code past you to see if this is the right idea then;
[20:00] michelp shales, I wonder if it makes sense to have some kind of "testing transport" where you can get normal and exceptional behavior out of the api based on how you configure it. just a thought
[20:00] michelp obviously it would require some more work than maybe what you were looking for
[20:06] jond minrk: is that 4.0 multiplexer mentioned on the list up on github anywhere?
[20:06] minrk not atm, but I can gist it quickly
[20:07] minrk I haven't cleaned it up for the public yet - I will put it in the pyzmq examples once I do
[20:07] staticfloat @minrk: Alright, here's the relevant code fragment;
[20:07] jond minrk that would be great.
[20:09] staticfloat Basically, the REP socket never gets anything in, and when I sniff via wireshark, I don't get anything being sent except tcp connection setup and teardown
[20:09] minrk jond:
[20:09] jond minrk: thx
[20:10] minrk staticfloat_: set identity before bind
[20:11] minrk jond: backend sockets need to participate in a quick handshake:
[20:11] minrk s.connect(addr)
[20:12] minrk s.recv_multipart() # check HAND, etc.
[20:12] minrk s.send_multipart([SHAKE, identity])
[20:13] jond minrk: noted, thx again
[20:25] jond minrk: wasnt the old model two xrep sockets in a queue with prefix flipping?
[20:25] minrk yes
[20:26] jond so I was expecting this example to be two router sockets?
[20:26] minrk You could certainly make it that way
[20:27] minrk but my use case is really multiplexed request-reply
[20:27] minrk everything coming from downstream is in reply to something upstream
[20:28] jond minrk: so xrep for the request/rep and the 'worker's talk to router
[20:28] minrk the only need to use a ROUTER socket instead of XREP is if you want to send messages to a peer without the prefix information an XREP socket gets on a request
[20:29] minrk yes, it's XREQ->XREP+ROUTER<-XREP
[20:29] minrk workers are XREP, and talk to the ROUTER
[20:29] minrk clients are XREQ, and talk to XREP
[20:30] jond minrk: it's certainly a bit more cumbersome than the old way ;-)
[20:31] minrk it is
[20:33] jond minrk: i'm gonna have a look at the 4.0 src and see what's going on. thx for that gist i'm going to have a more detailed look tomorrow, seems funny the way 'cmd' has to be handled.
[20:36] minrk I'm sure that's a bug - I asked on-list, but no reply
[20:37] minrk there's no way it's meant to be impossible to tell whether a cmd might be incoming
[20:41] jond minrk: try changing last line of router_t::xhas_in to return fq.has_in() || !pending_commands.empty ();
[20:41] jond that's a guess!
[20:44] sarkis wooooo 0mq
[20:44] sarkis hey guys, i have been trying to figure out how i can use 0mq to act as a job queue
[20:44] sarkis anyone know of some examples of this?
[20:45] sarkis i checked out the examples/ folder on github ... couldn't see what would be similar to what i wanted
[20:45] michelp sarkis, have you read the guide?
[20:46] michelp there are many patterns heavily documented there that could apply
[20:46] michelp it's a broad question, but a quick idea would be to have a producer putting jobs into a broker, that farms them out to workers, who do the job and then respond back to the producer with a result
[20:46] errordeveloper sarkis:
[20:53] errordeveloper mato: hey .. can I pm u ?
[20:53] sarkis thanks guys
[20:59] mato errordeveloper: Hi
[20:59] mato errordeveloper: yes, sure
[20:59] errordeveloper mato: hello
[21:37] minrk jond: more complete, with front/backend examples included:
[21:45] jond minrk: cheers:I think that one liner above fixes the POLLIN cmd issue. i just call evts.get(router) now in mux function
[21:46] minrk nice
[21:47] minrk If you want to post that patch to the list, that would be great
[21:47] jond give it a whirl and I'll reacquaint myself with git
[21:50] jond minrk: is it git format-patch -S. i use perforce all day!
[21:53] minrk I think so
[22:02] jond minrk: I've sent to the list.
[22:05] minrk thanks
[22:06] jond minrk: cyl