ZeroMq IRC Log

Wednesday September 28, 2011

[Time] Name	Message
[00:06] dcolish	is it an ability of the ROUTER/XREP socket type to be able to respond directly to REQ socket messages?
[00:10] dcolish	i'm not seeing that happen :/
[00:11] dcolish	http://paste.pocoo.org/show/483592/
[03:41] dublisk	Hi, I'm trying to improve my knowledge and gain some experience with low latency socket programming. Do you think learning about 0MQ and possibly trying to contribute code would be a good direction?
[05:50] sustrik	dublisk: yes
[05:50] sustrik	there are no much other open source options that do this
[05:51] sustrik	you can have a look at OpenPGM too, but multicast is order of magnitude more complex than TCP
[05:52] sustrik	dcolish: with REQ socket you have to send a request first, then get a reply
[05:52] sustrik	you are doing it other way round
[06:04] CIA-79	libzmq: 03Martin Sustrik 07master * r8485a5e 10/ (src/router.cpp src/xrep.cpp): Assert fixed in XREP & ROUTER when sending to terminating pipe (issue 258) ...
[13:34] pieterh	hi folks
[13:35] alfred1	hi piet
[13:35] pieterh	hi alfred1
[14:03] shales	I'm writing some unittests. How can I setup a socket so that a send blocks or returns EAGAIN? Right now I have two connected XREQ sockets, A and B, and have set low HWM and SNDBUF on skt A. I send 1000 small messages on A and don't recv on B, but it doesn't block. Should it?
[14:06] dcolish	sustrik: what do you mean?
[14:07] dcolish	in that pasted example, i combined two distinct files
[14:12] guido_g	howdy guys
[14:13] guido_g	pieterh: you wanted something a few days ago?
[14:41] dcolish	ah i see, the python bindings use send_mulitpart/recv_mulitpart when using a ROUTER socket
[17:38] sundbp	cremes: fyi I forked ffi-rzmq and made a few changes. EINTR was marked as not being used in the v2 api. pretty sure it is.
[17:38] cremes	sundbp: cool
[17:38] cremes	btw, i am probably going to rip out all of the exceptions stuff and just go back
[17:38] cremes	to pure return codes
[17:39] cremes	it's much simpler
[17:39] cremes	though it will definitely break the existing ruby api
[17:39] cremes	thoughts?
[17:40] sundbp	cremes: fine by me. i just started plauying and from reading the api doc on zeromq.org then needed to read through how the exceptions were created etc to know what to catch to handle things etc
[17:41] cremes	i fear it may piss off a bunch of folks who have a lot of code tied to the old api... oh well,
[17:41] cremes	they don't have to upgrade :)
[17:41] sundbp	cremes: always the case
[17:42] sundbp	cremes: i think it's good to have a simple wrapper around the C api as a base. may not be the most ruby idiomatic but makes it simple to translate from the guide etc. then one can wrap that up nicely in a layer on top to be more idiomatic
[17:42] cremes	agreed
[17:43] sundbp	cremes: can be part of the same gem, just two ways of viewing the world
[17:43] cremes	yes
[17:46] whack	sundbp: +1
[17:48] sundbp	cremes: for example i'm implementing the majordomo part of the zguide and on using socket.recvmsgs i need to figure out what exceptions to catch by looking at the code in socket.rb and util.rb (for error_check)
[17:49] cremes	right... it's just too confusing
[17:49] cremes	just using the rc along with zmq_errno() would be simpler
[17:49] sundbp	cremes: would be great if i could use return codes and errno in my first pass just as C api, and on 2nd pass update to a high level api, or wrap up my own low level stuff into a high level api suiting the domain
[17:50] cremes	sundbp: you are inspiring me to work on that today...
[17:50] cremes	i need to grab lunch but i may poke at this afterwards
[17:51] sundbp	cremes: you're welcome :) so my wishlist is a "carbon" copy C api in one namespace, and then later a more high level api in a different namespace
[17:56] mikko	sustrik: there?
[18:07] pieterh	guido_g: hi...
[18:07] guido_g	hi pieterh_
[18:08] guido_g	you tried to contact me a few days ago
[18:08] pieterh	I did... it was a random question
[18:15] mikko	finally ==20108== ERROR SUMMARY: 0 errors from 0 contexts
[18:15] mikko	it is very hard to get shutdown order right with C++ and shared ptrs
[18:16] pieterh	mikko: there's a persistent issue on Jenkins with czmq binding
[18:17] mikko	looks like a lot og builds stuck
[18:17] pieterh	hmm
[18:18] mikko	tests getting stuck
[19:02] guido_g	wb pieterh
[19:09] shales	I'm trying to force a zmq_send to block for a unit test. I have two connected XREQ sockets. I set HWM and SNDBUF to low numbers on one socket and send 10000 small messages through it without calling recv on the other socket. Should this block?
[19:10] minrk	no
[19:10] guido_g	depends on the size of the os-level buffers
[19:10] guido_g	on both sides
[19:12] shales	so is setting RCVBUF on the receiving socket enough?
[19:13] michelp	shales, do you want to test zmq_send, or test the thing that calls it? if the latter, then I would suggest mocking the zmq_send function instead of trying to force it to block by setting up an unrealistic configuration
[19:21] shales	michelp: I'm testing an integration of zeromq with another eventloop, so I'd like to have zmq_send actually return non-zero and set errno to EAGAIN. Trying to ensure I don't miss events on the sockets events FD. I don't think a mock zmq_send would work
[19:24] guido_g	then you have to set the os-level buffers to a very small size
[19:24] guido_g	which might be prevented by the os
[19:54] staticfloat	Hey guys, I'm having trouble using a ROUTER socket; I'm trying to use custom routing messages at the beginning of the my frame to route packets, and I think the router is dropping them for some reason
[19:55] staticfloat	I'm using zmq 3.0
[19:55] staticfloat	Does anyone know if I need to use LABELs instead of just normal message parts to differentiate the address of the peer I want to talk to from the data I want to send to it?
[19:55] minrk	ROUTERs do not use LABELs in 3.0
[19:56] staticfloat	Alright. Let me run some code past you to see if this is the right idea then;
[20:00] michelp	shales, I wonder if it makes sense to have some kind of "testing transport" where you can get normal and exceptional behavior out of the api based on how you configure it. just a thought
[20:00] michelp	obviously it would require some more work than maybe what you were looking for
[20:06] jond	minrk: is that 4.0 multiplexer mentioned on the list up on github anywhere?
[20:06] minrk	not atm, but I can gist it quickly
[20:07] minrk	I haven't cleaned it up for the public yet - I will put it in the pyzmq examples once I do
[20:07] staticfloat	@minrk: Alright, here's the relevant code fragment; http://pastebin.com/T11WDt0F
[20:07] jond	minrk that would be great.
[20:09] staticfloat	Basically, the REP socket never gets anything in, and when I sniff via wireshark, I don't get anything being sent except tcp connection setup and teardown
[20:09] minrk	jond: https://gist.github.com/1249108
[20:09] jond	minrk: thx
[20:10] minrk	staticfloat_: set identity before bind
[20:11] minrk	jond: backend sockets need to participate in a quick handshake:
[20:11] minrk	s.connect(addr)
[20:12] minrk	s.recv_multipart() # check HAND, etc.
[20:12] minrk	s.send_multipart([SHAKE, identity])
[20:13] jond	minrk: noted, thx again
[20:25] jond	minrk: wasnt the old model two xrep sockets in a queue with prefix flipping?
[20:25] minrk	yes
[20:26] jond	so I was expecting this example to be two router sockets?
[20:26] minrk	You could certainly make it that way
[20:27] minrk	but my use case is really multiplexed request-reply
[20:27] minrk	everything coming from downstream is in reply to something upstream
[20:28] jond	minrk: so xrep for the request/rep and the 'worker's talk to router
[20:28] minrk	the only need to use a ROUTER socket instead of XREP is if you want to send messages to a peer without the prefix information an XREP socket gets on a request
[20:29] minrk	yes, it's XREQ->XREP+ROUTER<-XREP
[20:29] minrk	workers are XREP, and talk to the ROUTER
[20:29] minrk	clients are XREQ, and talk to XREP
[20:30] jond	minrk: it's certainly a bit more cumbersome than the old way ;-)
[20:31] minrk	it is
[20:33] jond	minrk: i'm gonna have a look at the 4.0 src and see what's going on. thx for that gist i'm going to have a more detailed look tomorrow, seems funny the way 'cmd' has to be handled.
[20:36] minrk	I'm sure that's a bug - I asked on-list, but no reply
[20:37] minrk	there's no way it's meant to be impossible to tell whether a cmd might be incoming
[20:41] jond	minrk: try changing last line of router_t::xhas_in to return fq.has_in() \|\| !pending_commands.empty ();
[20:41] jond	that's a guess!
[20:44] sarkis	wooooo 0mq
[20:44] sarkis	hey guys, i have been trying to figure out how i can use 0mq to act as a job queue
[20:44] sarkis	anyone know of some examples of this?
[20:45] sarkis	i checked out the examples/ folder on github ... couldn't see what would be similar to what i wanted
[20:45] michelp	sarkis, have you read the guide?
[20:46] michelp	there are many patterns heavily documented there that could apply
[20:46] michelp	it's a broad question, but a quick idea would be to have a producer putting jobs into a broker, that farms them out to workers, who do the job and then respond back to the producer with a result
[20:46] errordeveloper	sarkis: zguide.zeromq.org
[20:53] errordeveloper	mato: hey .. can I pm u ?
[20:53] sarkis	thanks guys
[20:59] mato	errordeveloper: Hi
[20:59] mato	errordeveloper: yes, sure
[20:59] errordeveloper	mato: hello
[21:37] minrk	jond: more complete, with front/backend examples included: https://gist.github.com/1249324
[21:45] jond	minrk: cheers:I think that one liner above fixes the POLLIN cmd issue. i just call evts.get(router) now in mux function
[21:46] minrk	nice
[21:47] minrk	If you want to post that patch to the list, that would be great
[21:47] jond	give it a whirl and I'll reacquaint myself with git
[21:50] jond	minrk: is it git format-patch -S. i use perforce all day!
[21:53] minrk	I think so
[22:02] jond	minrk: I've sent to the list.
[22:05] minrk	thanks
[22:06] jond	minrk: cyl

ZeroMq Home

Wednesday September 28, 2011