ZeroMq IRC Log

Wednesday July 6, 2011

[Time] Name	Message
[02:36] meanphil	hey guys, I'm currently using Spread to multicast messages that come from a webapp, to all the other application servers... so webapp connects to locally running spread daemon, multicasts to other nodes where they are received by scripts listening to their locally running spread daemon
[02:37] meanphil	if I replace the spread daemon with one using 0MQ, it seems like I'd need a #nodes pubsub sockets per each daemon
[02:37] meanphil	does that sound right?
[02:38] meanphil	ie, for 3 servers, each 0MQdaemon running would need 2 'sub' sockets, and 1 'pub' sockets, and they'd each have to connect to each other
[02:39] meanphil	along with a REP socket to listen to the local webapp
[02:47] shimon	meanphil: i'm a zmq noob, but it doesn't seem like REQ/REP is the right pattern. Maybe fanout?
[04:19] meanphil	hrm, can anyone tell me why the following works once, but then never stops polling for the second message sent?
[04:19] meanphil	https://gist.github.com/1066552
[06:23] sustrik	bacause you send only one message?
[13:36] katkee	hello, i installed 0mq from git but i cannot find zmq_queue. where can i find it please?
[13:43] reuben	katkee, what git, zeromq/libzmq from github?
[14:04] mikko	vinnyt: i think so
[14:04] mikko	someone mailed some time ago that they are using it on iOS
[14:19] vinnyt	I'll keep googling...
[14:34] jd10	https://github.com/kro/akka-zmq
[15:25] katkee	reuben: zmq_queue is present in the ubuntu package zeromq-bin
[15:25] reuben	..
[15:27] katkee	reuben: i am looking for zmq_queue and it is referenced here https://github.com/zeromq/zeromq2-1/blob/master/debian/zeromq-bin.install
[15:28] reuben	when you build zmq they get built too
[15:28] reuben	not sure if they are installed
[15:28] guido_g	they are
[15:28] katkee	i looked in /usr/local/bin/ but don't find them...
[15:29] reuben	what prefix did you use when configuring?
[15:29] katkee	i just did ./configure make make install
[15:32] guido_g	they're really gone from 2.1
[15:33] guido_g	ok, it's stated in the NEWS file for the 2.1.3 release from 2011-03-30
[15:34] katkee	guido_g: ok, how can i get it back, i need to get an older release?
[15:35] ssi	what happens if a client connects to my socket, and I don't poll it because I have no data available, and the client times it out?
[15:35] ssi	or more specifically, if client A and client B connect in that order
[15:35] ssi	and A times out before I poll
[15:35] ssi	will I read B's request on the next poll?
[15:36] guido_g	katkee: if you really need these exact programs, yes that seems to be the only way
[15:37] katkee	ok, i will look in older releases
[15:42] katkee	guido_g: found it here 7d9603d722c9c2752dccd0c51f470e68d0e0c48c in zeromq2-1
[15:42] katkee	thanks guido_g
[17:13] pieterh	katkee: hi
[17:23] ssi	pieterh: if I have two separate applications running on one machine using the same inproc:// addresses, do those namespaces collide?
[17:23] pieterh	ssi: different contexts = different namespaces for inproc
[17:23] ssi	yay!
[17:26] ssi	pieterh: do you see my question about client timeouts in the backscroll from 2 hours ago?
[17:27] pieterh	what socket types? does A send a message?
[17:27] ssi	it's REQ/REP, and yes
[17:27] pieterh	there's no notion of timeout in REQ/REP
[17:27] pieterh	nor anywhere else, unless you add it on top
[17:27] ssi	hrm, that might be an issue
[17:28] pieterh	given the way you explain it, yes, I assume you want to build your own heartbeating
[17:28] ssi	well my application is that I'm building a stripped down JMS provider that uses 0mq as a transport
[17:28] ssi	a MessageConsumer has the option to receive blocking indefinitely, or receive nowait, or with a specified timeout
[17:29] ssi	if a consumer tries to receive from an empty queue with a timeout, and another tries to receive behind that one with no timeout,
[17:29] ssi	I need a way for the first consumer to timeout without screwing up my message handling
[17:29] ssi	I have control of both ends, but maybe REQ/REP isn't the right way to go
[17:30] ssi	maybe the consumer end needs to be a DEALER instead
[17:31] ssi	in that case, the timeout would need to be sent along with the request... if the broker doesn't get to it before the timeout expires, when it does get to it it'll just ignore it
[17:31] ssi	although that'd require clock sync, which is probably the wrong way to handle it
[17:34] pieterh	for a start I'd use a router-dealer pattern
[17:35] ssi	the broker itself is router-dealer
[17:35] ssi	when the broker receives a request, specifically a consumer request, it passes it off to a worker which handles the specific destination
[17:35] ssi	the reason for that is an empty queue can have blocking requests
[17:35] pieterh	and between client and broker you have req/rep?
[17:35] ssi	well broker frontend is router
[17:36] ssi	client I intended to use req
[17:36] ssi	so client sends a req to broker, which routes it to a worker
[17:36] pieterh	bleh, req is really limited to straight request/response
[17:36] ssi	that's fine, I can use dealer in the client instead with practically no impact
[17:36] pieterh	for sure
[17:36] pieterh	you create a compatible envelope
[17:37] ssi	will I need to add the empty frame if I switch from REQ to DEALER?
[17:37] ssi	the client address gets added in the router socket if I remember correctly
[17:37] pieterh	yes, if your downstream code expects it
[17:38] pieterh	if you're in control of all the code and don't need the REP stack handling you can forget the empty frame
[17:38] pieterh	and then you probably want to empty your router socket as input arrives
[17:38] pieterh	and store requests explicitly in your own queues until they can be processed
[17:41] ssi	what would I replace the workers' REP socket with?
[17:41] ssi	another DEALER?
[17:41] pieterh	that's usually how it seems to turn out
[17:41] pieterh	dealer at both ends so you can do async work like heartbeats
[17:41] ssi	right
[17:41] pieterh	router for broker frontend and backend
[17:42] ssi	not sure I even need it for backend
[17:42] ssi	I don't care for the client to get a worker's address
[17:42] pieterh	if you need worker heartbeating, for instance, that means a ROUTER
[17:42] ssi	sure
[17:42] pieterh	clients should never see workers, period
[17:43] pieterh	you've looked at the pirate examples in ch4?
[17:43] ssi	ya
[17:43] pieterh	this is pretty much majordomo, I think
[17:46] ssi	you are correct!
[17:52] ssi	yeah I think you're right... it's very close to majordomo pattern
[17:56] ssi	ok here's what I'm unclear on
[17:56] ssi	I'm looking at the asynchronous majordomo example
[17:56] ssi	it has a timeout in its receive
[17:56] ssi	but what exactly happens in the case of timeout?
[17:56] ssi	the broker's unaware that it's timed out, yes?
[17:57] ssi	in my application, the broker needs to be timeout aware, somehow... otherwise it's gonna dequeue and send a message to someone who's either not listening or not expecting
[17:57] ssi	and that JMS-layer message will be lost
[17:58] pieterh	client timesout, creates a new socket
[17:58] pieterh	this is the lazy pirate approach
[17:58] pieterh	broker routes back any replies to that client, they are silently discarded
[17:58] pieterh	a client never gets a reply it was not expecting
[17:59] ssi	right, but the broker consumed a message from the queue and sent it to a client that discards it
[17:59] pieterh	this is the one way I was able to use REQ sockets successfully in a reliability scenario
[17:59] ssi	so that message is lost
[17:59] pieterh	the majordomo pattern is for handling client requests
[17:59] pieterh	clients do not consume messages from a queue
[17:59] pieterh	your scenario is somewhat different, I guess
[17:59] ssi	sorry, we're speaking in different layers here
[17:59] pieterh	yes
[17:59] pieterh	sorry, have a meeting, have to go, cyl
[18:28] ssi	thinking about it more, I don't think I actually need to handle timeout at all!
[19:16] pieterh	ssi: if not, that makes things simpler
[19:27] ssi	well, through the magic of zeromq
[19:27] ssi	if the broker responds to a client request after the client has timed out, it'll just queue on the client side at the zmq layer
[19:27] ssi	then when the client goes to receive again, it just needs to try receiving off the socket before it sends out a request
[19:27] ssi	if there's a message already waiting, it reads that and returns immediately
[19:28] ssi	the only downside to that is the case where the broker sends a reply, but the client is timed out, and then the client dies without ever actually reading it
[19:28] ssi	but I'm not worried about that edge case right now :D
[20:06] damien	hi
[20:06] damien	quick question
[20:06] damien	the changelog for stable says 2.1.8
[20:06] damien	but the webpage says 2.1.7
[20:07] damien	is 2.1.8 anywhere?
[20:15] ssi	here's a new one
[20:15] ssi	Assertion failed: rc == 0 (zmq_connecter.cpp:48)
[20:23] ssi	k so, this is actually my first time doing zmq with anything but inproc sockets...
[20:24] ssi	I'm binding to "tcp://*:62626", should I see it show up as listening in netstat?
[20:24] ssi	cause I don't
[20:52] whack	ssi: pastebin your code sample?
[21:18] ssi	I got it figured out