Wednesday July 6, 2011

[Time] NameMessage
[02:36] meanphil hey guys, I'm currently using Spread to multicast messages that come from a webapp, to all the other application servers... so webapp connects to locally running spread daemon, multicasts to other nodes where they are received by scripts listening to their locally running spread daemon
[02:37] meanphil if I replace the spread daemon with one using 0MQ, it seems like I'd need a #nodes pubsub sockets per each daemon
[02:37] meanphil does that sound right?
[02:38] meanphil ie, for 3 servers, each 0MQdaemon running would need 2 'sub' sockets, and 1 'pub' sockets, and they'd each have to connect to each other
[02:39] meanphil along with a REP socket to listen to the local webapp
[02:47] shimon meanphil: i'm a zmq noob, but it doesn't seem like REQ/REP is the right pattern. Maybe fanout?
[04:19] meanphil hrm, can anyone tell me why the following works once, but then never stops polling for the second message sent?
[04:19] meanphil
[06:23] sustrik bacause you send only one message?
[13:36] katkee hello, i installed 0mq from git but i cannot find zmq_queue. where can i find it please?
[13:43] reuben katkee, what git, zeromq/libzmq from github?
[14:04] mikko vinnyt: i think so
[14:04] mikko someone mailed some time ago that they are using it on iOS
[14:19] vinnyt I'll keep googling...
[14:34] jd10
[15:25] katkee reuben: zmq_queue is present in the ubuntu package zeromq-bin
[15:25] reuben ..
[15:27] katkee reuben: i am looking for zmq_queue and it is referenced here
[15:28] reuben when you build zmq they get built too
[15:28] reuben not sure if they are installed
[15:28] guido_g they are
[15:28] katkee i looked in /usr/local/bin/ but don't find them...
[15:29] reuben what prefix did you use when configuring?
[15:29] katkee i just did ./configure make make install
[15:32] guido_g they're really gone from 2.1
[15:33] guido_g ok, it's stated in the NEWS file for the 2.1.3 release from 2011-03-30
[15:34] katkee guido_g: ok, how can i get it back, i need to get an older release?
[15:35] ssi what happens if a client connects to my socket, and I don't poll it because I have no data available, and the client times it out?
[15:35] ssi or more specifically, if client A and client B connect in that order
[15:35] ssi and A times out before I poll
[15:35] ssi will I read B's request on the next poll?
[15:36] guido_g katkee: if you really need these exact programs, yes that seems to be the only way
[15:37] katkee ok, i will look in older releases
[15:42] katkee guido_g: found it here 7d9603d722c9c2752dccd0c51f470e68d0e0c48c in zeromq2-1
[15:42] katkee thanks guido_g
[17:13] pieterh katkee: hi
[17:23] ssi pieterh: if I have two separate applications running on one machine using the same inproc:// addresses, do those namespaces collide?
[17:23] pieterh ssi: different contexts = different namespaces for inproc
[17:23] ssi yay!
[17:26] ssi pieterh: do you see my question about client timeouts in the backscroll from 2 hours ago?
[17:27] pieterh what socket types? does A send a message?
[17:27] ssi it's REQ/REP, and yes
[17:27] pieterh there's no notion of timeout in REQ/REP
[17:27] pieterh nor anywhere else, unless you add it on top
[17:27] ssi hrm, that might be an issue
[17:28] pieterh given the way you explain it, yes, I assume you want to build your own heartbeating
[17:28] ssi well my application is that I'm building a stripped down JMS provider that uses 0mq as a transport
[17:28] ssi a MessageConsumer has the option to receive blocking indefinitely, or receive nowait, or with a specified timeout
[17:29] ssi if a consumer tries to receive from an empty queue with a timeout, and another tries to receive behind that one with no timeout,
[17:29] ssi I need a way for the first consumer to timeout without screwing up my message handling
[17:29] ssi I have control of both ends, but maybe REQ/REP isn't the right way to go
[17:30] ssi maybe the consumer end needs to be a DEALER instead
[17:31] ssi in that case, the timeout would need to be sent along with the request... if the broker doesn't get to it before the timeout expires, when it does get to it it'll just ignore it
[17:31] ssi although that'd require clock sync, which is probably the wrong way to handle it
[17:34] pieterh for a start I'd use a router-dealer pattern
[17:35] ssi the broker itself is router-dealer
[17:35] ssi when the broker receives a request, specifically a consumer request, it passes it off to a worker which handles the specific destination
[17:35] ssi the reason for that is an empty queue can have blocking requests
[17:35] pieterh and between client and broker you have req/rep?
[17:35] ssi well broker frontend is router
[17:36] ssi client I intended to use req
[17:36] ssi so client sends a req to broker, which routes it to a worker
[17:36] pieterh bleh, req is really limited to straight request/response
[17:36] ssi that's fine, I can use dealer in the client instead with practically no impact
[17:36] pieterh for sure
[17:36] pieterh you create a compatible envelope
[17:37] ssi will I need to add the empty frame if I switch from REQ to DEALER?
[17:37] ssi the client address gets added in the router socket if I remember correctly
[17:37] pieterh yes, if your downstream code expects it
[17:38] pieterh if you're in control of all the code and don't need the REP stack handling you can forget the empty frame
[17:38] pieterh and then you probably want to empty your router socket as input arrives
[17:38] pieterh and store requests explicitly in your own queues until they can be processed
[17:41] ssi what would I replace the workers' REP socket with?
[17:41] ssi another DEALER?
[17:41] pieterh that's usually how it seems to turn out
[17:41] pieterh dealer at both ends so you can do async work like heartbeats
[17:41] ssi right
[17:41] pieterh router for broker frontend and backend
[17:42] ssi not sure I even need it for backend
[17:42] ssi I don't care for the client to get a worker's address
[17:42] pieterh if you need worker heartbeating, for instance, that means a ROUTER
[17:42] ssi sure
[17:42] pieterh clients should never see workers, period
[17:43] pieterh you've looked at the pirate examples in ch4?
[17:43] ssi ya
[17:43] pieterh this is pretty much majordomo, I think
[17:46] ssi you are correct!
[17:52] ssi yeah I think you're right... it's very close to majordomo pattern
[17:56] ssi ok here's what I'm unclear on
[17:56] ssi I'm looking at the asynchronous majordomo example
[17:56] ssi it has a timeout in its receive
[17:56] ssi but what exactly happens in the case of timeout?
[17:56] ssi the broker's unaware that it's timed out, yes?
[17:57] ssi in my application, the broker needs to be timeout aware, somehow... otherwise it's gonna dequeue and send a message to someone who's either not listening or not expecting
[17:57] ssi and that JMS-layer message will be lost
[17:58] pieterh client timesout, creates a new socket
[17:58] pieterh this is the lazy pirate approach
[17:58] pieterh broker routes back any replies to that client, they are silently discarded
[17:58] pieterh a client never gets a reply it was not expecting
[17:59] ssi right, but the broker consumed a message from the queue and sent it to a client that discards it
[17:59] pieterh this is the one way I was able to use REQ sockets successfully in a reliability scenario
[17:59] ssi so that message is lost
[17:59] pieterh the majordomo pattern is for handling client requests
[17:59] pieterh clients do not consume messages from a queue
[17:59] pieterh your scenario is somewhat different, I guess
[17:59] ssi sorry, we're speaking in different layers here
[17:59] pieterh yes
[17:59] pieterh sorry, have a meeting, have to go, cyl
[18:28] ssi thinking about it more, I don't think I actually need to handle timeout at all!
[19:16] pieterh ssi: if not, that makes things simpler
[19:27] ssi well, through the magic of zeromq
[19:27] ssi if the broker responds to a client request after the client has timed out, it'll just queue on the client side at the zmq layer
[19:27] ssi then when the client goes to receive again, it just needs to try receiving off the socket before it sends out a request
[19:27] ssi if there's a message already waiting, it reads that and returns immediately
[19:28] ssi the only downside to that is the case where the broker sends a reply, but the client is timed out, and then the client dies without ever actually reading it
[19:28] ssi but I'm not worried about that edge case right now :D
[20:06] damien hi
[20:06] damien quick question
[20:06] damien the changelog for stable says 2.1.8
[20:06] damien but the webpage says 2.1.7
[20:07] damien is 2.1.8 anywhere?
[20:15] ssi here's a new one
[20:15] ssi Assertion failed: rc == 0 (zmq_connecter.cpp:48)
[20:23] ssi k so, this is actually my first time doing zmq with anything but inproc sockets...
[20:24] ssi I'm binding to "tcp://*:62626", should I see it show up as listening in netstat?
[20:24] ssi cause I don't
[20:52] whack ssi: pastebin your code sample?
[21:18] ssi I got it figured out