ZeroMq IRC Log

Friday September 16, 2011

[Time] Name	Message
[00:13] rb	Anyone there?
[00:19] cremes	rb_: have you read the 0mq guide? it covers lots and lots of use cases
[00:40] narobi	any gurus around?
[00:48] narobi	ah well, i'll try anyway, i have a pretty basic question
[00:49] narobi	have worker thread and non worker thread, worker thread sits on a blocked receive on a pull socket waiting for instructions
[00:49] narobi	i have a main push socket that pushes to workers
[00:49] narobi	thenw hen i am shutting down, i create another push socket and connect it to the workers and send them a message to tell them to quit
[00:49] narobi	on windows, the code works fine
[00:50] narobi	on linux, using zeromq 2.1.9, the worker never gets the message from the secondary push socket
[00:50] narobi	the push socket only connects to that one pull socket
[00:50] narobi	so it's not a fair queuing issue
[00:51] narobi	i have tried sleeping for 5 seconds after send and making sure the push socket does not close until the worker has quit
[00:51] narobi	no dice
[00:51] narobi	nothing is working
[00:51] narobi	if anyone has any ideas, iw ould be greatful
[01:21] cremes	narobi: i don't understand your setup
[01:21] cremes	so you have a PUSH socket that connects to your PULL sockets in the non-worker thread
[01:22] cremes	if blocks on recv and sends data out as it comes in
[01:22] cremes	when you want to shut down, you create a second PUSH socket and connect it to your PULL sockets
[01:22] cremes	and then send a termination message
[01:22] cremes	is that correct?
[01:23] cremes	if so, is the transport tcp or inproc?
[06:01] MickeM	Whats the best way to store "information" with a connection in req/rep (for instance autenticated as and that kind of thing)? (one option would be to have a hashmap with subscriber id and the relevant information I guess, but is there anything fancy?)
[07:32] CIA-121	libzmq: 03Martin Sustrik 07master * r06bdf2c 10/ (8 files): Check message syntax in REQ asynchronously ...
[07:42] mikko	MickeM: what language?
[08:46] MickeM	mikko: C++
[08:52] mikko	MickeM: i guess it depends on what information you store and how you want to access it
[08:54] MickeM	mikko: well, a struct with some stuff and as I said I want to have clients "login once" and then allow them to be logged in... problem with "subscriber" is ofcourse that someone could fake it, hence some form of "session state" would be preferable (as zeromq could reset that when a connection is reset)
[08:58] mikko	MickeM: zeromq doesn't really expose the connections to the application level
[08:59] mikko	MickeM: there are several ways to tackle this, i guess
[08:59] mikko	MickeM: you could use PKI i guess
[09:00] mikko	MickeM: or after login give a trancient secret to the client and encrypt messages using that
[09:00] mikko	i remember there being some talk about more client oriented library that would communicate with zeromq servers
[09:00] mikko	more of a one connection concept
[09:02] MickeM	mikko: humm.. ok... I guess the poor mans version would then be to just send a cookie to the clients then... signing messages will be used at some point but I am not there yet...
[13:03] CIA-121	libzmq: 03Martin Sustrik 07master * r5936379 10/ src/mtrie.cpp : Bug in mtrie fixed ...
[14:16] taghawi	Hello I am looking for binaries of zeromq for a Mac OS X Server v10.5 Leopard
[14:16] taghawi	with a One 2.8GHz Quad-Core Intel Xeon (quad-core)
[14:32] cremes	taghawi: no one is distributing binaries for osx; you can use macports (zmq) or homebrew (zeromq) to handle
[14:32] cremes	the compilation and installation for you
[14:32] cremes	or, download the posix tarball and build & install it yourself
[14:32] cremes	we can provide help if you are unfamiliar with how to do that
[14:35] CIA-121	libzmq: 03Martin Sustrik 07master * re170136 10/ (src/mtrie.cpp src/mtrie.hpp): More bugs in mtrie fixed ...
[14:50] errordeveloper	hm .. I'm not getting much response on #msgpack ..
[14:50] errordeveloper	can anyone have a look here
[14:50] errordeveloper	msgpack_pack_int (&mpkg, i);
[14:50] errordeveloper	this function results in packing 0 instead of incr value
[14:51] errordeveloper	https://gist.github.com/1222271#file_client.c
[16:01] cremes	sustrik: is 3.x and 4.x going to set the RCVMORE flag on the socket even for label parts?
[16:01] cremes	sustrik: according to a thread on the ML from a few weeks ago, that seemed to be the concensus...
[16:13] cremes	sustrik: i also notice that printing the contents of the label message does not return the identity data
[16:13] cremes	that i have previously set
[16:13] cremes	is this a bug (on 3.0 master) or is the custom identity no longer sent through?
[16:46] pieterh	cremes: nice catch... I've not actually tested multipart messages in CZMQ over 0MQ/3.x
[16:47] cremes	pieterh_: i don't know if it's a catch or not... i'm thoroughly baffled over what should happen! :)
[16:47] pieterh	check the design documents, it's all there
[16:47] pieterh	oh wait...
[16:47] cremes	;)
[16:48] cremes	btw, john murphy says hi
[16:48] cremes	i see him all of the time now...
[16:48] pieterh	hey!
[16:48] pieterh	awesome
[16:48] pieterh	give him my highest five
[16:48] cremes	will do
[16:49] pieterh	so wrt the labels IMO we need to just insist on the behavior we want
[16:49] pieterh	(a) patch 3.x until it does what we want and (b) lobby for this to go into 4.x
[16:49] cremes	ok
[16:50] pieterh	wrt identities, I'd suggest treating this as a bug: test case, issue, etc.
[16:50] cremes	i'll write one up and attach some C
[16:50] pieterh	yup
[16:50] cremes	that will be an easy one (i like easy ones)
[16:51] pieterh	I'll take a quick look if there's a simple way to make RCVMORE work on labels
[16:52] pieterh	cremes: would you also file an issue for the RCVMORE change? we will treat this as a bug too
[16:52] pieterh	don't need a test case for that
[16:52] cremes	sure
[16:53] cremes	let me do that first
[16:53] cremes	i won't get to the identity repro until tomorrow
[16:53] pieterh	I'll change 3-0 master and we can then work on how that'd go into 4.0
[16:55] pieterh	I've made the change, it's trivial
[16:57] cremes	great
[16:57] pieterh	you have an issue number for me?
[16:57] cremes	LIBZMQ-255 tracks it
[16:57] pieterh	255 I assume
[16:58] pieterh	sweet
[17:00] errordeveloper	I have this strange issue with msgpack
[17:00] errordeveloper	https://gist.github.com/1222271#file_client.c
[17:00] errordeveloper	when I try to do 'msgpack_pack_int (&mpkg, i);',
[17:00] pieterh	cremes: ok, you can grab it from 3-0 master
[17:01] errordeveloper	it doesn't pick-up the right value of 'i'
[17:01] errordeveloper	I don't use any otimization flags on my code
[17:02] errordeveloper	is one who is using msgpack around ?
[17:03] cremes	errordeveloper: is there a msgpack channel?
[17:03] cremes	or a mailing list?
[17:03] cremes	i've played with it lately but using the ruby bindings
[17:03] cremes	it works okay for me
[17:05] cremes	pieterh_: did you confirm it worked? because my test still shows false
[17:06] pieterh	you mean like test the code?
[17:06] cremes	heh
[17:06] pieterh	for that I'd need a test case, if you want to make me one I'll use it NP
[17:06] cremes	i'll take that as a no
[17:06] cremes	ok
[17:06] pieterh	lol
[17:06] pieterh	:)
[17:06] cremes	i'll do that tomorrow too
[17:07] pieterh	there is probably something evil in the code that I missed
[17:07] cremes	probably... sustrik likes evil code
[17:07] cremes	:)
[17:08] pieterh	evil slavic code...
[17:08] pieterh	wouldn't trust my business to anything less
[17:08] cremes	nor would i
[17:08] cremes	btw, are you still in texas?
[17:09] pieterh	yup, Dallas
[17:09] pieterh	it's surprisingly fun tbh
[17:09] cremes	i believe it... texans are good folks
[17:09] cremes	gotta run... back in a bit
[17:10] pieterh	cyl
[17:15] errordeveloper	cremes: #msgpack has a few people there :)
[17:25] errordeveloper	cremes: ok, I see where the problem is
[17:25] errordeveloper	cremes: msgpack_pack_<type> are macros
[17:25] errordeveloper	that's what I thought they were :))
[17:26] errordeveloper	well, they are macros expanding to inline functions
[17:34] errordeveloper	it's nuts, even using a pointer to the counter varieble still does the same
[18:21] errordeveloper	ok, i'll see if I get an answer to this on SO :)
[18:22] errordeveloper	http://stackoverflow.com/questions/7448693/counter-variable-passed-to-msgpack-pack-int-macro-doesnt-increment
[18:46] rb	Any Gurus?
[18:49] michelp	rb_, there's 95 people in here. If you have a question, go ahead and ask it
[18:50] benbangert	is there any way to see how many undelivered messages are in memory for a zmq socket?
[18:51] socksandsandals	benbangert: AFAIK no
[18:51] benbangert	bummer
[18:51] socksandsandals	I too wish for that ;-)
[18:52] benbangert	the closest I saw was the ability to see whether or not adding a message would cause blocking
[18:52] socksandsandals	yeah, but that just means the next message will hit the disk queue
[18:52] benbangert	right, it just means, yea, there's stuff that isn't delivered yet
[18:52] socksandsandals	you can infer from that whether you're at the HWM and then you'd know at least that much
[18:52] benbangert	but not the 'how'
[18:52] benbangert	well, assuming I didn't have the disk based messages
[18:53] socksandsandals	pretty ghetto, though
[18:53] benbangert	which reminds me, I didn't see anything indicating whether the disk based messages will resume upon an app restart
[18:53] socksandsandals	ah right, yeah
[18:53] socksandsandals	good question
[18:53] socksandsandals	idk about that
[18:54] benbangert	easy enough to try it I guess :)
[18:54] socksandsandals	actually, I'm pretty sure they won't
[18:54] socksandsandals	since otherwise, nothing I've written with 0MQ would be working ;-)
[18:54] benbangert	since there's nothing explicit about naming the file to use, I'd guess not
[18:55] rb	I am looking for a solution for this : 1, there is a message bus; 2, any number of processes can attach themselves to the bus; 3, each process attached to the bus behave as both a client and server; 4, each process will send requests to any other process and will get reply from it, at the same time, it will answer requests from other process that are addressed to it. Thankx
[18:56] benbangert	rb_: xmpp? :)
[18:56] rb	benbangert, What is xmpp?
[18:57] benbangert	jabber protocol, just ahve them all go to the same 'channel'
[18:57] benbangert	presence handles indicating what processes are available, they can all send/recieve messages on the channel, and directly
[18:57] benbangert	(assuming you only care about real-time)
[18:58] socksandsandals	rb_: the "bus" would be a message broker of some kind
[18:58] rb	How is it related to 0mq?
[18:59] benbangert	rb_: it isn't, its just a soution to the problem you mentioned, cause that sounds like a better match
[18:59] benbangert	zmq has no such central broker, you'd be implementing it yourself
[18:59] benbangert	especially if you want new processes that join this 'bus' to see messages that occured prior to them joining
[18:59] socksandsandals	rb_: you could do that by having each node have a strong name and one pub socket and one sub socket
[19:00] socksandsandals	the broker then filters on strong names, similar to what a switch would do
[19:00] benbangert	socksandsandals: they'd have to all find each other and link up...
[19:00] socksandsandals	indeed
[19:01] shales	rb_: I wouldn't call it a bus, but I do something similar using one process as a "router" that binds an XREP socket and all the other processes connect to that socket using XREP or XREQ sockets
[19:04] rb	shales, I haven't finished reading the GUIDE , do you mean ROUTER when you say XREQ?
[19:05] rb	socksandsandals, I need request/reply kind of communication so PUB/SUB is not an option
[19:06] socksandsandals	rb_: XREP/XREQ then
[19:06] shales	oh, XREP=ROUTER and XREQ=DEALER
[19:12] rb	I'm thinking to create a customized device that have ROUTER on both ends, or shoud I just use the normal QUEUE device and have each process connect to both ends of the QUEUE device?
[19:13] socksandsandals	you'd need two ROUTERS and two DEALERS, I'd think
[19:13] socksandsandals	one going one way and the other the other
[19:14] socksandsandals	the question in my mind is still: how are the nodes on either side going to find each other?
[19:14] socksandsandals	how will they know which others to address?
[19:15] shales	why does there need to be two "sides" to this central thing? Why can't it bind a single ROUTER socket at a well-known address that all the other processes connect to?
[19:20] shales	as for the addressing problem, in my system all of the processes have fixed identities and my router process knows to route a particular kind of request to identity "foo". If the number of roles of the processes aren't fixed in your system then I guess they'd each have to generate unique identities and send a registration message after connecting to the router.
[19:21] shales	Up to you how processes would discover eachother.
[19:31] CIA-121	pyzmq: 03Min RK 07master * rad1fa9f 10/ (11 files in 5 dirs): Merge pull request #135 from minrk/3.0 ...
[20:02] calvin	in zmq3, can the same XSUB socket send multiple messages to subscribe for multiple filters?
[20:46] rb	shales. I'll try out your single router solution. Thx a lot for your help!
[20:54] rb	socksandsandals, you are right. If I use 2 queues 1 for each direction, the DEALER's load banlancing behavour is still problematic, what I need is broadcasting on the dealer end with which each node can filter out messages destinated to it.

ZeroMq Home

Friday September 16, 2011