ZeroMq IRC Log

Wednesday July 13, 2011

[Time] Name	Message
[00:00] CIA-32	pyzmq: 03Stefan van der Walt 07master * ra6d9846 10/ zmq/core/context.pyx : Do not attempt to acquire the GIL when it is already held. - http://bit.ly/rdRQbA
[02:01] gdan	if you are using the push pull fan-out, does only one of the pull sockets receive a message per message pushed
[02:03] whack	gdan: I think that's the idea, yeah
[02:05] gdan	i wonder with a lot of nodes to push to... if you use tcp, the push socket will choose one path to send to... thus there is no bandwidth saving using tcp over pgm
[02:07] whack	well with push only one receiver gets each message; so multicast vs tcp shouldn't matter
[02:07] whack	if you have 10 recievers, 1 message, 1 receiver gets it
[02:10] gdan	thanks, I appreciate. i thought it was something like that reading the guide but it wasn't spelled out as that
[02:10] gdan	as clearly as that...
[06:01] c_nick	Can someone help me with this http://ideone.com/JQiia
[10:31] omarkj	Is exposing zeromq to the internet safe?
[10:32] omarkj	I remember there being an issue with that?
[11:54] guido_g	not using pair sockets in the first place
[11:56] guido_g	there're a lot of things "wrong" with them
[11:56] guido_g	not a usage pattern, no re-connect and so on
[11:57] guido_g	http://api.zeromq.org/2-1:zmq-socket <- there is even a big warning
[11:58] guido_g	alternatives depend on what you want to achieve
[12:00] guido_g	case closed :)
[12:00] guido_g	don't miss the guide! http://zguide.zeromq.org/page:all explains a lot, really
[12:08] reuben	omg
[12:09] reuben	theY4Kman`lab!
[12:10] reuben	didn't know you were using 0MQ :)
[12:10] reuben	haha
[12:10] reuben	yes it is :)
[12:11] reuben	I'm gonna do a talk about it on PyCon Brasil later this year
[12:13] reuben	I'm in the process of convincing the people at work to use 0MQ
[12:13] reuben	writing a groovy binding, showing cool examples, etc
[12:14] reuben	https://gist.github.com/1018752
[12:14] reuben	just mapping technology they're already familiar with
[12:16] reuben	theY4Kman`lab, is that for the kinect minecraft generation thing?
[12:18] reuben	cool
[12:25] reuben	yeah that's groovy
[12:26] reuben	whoa, cool stuff
[12:26] reuben	I didn't know about SIGGRAPH
[12:36] kontinuity	hey folks
[12:37] kontinuity	we were looking for a xrep/xreq client server example
[12:37] kontinuity	without a dealer router
[12:37] kontinuity	is that pattern possible?
[12:48] reuben	kontinuity, yes
[12:48] reuben	http://pastie.org/2206980
[12:49] reuben	the weird messages are the metadata needed for XREQ/XREP stuff
[12:49] reuben	AFAIK
[12:50] kontinuity	reuben: probably the routing data
[12:50] kontinuity	:)
[12:50] guido_g	it's the socket id as assumed by the receiver
[12:50] reuben	yeah
[12:50] guido_g	yes, for routing purposes
[12:50] kontinuity	guido_g: how do know the valid messages from them?
[12:51] guido_g	pardon?
[12:51] reuben	if it doesn't start with 0x00?
[12:51] kontinuity	guido_g: in reuben's example he did a standard recv
[12:51] kontinuity	guido_g: how do we know its the socket or real data
[12:52] guido_g	it's written in the xrep/xreq spec and also discussed in detail in the guide
[12:53] kontinuity	guido_g: thanks
[12:56] kontinuity	reuben: the example looks more like PUSH-PULL
[12:56] guido_g	btw, i'm somwhat shocked that the example above works
[12:56] kontinuity	guido_g: precisely :)
[12:56] reuben	why?
[12:56] kontinuity	guido_g: but not that it works or not but rather where is the async rep in the example?
[12:56] guido_g	should be one multipart message
[12:57] guido_g	ah forget it, my fault
[13:04] kontinuity	guido_g: how do we use timeouts with rep/req?
[13:05] guido_g	you don't
[13:05] guido_g	you've to use poll for that
[13:15] kontinuity	guido_g: using poll now thanks. In case of a req for which we may have no rep how do skip and go to next
[13:15] kontinuity	?
[13:15] reuben	guido_g, it is a multipart message
[13:15] reuben	:)
[13:16] guido_g	yep
[13:16] reuben	kontinuity, I don't think you can, but can't you just send an empty message?
[13:16] guido_g	forgot that the plain recv fetches whatever is there
[13:16] guido_g	damn wrappers! :)
[13:17] kontinuity	reuben: in our case the server may have bounced and the client will wait for something that the server has lost reference to
[13:17] reuben	hm
[13:18] kontinuity	reuben: should the client reinit the connection?
[13:19] reuben	I don't know, that's probably what I would do in this case
[13:20] guido_g	http://zguide.zeromq.org/page:all#Chapter-Three-Advanced-Request-Reply-Patterns <- there might be some more info
[13:22] kontinuity	guido_g: saw it superficially but we don't want to introduce a dealer in between
[13:22] kontinuity	guido_g: just use req-rep or xreq/xrep with recovery from failed reps
[13:23] kontinuity	guido_g: its a fairly simple client - server with ack problem we have ;)
[13:23] kontinuity	guido_g: and missing acks as I mentioned above
[13:33] stodge	I'm trying to get epgm working with pyzmq. I installed zeromq 3.0 from source using ran ./configure --with-pgm. This built and installed ok. I then did easy_install pyzmq and that installed. But my simple test app tells me that epgm isn't supported. Did I miss a step?
[13:38] guido_g	might be a problem w/ Ã¸mq3 vs. pyzmq
[13:38] guido_g	if you don't need the 3.0 features, try to use the latest stable
[13:49] stodge	I did - it didn't work but let me try the stable release again
[13:51] guido_g	you should try the latest pyzmq from github and build from source
[13:51] stodge	I tried that but it didnt build for me
[13:51] stodge	I cloned the repo, instead of using a download
[13:52] stodge	This is on Fedora 14 64bit
[13:52] stodge	With zeromq stable and latest pyzqm through easy_install I get "protocol not supported"
[13:52] stodge	Let me try to build pyzqm from source again
[13:52] stodge	Bah - my typing/spelling is off
[13:55] stodge	Sorry - pyzqm did build from source, but I get a runtime error undefined symbol zmq_sendmsg
[13:55] reuben	stodge, grab 0MQ 3.0, build it with PGM enabled, grab pyzmq from github, build it with PGM enabled too
[13:56] reuben	or 0MQ stable and pyzmq stable if you need PGM on 2.1.7
[13:56] reuben	0MQ builds with PGM disabled by default
[13:57] stodge	Building pyzmq is done with python setup.py build ?
[13:58] reuben	just do python setup.py install
[13:58] reuben	it'll build and install
[13:59] stodge	How do I specify "with pgm"?
[13:59] reuben	--with-openpgm IIRC
[13:59] reuben	let me check
[13:59] reuben	--with-pgm
[13:59] reuben	stodge, ^
[14:00] reuben	or --with-system-pgm if you want to use openpgm from the package management
[14:00] stodge	Right when building zeromq, but python setup.py install doesn't accept --with-pgm
[14:00] stodge	Or am I just being incredibly dense today?
[14:00] reuben	you shouldn't need anything special to have PGM on pyzmq
[14:00] reuben	as long as 0MQ has it enabled
[14:01] stodge	Ok let me build stable zeromq and pyzmq from source
[14:05] stodge	Still get protocol not supported
[14:05] guido_g	please show the code you're using
[14:06] stodge	I'll have to add it to my issue on the pyzmq github site - I'm using different machines for IRC and dev
[14:06] guido_g	jsut use a paste bin
[14:07] stodge	ok
[14:07] guido_g	ahh, read my comment on your issue
[14:07] reuben	stodge, I just built it here and it worked fine
[14:07] guido_g	you're using a wrong address for multicast
[14:07] reuben	I didn't even have to rebuild pyzmq
[14:08] guido_g	ack
[14:08] stodge	I put the multicast address in instead of localhost. I used to have the mc address but for some reason took it out
[14:08] guido_g	it's nevertheless wrong
[14:08] stodge	http://pastebin.com/ukM91ZgE
[14:08] guido_g	and try to avoid interface names, that's also somewhat flaky
[14:09] reuben	http://pastie.org/2207313
[14:09] guido_g	copy&paste works here
[14:09] reuben	I had pyzmq installed with \|sudo pip install pyzmq\| before
[14:10] stodge	hmmm
[14:10] stodge	zeromq installs to /usr/local/lib? I better make sure I don't have multiple copies of zeromq lying around
[14:10] guido_g	omg
[14:10] reuben	everything defaults to /usr/local
[14:10] reuben	lol
[14:11] stodge	I don't thikn I have multiple copies - I can only find the one I built
[14:14] reuben	http://pastie.org/2207340 <- make sure you do it like that
[14:14] reuben	uninstall the versions you already have before doing that
[14:15] stodge	ok
[14:20] stodge	I don't have pip installed but easy_install should work
[14:21] stodge	Oh - when I do easy_install pyzmq I see a message that says that ZMQ 2.1.4 was detected. I don't have that version installed anywhere
[14:23] stodge	Crap - I yum removed all zmq stuff except I forgot to remove the zeromq package itself... :P
[14:24] guido_g	sigh
[14:24] reuben	haha
[14:24] stodge	I need more caffeine
[14:24] reuben	stodge, if you don't have pip, here's another useful command for you:
[14:24] reuben	sudo easy_install pip
[14:24] kontinuity	guido_g: is there any documentation on what each socket type does? I am seeing examples like REQ - XREP \| REP - REP \| REQ - REQ
[14:24] kontinuity	totally confused
[14:25] kontinuity	XREQ - ZREQ*
[14:25] kontinuity	XREQ - XREQ**
[14:25] reuben	XREP and XREQ are routers and dealers, they can interoperate with most socket types
[14:25] stodge	Of course it works now hides in corner in shame
[14:25] stodge	Thanks
[14:25] guido_g	there is the reference and the guide
[14:25] guido_g	that should do
[14:25] reuben	yes
[14:25] guido_g	but you have to read it
[14:26] reuben	you should really read ze guide
[14:26] stodge	Now to get the consumer to read correctly...
[14:27] reuben	it's kinda hard to do that, pieter did an amazing job :)
[14:34] stodge	Are there any example of pyzmq & pgm?
[14:36] guido_g	not really
[14:37] guido_g	but there are close to no examples for (e)pgm anyway
[14:37] stodge	:)
[14:37] stodge	receiver isn't receiving anything
[14:38] guido_g	check multicast routing and mc loopback flag
[14:38] guido_g	the later only if sender and receiver are on the same machine
[14:39] stodge	Ah it is the same machine
[14:41] reuben	it worked just fine for me, on the same machine
[14:42] reuben	http://pastie.org/2207313
[14:42] reuben	:P
[14:42] guido_g	yeah, default is "do loopback"
[14:43] guido_g	i guess the subscribe is missing or the the subscriber was activated to late
[14:45] stodge	I'm still trying to find out how to change that flag - you guys are fast :)
[14:45] guido_g	setsockopt
[14:52] stodge	That pastie code doesn't work for me - the code sits there doing nothing. I'll have to check my network config
[15:12] stodge	membership join requests are being sent but no data is received. Ah well; need to move onto something else for the day. Thanks
[15:19] pieter_hintjens	sustrik: ping
[15:20] pieter_hintjens	any chance of backporting https://zeromq.jira.com/browse/LIBZMQ-228 to 2.1?
[15:32] pieter_hintjens	ok, done it myself
[16:29] ssi	is there a significant performance difference between sending a 2 part x byte per part message vs a 1 part 2x byte message?
[16:37] mikko	ssi: the latter might be slightly faster
[16:55] cremes	ssi: one less iteration of the loop to read in a message part
[17:01] rev	does zmq 3.0 support inproc for windows? i googled and got nothing and there is no mention in the change log. i noticed new msvc project files that has me curious.
[17:10] ssi	ugh these patterns get out of control in a damned hurry
[17:44] rev	it's awfully quiet in here.
[17:44] ssi	it usually is :)
[17:45] rev	perhaps you can answer my question then
[17:45] ssi	I wasn't aware there was a 3.0, tbh
[17:45] ssi	and I haven't touched zmq on windows at all
[17:45] rev	oh hehe
[17:45] rev	that explains the silence
[17:45] rev	hehe
[17:45] ssi	3.0 happened yesterday :D
[17:46] rev	yeah
[17:46] ssi	if you're that new, you probably should run with 2.1.7 for now
[17:46] rev	thats why i thought it would be more chatty in here
[17:46] rev	i am running 2.1.7
[17:46] rev	i'm trying to decide if the upgrade is worth the effort
[17:46] ssi	beats me! :D
[17:46] rev	normally i'd let the dust settle
[17:47] rev	it looks like they sneaked inproc in for windows ... surprised me tho -
[17:47] ssi	2.x doesn't support inproc in windows?
[17:47] rev	nope
[17:48] ssi	hooboy
[17:48] rev	something about "it's complicated"
[17:48] rev	hehe
[17:48] ssi	that's gonna cause me some issues :D
[17:48] rev	bazinga
[17:48] ssi	the first thingy that I wrote relies entirely on inproc
[17:48] ssi	and I dev on OSX
[17:48] ssi	but I'm gonna have to make it so that our worthless windows developers can use it
[17:48] ssi	that's gonna make my life hard
[17:49] rev	you're going to have to use sockets on localhost
[17:49] rev	it works really well
[17:49] rev	but its not inproc
[17:49] ssi	I wonder if I can sense and switch internally
[17:49] ssi	cause it'll all be deployed on linux, and I want the performance
[17:49] rev	probably
[17:50] rev	well, hehe - i came in looking for help and helped someone else. go figure
[17:51] harris	hi guys!
[17:51] harris	is there any way to do fail over in zmq ?
[17:51] ssi	rev: that's how it goes!
[17:51] ssi	harris: it's all in the guide! (just warning you cause that's what martin and pieter will say)
[17:52] harris	ssi: thanks I will dig in then
[17:59] panu	Hi. Anybody knows any know reason why zmq_poll might not alter the poll item lista at all (I am using 0mq version 2.1.7)?
[18:24] pieter_hintjens	panu: either the timeout expired, or there was an error, or there was an interrupt
[18:24] pieter_hintjens	check for rc != 0 and then check errno
[18:50] panu	pieter_hintjens: poll returns positive value..
[18:51] panu	I am writing bindings from scratch so this is most likely problem in my code
[18:52] pieter_hintjens	panu: interesting, and revents is zero on all items?
[18:52] panu	No... revents and events seems to be unchanged.
[18:58] ssi	argh... assertions D:
[18:58] ssi	Assertion failed: (exception_class), function raise_exception, file util.cpp, line 33.
[18:58] ssi	this one's in jzmq it seems
[18:58] ssi	jclass exception_class = env->FindClass ("org/zeromq/ZMQException");
[18:58] ssi	assert (exception_class);
[19:19] panu	If the socket pointer passed to poll in polled item structure is rubbish should 0mq detect this or will it just behave odd?
[21:34] pauldix	hi all. wondering if anyone can help. I'm working with a XREQ and REP socket combo in Ruby and the REP isn't reading correctly
[21:34] pauldix	code and output here: https://gist.github.com/1081387
[21:35] pauldix	please to help
[21:35] pauldix	<3