ZeroMq IRC Log

Sunday October 10, 2010

[Time] Name	Message
[01:19] emacsen	Hi. I'm trying to build the python bindings to zeromq, but they need a constants.c, but that's not part of either the python git repo, nor zeromq's tarball
[01:29] mikko	you need cython
[01:29] mikko	at least version 0.13
[01:29] mikko	python setup.py cython
[01:30] mikko	python setup.py build_ext
[01:30] mikko	etc
[01:30] emacsen	okay, let me try
[01:30] emacsen	wow I need 0.13?
[01:30] emacsen	that's quite new
[01:31] jsimmons	I've got two zmq sockets in a single process, one per thread with a shared context using the ahhhh, I need to join on the worker thread before terminating the context right that makes sense. cool
[01:38] emacsen	mikko, thx for your help
[06:49] jsimmons	"When connecting a socket to a peer address using zmq_connect() with the inproc transport, the endpoint shall be interpreted as an arbitrary string identifying the name to connect to. The name must have been previously created by assigning it to at least one socket within the same ÃMQ context as the socket being connected."
[06:50] jsimmons	This is saying I can't connect to an inproc before I bind to the given name, right?
[06:53] sustrik	jsimmons: yes
[06:53] jsimmons	damn
[06:53] sustrik	this will be solved in the future
[06:53] sustrik	but right now it's that way
[06:54] jsimmons	sleep ftw then I suppose :P
[06:56] sustrik	you can try to patch 0mq yourself as well :)
[07:02] jsimmons	I was just thinking of that actually
[07:05] sustrik	jsimmons: well, making "delayed bind" work is relatively easy
[07:05] sustrik	what's hard is real reconnect
[07:05] sustrik	i.e. re-establishing the connection after peer closes the socket and re-opens it
[07:06] jsimmons	mm
[07:06] sustrik	do you need only the former?
[07:06] jsimmons	yeah
[07:06] sustrik	ok, i'll send you some patches i have
[07:06] sustrik	let me see
[07:09] jsimmons	actually I don't think that's the actual problem really, it'd be nice but I still need to sync them up so I don't start publishing before there's an endpoint, and that's more of and an issue for the tests than actual usage.
[07:11] sustrik	ok, so you are not interested in the code, right?
[07:11] jsimmons	I'm interested, but I probably won't do anything with it so don't waste too much of your time :)
[07:12] sustrik	ok then
[07:14] jsimmons	it's easy enough to connect after the bind, it's harder to ensure a connect from another thread before the send in the master. It'd just about be easier to add a REQ/REP socket to sync up the start.
[07:16] jsimmons	master binds PUB and REP; thread connects SUB and REQ; sends READY over REQ master replies READY and starts broadcasting over PUB.
[07:24] CIA-14	zeromq2: 03Martin Sustrik 07master * r73e7ef3 10/ src/xrep.cpp :
[07:24] CIA-14	zeromq2: When XREP silently drops message, it does not empty it -- fixed
[07:24] CIA-14	zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/dh6Ujo
[07:59] pieterh	sustrik: for inproc, we could do without reconnection
[07:59] pieterh	the use case is not threads that come and go :-)
[08:00] pieterh	the use case is synchronizing thread startup
[08:00] pieterh	typical problem is you start two threads that want to talk to each other
[08:19] sustrik	ack
[09:32] pieterh	sustrik: for issue 94, did you try the two test programs I posted? They're pretty minimal and they reproduce the issue 100% of the time on two different machines here
[09:32] sustrik	i've tried ut
[09:32] sustrik	it
[09:32] sustrik	no luck
[09:33] pieterh	? weird, I tried on two different boxes, fresh clones of zeromq...
[09:33] sustrik	may be a timing issue
[09:33] pieterh	well... let me try on an eee pc :-)
[09:33] sustrik	anyway, the example is ~1000 lines
[09:33] sustrik	pretty hard to check
[09:33] pieterh	it has to be two programs
[09:34] sustrik	maybe you accidentally atteched more files?
[09:34] pieterh	zhelpers.h is a standard file, used in dozens of places and not relevant to the particular issue
[09:35] pieterh	the two test cases are exactly 200 lines of code together
[09:36] pieterh	would it help to have binaries and a core dump?
[09:36] pieterh	sorry, zmsg.c...
[09:36] sustrik	sure, however build it with debug symbols and no optimisations
[09:36] pieterh	i guess that's the default, no?
[09:36] pieterh	hang on...
[09:36] sustrik	default is -O2
[09:37] pieterh	so what arguments give the best executable for debugging?
[09:37] sustrik	-g -O0
[09:37] pieterh	ack
[09:42] pieterh	sustrik: sent
[09:42] pieterh	I'm not sure this'll work since I'm working on a 64-bit box, if necessary I'll make a package on a 32-bit box.
[09:42] pieterh	Am now going to try on an EEE to clear any possibility it's timing
[09:48] sustrik	ph: there are no debug symbols available
[09:48] sustrik	i would say the problem is you've linked with libzmq dynamically
[09:50] pieterh	ah, no debug symbols in zmq?
[10:01] pieterh	sustrik: I've now reproduced the problem on 3 totally different machines, first try
[10:02] pieterh	what I'm doing is starting the handler, then running the gateway for two seconds, then Ctrl-C'ing the gateway
[10:02] pieterh	however letting the gateway run longer makes no difference
[13:17] andrewvc	cremes, I was wondering if you had a minute, I had a zmq_machine question for you
[13:27] andrewvc	Anyone know if it's possible to see which filters have been set on a SUB socket via ZMQ_SUBSCRIBE
[13:27] andrewvc	or do you need to track that yourself
[13:45] sustrik	you need to track it yourself
[14:07] andrewvc	sustrik, thanks
[14:31] cremes	andrewvc: tracking subscription filters sounds like a good enhancement to zmqmachine... patch?
[14:31] andrewvc	yep
[14:31] andrewvc	that's what I was thinking
[14:31] andrewvc	no patch yet
[14:31] cremes	i think i already store that data in a separate array (i'll have to check) so it just needs to be exposed via an accessor
[14:31] andrewvc	cool
[14:32] andrewvc	pub/sub's been a bit trickier to write specs for btw
[14:33] cremes	they are all tricky :)
[14:33] andrewvc	lol, yes
[14:34] andrewvc	I've finally started adding some specs to dripdrop, so I think I'm getting a little better with it, but I was getting weird, non-repeatable things with pub/sub. Not in practice, just specs
[14:34] andrewvc	my hunch is that I should connect the pub and bind the sub, since that seems to buffer the messages
[14:38] cremes	andrewvc: i have found that sequence to work best for pub/sub
[14:38] cremes	most of my practical code uses a forwarder in between since i usually have a few publishers and lots of subscribers
[14:38] cremes	the forwarder binds to the necessary ports while both the publishers and subscribers connect to them
[14:39] andrewvc	same here
[14:39] cremes	oops, gotta run...
[14:39] cremes	back tomorrow...
[14:39] andrewvc	thanks chuck, see ya around
[14:46] Hosi	Hi
[14:46] Hosi	i have a question about multi threading,
[14:46] Hosi	anyone can answer me?
[14:49] sustrik	go on
[17:15] plq	hi, when is 2.1 scheduled to be released?
[20:17] petrilli	Has anyone ever gotten ZeroMQ and Clojure working together reliably? :)
[20:21] Guthur	petrilli, The Common Lisp bindings are good if you fancy using CL instead of clojure
[20:21] Guthur	The java bindings are not very mature though
[20:21] petrilli	Not really. Yeah, and they seem to combine a lot of ugly things, like JNI :)
[20:22] petrilli	I'm reminded why I left java years ago. Alas, CL isn't an acceptable tech for this project
[20:22] Guthur	There is a CL implementation that targets the JVM if you are interested
[20:22] Guthur	ABCL
[20:22] petrilli	Wouldn't that suffer from the same issues? :)
[20:23] Guthur	Depends if you see it as an issue, they are language bindings so generally its going to be foreign function calls
[20:23] petrilli	I mean, it ends up using JNI to talk to the underlying libraries, correct?
[20:23] Guthur	ABCL could in theory use the CL binding
[20:24] Guthur	there is one minor issue though
[20:24] Guthur	one minor dependency of the CL binding is not supported in ABCL yet
[20:24] Guthur	trivial-garbage
[20:24] Guthur	Which I had forgot about
[20:25] petrilli	Yes, who need GC :)
[20:25] Guthur	not sure what trivial-garbage is, obviously CL is a GC language environment
[20:25] Guthur	Trivial might add something
[20:26] Guthur	http://www.cliki.net/trivial-garbage
[20:26] petrilli	Not sure. Alas, as far as I can tell, there's no effort to implement the ZeroMQ bindings as pure Java, which means that deployment is always going to be VERY brittle.
[20:27] Guthur	I'm not surprised, as far as I am aware ZMQ is meant to be language neutral
[20:27] Guthur	developing it in Java certainly is not that
[20:27] petrilli	Right, but unfortunately, I've yet to find any JNI-based solution that isn't horribly brittle.
[20:27] petrilli	language neutral doesn't mean "one implementation"
[20:27] Guthur	petrilli, Possibly, the current bindings need a lot of work
[20:28] Guthur	brittleness aside
[20:28] petrilli	That's an understatement :(
[20:28] Guthur	Java binding that is
[20:28] petrilli	I'm just surprised, since it seems like usually Java and C are the two that, in most projects, have the best bindings, and everything else is 2nd class.
[20:28] Guthur	maybe start implementing it in Java, hehe
[20:29] Guthur	I think the lack of motivation in the Java camp maybe because a prejudice towards using Java EE
[20:29] Guthur	that's just a hunch though
[20:29] petrilli	Could be. I have no interest in Java itself. Just it's easier to "sell" solutions to clients if they run on the JVM
[20:30] petrilli	Java holds as much interest and excitement as Ada ;)
[20:30] Guthur	hehe, yep i'd agree there
[20:31] Guthur	I have to use it at work, not much choice unfortunately
[20:31] petrilli	Why solve a problem with a simple solution when I can use XML, DI, AOP, and 42 other acronyms to solve it in a way that nobody can ever use differently because of the amount of "flexibility" I've forced you to deal with.
[20:32] Guthur	But it will at least be a multi-acronym compliant solution
[20:32] petrilli	Alas, instead of zeromq, which I'd rather use, I'll probably go back to using AMQP
[20:32] petrilli	even though it's got other issues, it does work
[20:33] Guthur	Hopefully 0mq will keep maturing
[20:33] Guthur	it seems a really nice solution
[20:33] petrilli	It seems pretty well baked in C and other C-based areas. I've used the Python and Ruby bindings as well
[20:33] Guthur	I've only played around some with it, trying to find a real project to use it on
[20:34] Guthur	Yeah there is good bindings I think, just Java is the stand out weak point
[20:34] Guthur	The CL solution is very good like I said, has all the current features, it does use FFI though
[20:34] Guthur	I don't really mind about that though
[20:35] petrilli	Kinda shocking that CL is up-to-date on anything :)
[20:35] PerfDave	CL's undergoing a bit of a revival these days
[20:35] Guthur	hehe, there is quite an active CL community, they just tend to go unnoticed
[20:35] petrilli	That's a good thing. the standard was the worstthing that ever happened to Lisp.
[20:36] petrilli	As someone who used to own 2 different Lisp machines... it's nice. But in the communities I deal with, Clojure is an easier sell since it fits into most companies "infrastructure" better.
[20:36] Guthur	CL would definitely be my tool of choice for programming,
[20:36] Guthur	petrilli, Check out ABCL if you can
[20:37] Guthur	it is being actively developed
[20:37] petrilli	Will do, though I find a lot of pieces in CL to be horrendously ugly.
[20:37] Guthur	hehe, I kind of feel the same about Clojure
[20:37] Guthur	it's decidedly unlispy in places
[20:37] petrilli	Sorta like LISP-1 v. LISP-2
[20:38] petrilli	See, it's actually closer to the Lisp I grew up with, INTERLISP
[20:38] Guthur	It's those darn [] that put me off
[20:39] petrilli	I suppose. That part took a little understanding, but a lot of the concurrency prims make up for it
[20:40] Guthur	i think I will try 0mq if I want parallelism
[20:41] petrilli	It's useful for certain types, but not others.
[20:41] petrilli	I use concurrency for modeling things that aren't particularly designed to "go faster", but simply to make more sense when you look at them
[20:41] petrilli	I'm a big believer in having a lot of tools in the tool belt
[20:42] Guthur	hehe that's why I like CL
[20:42] Guthur	It allows the programmer to be very expressive
[20:42] petrilli	CL just seems like going to Williams Sonoma... lots of interesting gadgets, but I haven't the slighest clue when I'd use it :)
[20:42] petrilli	But then, that's a pre-post CL view, I guess.
[20:43] Guthur	I know what you mean, its only from seeing plenty of CL code that I begun to find a lot of the interesting uses
[20:44] Guthur	I just like flexibility, and I love first class functions
[20:45] Guthur	Every time I find myself in a language without a simple way of passing functions I weep a little inside, hehe
[20:45] petrilli	And honestly, I gave up on CL after 20+ years of stagnation and fragmentation. It's nice to see it's started picking back up. I've not really touched it since AKCL
[20:47] Guthur	If nothing else it at least helped me become a better programmer imho
[20:47] Guthur	even if I never get to use it on a commercial project
[20:47] petrilli	heh
[20:48] petrilli	Now i have to find a better solution for messaging than ZeroMQ
[20:48] Guthur	happy searching
[20:49] petrilli	Yeah, so far only AMQP has all the pieces, but it comes with a bunch of issues.
[20:50] petrilli	That's what I liked about ZeroMQ, it seemed to be able to be stripped down light enough to solve more problems.