Sunday October 10, 2010

[Time] NameMessage
[01:19] emacsen Hi. I'm trying to build the python bindings to zeromq, but they need a constants.c, but that's not part of either the python git repo, nor zeromq's tarball
[01:29] mikko you need cython
[01:29] mikko at least version 0.13
[01:29] mikko python cython
[01:30] mikko python build_ext
[01:30] mikko etc
[01:30] emacsen okay, let me try
[01:30] emacsen wow I need 0.13?
[01:30] emacsen that's quite new
[01:31] jsimmons I've got two zmq sockets in a single process, one per thread with a shared context using the ahhhh, I need to join on the worker thread before terminating the context right that makes sense. cool
[01:38] emacsen mikko, thx for your help
[06:49] jsimmons "When connecting a socket to a peer address using zmq_connect() with the inproc transport, the endpoint shall be interpreted as an arbitrary string identifying the name to connect to. The name must have been previously created by assigning it to at least one socket within the same ØMQ context as the socket being connected."
[06:50] jsimmons This is saying I can't connect to an inproc before I bind to the given name, right?
[06:53] sustrik jsimmons: yes
[06:53] jsimmons damn
[06:53] sustrik this will be solved in the future
[06:53] sustrik but right now it's that way
[06:54] jsimmons sleep ftw then I suppose :P
[06:56] sustrik you can try to patch 0mq yourself as well :)
[07:02] jsimmons I was just thinking of that actually
[07:05] sustrik jsimmons: well, making "delayed bind" work is relatively easy
[07:05] sustrik what's hard is real reconnect
[07:05] sustrik i.e. re-establishing the connection after peer closes the socket and re-opens it
[07:06] jsimmons mm
[07:06] sustrik do you need only the former?
[07:06] jsimmons yeah
[07:06] sustrik ok, i'll send you some patches i have
[07:06] sustrik let me see
[07:09] jsimmons actually I don't think that's the actual problem really, it'd be nice but I still need to sync them up so I don't start publishing before there's an endpoint, and that's more of and an issue for the tests than actual usage.
[07:11] sustrik ok, so you are not interested in the code, right?
[07:11] jsimmons I'm interested, but I probably won't do anything with it so don't waste too much of your time :)
[07:12] sustrik ok then
[07:14] jsimmons it's easy enough to connect after the bind, it's harder to ensure a connect from another thread before the send in the master. It'd just about be easier to add a REQ/REP socket to sync up the start.
[07:16] jsimmons master binds PUB and REP; thread connects SUB and REQ; sends READY over REQ master replies READY and starts broadcasting over PUB.
[07:24] CIA-14 zeromq2: 03Martin Sustrik 07master * r73e7ef3 10/ src/xrep.cpp :
[07:24] CIA-14 zeromq2: When XREP silently drops message, it does not empty it -- fixed
[07:24] CIA-14 zeromq2: Signed-off-by: Martin Sustrik <> -
[07:59] pieterh sustrik: for inproc, we could do without reconnection
[07:59] pieterh the use case is not threads that come and go :-)
[08:00] pieterh the use case is synchronizing thread startup
[08:00] pieterh typical problem is you start two threads that want to talk to each other
[08:19] sustrik ack
[09:32] pieterh sustrik: for issue 94, did you try the two test programs I posted? They're pretty minimal and they reproduce the issue 100% of the time on two different machines here
[09:32] sustrik i've tried ut
[09:32] sustrik it
[09:32] sustrik no luck
[09:33] pieterh ? weird, I tried on two different boxes, fresh clones of zeromq...
[09:33] sustrik may be a timing issue
[09:33] pieterh well... let me try on an eee pc :-)
[09:33] sustrik anyway, the example is ~1000 lines
[09:33] sustrik pretty hard to check
[09:33] pieterh it has to be two programs
[09:34] sustrik maybe you accidentally atteched more files?
[09:34] pieterh zhelpers.h is a standard file, used in dozens of places and not relevant to the particular issue
[09:35] pieterh the two test cases are exactly 200 lines of code together
[09:36] pieterh would it help to have binaries and a core dump?
[09:36] pieterh sorry, zmsg.c...
[09:36] sustrik sure, however build it with debug symbols and no optimisations
[09:36] pieterh i guess that's the default, no?
[09:36] pieterh hang on...
[09:36] sustrik default is -O2
[09:37] pieterh so what arguments give the best executable for debugging?
[09:37] sustrik -g -O0
[09:37] pieterh ack
[09:42] pieterh sustrik: sent
[09:42] pieterh I'm not sure this'll work since I'm working on a 64-bit box, if necessary I'll make a package on a 32-bit box.
[09:42] pieterh Am now going to try on an EEE to clear any possibility it's timing
[09:48] sustrik ph: there are no debug symbols available
[09:48] sustrik i would say the problem is you've linked with libzmq dynamically
[09:50] pieterh ah, no debug symbols in zmq?
[10:01] pieterh sustrik: I've now reproduced the problem on 3 totally different machines, first try
[10:02] pieterh what I'm doing is starting the handler, then running the gateway for two seconds, then Ctrl-C'ing the gateway
[10:02] pieterh however letting the gateway run longer makes no difference
[13:17] andrewvc cremes, I was wondering if you had a minute, I had a zmq_machine question for you
[13:27] andrewvc Anyone know if it's possible to see which filters have been set on a SUB socket via ZMQ_SUBSCRIBE
[13:27] andrewvc or do you need to track that yourself
[13:45] sustrik you need to track it yourself
[14:07] andrewvc sustrik, thanks
[14:31] cremes andrewvc: tracking subscription filters sounds like a good enhancement to zmqmachine... patch?
[14:31] andrewvc yep
[14:31] andrewvc that's what I was thinking
[14:31] andrewvc no patch yet
[14:31] cremes i think i already store that data in a separate array (i'll have to check) so it just needs to be exposed via an accessor
[14:31] andrewvc cool
[14:32] andrewvc pub/sub's been a bit trickier to write specs for btw
[14:33] cremes they are *all* tricky :)
[14:33] andrewvc lol, yes
[14:34] andrewvc I've finally started adding some specs to dripdrop, so I think I'm getting a little better with it, but I was getting weird, non-repeatable things with pub/sub. Not in practice, just specs
[14:34] andrewvc my hunch is that I should connect the pub and bind the sub, since that seems to buffer the messages
[14:38] cremes andrewvc: i have found that sequence to work best for pub/sub
[14:38] cremes most of my practical code uses a forwarder in between since i usually have a few publishers and lots of subscribers
[14:38] cremes the forwarder binds to the necessary ports while both the publishers and subscribers connect to them
[14:39] andrewvc same here
[14:39] cremes oops, gotta run...
[14:39] cremes back tomorrow...
[14:39] andrewvc thanks chuck, see ya around
[14:46] Hosi Hi
[14:46] Hosi i have a question about multi threading,
[14:46] Hosi anyone can answer me?
[14:49] sustrik go on
[17:15] plq hi, when is 2.1 scheduled to be released?
[20:17] petrilli Has anyone ever gotten ZeroMQ and Clojure working together reliably? :)
[20:21] Guthur petrilli, The Common Lisp bindings are good if you fancy using CL instead of clojure
[20:21] Guthur The java bindings are not very mature though
[20:21] petrilli Not really. Yeah, and they seem to combine a lot of ugly things, like JNI :)
[20:22] petrilli I'm reminded why I left java years ago. Alas, CL isn't an acceptable tech for this project
[20:22] Guthur There is a CL implementation that targets the JVM if you are interested
[20:22] Guthur ABCL
[20:22] petrilli Wouldn't that suffer from the same issues? :)
[20:23] Guthur Depends if you see it as an issue, they are language bindings so generally its going to be foreign function calls
[20:23] petrilli I mean, it ends up using JNI to talk to the underlying libraries, correct?
[20:23] Guthur ABCL could in theory use the CL binding
[20:24] Guthur there is one minor issue though
[20:24] Guthur one minor dependency of the CL binding is not supported in ABCL yet
[20:24] Guthur trivial-garbage
[20:24] Guthur Which I had forgot about
[20:25] petrilli Yes, who need GC :)
[20:25] Guthur not sure what trivial-garbage is, obviously CL is a GC language environment
[20:25] Guthur Trivial might add something
[20:26] Guthur
[20:26] petrilli Not sure. Alas, as far as I can tell, there's no effort to implement the ZeroMQ bindings as pure Java, which means that deployment is always going to be VERY brittle.
[20:27] Guthur I'm not surprised, as far as I am aware ZMQ is meant to be language neutral
[20:27] Guthur developing it in Java certainly is not that
[20:27] petrilli Right, but unfortunately, I've yet to find any JNI-based solution that isn't horribly brittle.
[20:27] petrilli language neutral doesn't mean "one implementation"
[20:27] Guthur petrilli, Possibly, the current bindings need a lot of work
[20:28] Guthur brittleness aside
[20:28] petrilli That's an understatement :(
[20:28] Guthur Java binding that is
[20:28] petrilli I'm just surprised, since it seems like usually Java and C are the two that, in most projects, have the best bindings, and everything else is 2nd class.
[20:28] Guthur maybe start implementing it in Java, hehe
[20:29] Guthur I think the lack of motivation in the Java camp maybe because a prejudice towards using Java EE
[20:29] Guthur that's just a hunch though
[20:29] petrilli Could be. I have no interest in Java itself. Just it's easier to "sell" solutions to clients if they run on the JVM
[20:30] petrilli Java holds as much interest and excitement as Ada ;)
[20:30] Guthur hehe, yep i'd agree there
[20:31] Guthur I have to use it at work, not much choice unfortunately
[20:31] petrilli Why solve a problem with a simple solution when I can use XML, DI, AOP, and 42 other acronyms to solve it in a way that nobody can ever use differently because of the amount of "flexibility" I've forced you to deal with.
[20:32] Guthur But it will at least be a multi-acronym compliant solution
[20:32] petrilli Alas, instead of zeromq, which I'd rather use, I'll probably go back to using AMQP
[20:32] petrilli even though it's got other issues, it does work
[20:33] Guthur Hopefully 0mq will keep maturing
[20:33] Guthur it seems a really nice solution
[20:33] petrilli It seems pretty well baked in C and other C-based areas. I've used the Python and Ruby bindings as well
[20:33] Guthur I've only played around some with it, trying to find a real project to use it on
[20:34] Guthur Yeah there is good bindings I think, just Java is the stand out weak point
[20:34] Guthur The CL solution is very good like I said, has all the current features, it does use FFI though
[20:34] Guthur I don't really mind about that though
[20:35] petrilli Kinda shocking that CL is up-to-date on anything :)
[20:35] PerfDave CL's undergoing a bit of a revival these days
[20:35] Guthur hehe, there is quite an active CL community, they just tend to go unnoticed
[20:35] petrilli That's a good thing. the standard was the worstthing that ever happened to Lisp.
[20:36] petrilli As someone who used to own 2 different Lisp machines... it's nice. But in the communities I deal with, Clojure is an easier sell since it fits into most companies "infrastructure" better.
[20:36] Guthur CL would definitely be my tool of choice for programming,
[20:36] Guthur petrilli, Check out ABCL if you can
[20:37] Guthur it is being actively developed
[20:37] petrilli Will do, though I find a lot of pieces in CL to be horrendously ugly.
[20:37] Guthur hehe, I kind of feel the same about Clojure
[20:37] Guthur it's decidedly unlispy in places
[20:37] petrilli Sorta like LISP-1 v. LISP-2
[20:38] petrilli See, it's actually closer to the Lisp I grew up with, INTERLISP
[20:38] Guthur It's those darn [] that put me off
[20:39] petrilli I suppose. That part took a little understanding, but a lot of the concurrency prims make up for it
[20:40] Guthur i think I will try 0mq if I want parallelism
[20:41] petrilli It's useful for certain types, but not others.
[20:41] petrilli I use concurrency for modeling things that aren't particularly designed to "go faster", but simply to make more sense when you look at them
[20:41] petrilli I'm a big believer in having a lot of tools in the tool belt
[20:42] Guthur hehe that's why I like CL
[20:42] Guthur It allows the programmer to be very expressive
[20:42] petrilli CL just seems like going to Williams Sonoma... lots of interesting gadgets, but I haven't the slighest clue when I'd use it :)
[20:42] petrilli But then, that's a pre-post CL view, I guess.
[20:43] Guthur I know what you mean, its only from seeing plenty of CL code that I begun to find a lot of the interesting uses
[20:44] Guthur I just like flexibility, and I love first class functions
[20:45] Guthur Every time I find myself in a language without a simple way of passing functions I weep a little inside, hehe
[20:45] petrilli And honestly, I gave up on CL after 20+ years of stagnation and fragmentation. It's nice to see it's started picking back up. I've not really touched it since AKCL
[20:47] Guthur If nothing else it at least helped me become a better programmer imho
[20:47] Guthur even if I never get to use it on a commercial project
[20:47] petrilli heh
[20:48] petrilli Now i have to find a better solution for messaging than ZeroMQ
[20:48] Guthur happy searching
[20:49] petrilli Yeah, so far only AMQP has all the pieces, but it comes with a bunch of issues.
[20:50] petrilli That's what I liked about ZeroMQ, it seemed to be able to be stripped down light enough to solve more problems.