Friday September 17, 2010

[Time] NameMessage
[01:41] mattp Hi. I'm trying to build using Cygwin. I'm getting me an error message telling me to install e2fsprog even though I've already got it.
[01:41] mattp Any suggestions?
[01:42] mattp (not sure how that "me" got into my third sentence. :) )
[01:49] mattp crickets...
[01:52] mpassell thought I'd try through a different client. Anyone out there?
[03:51] chowmeined can one multiplex different message patterns on a single socket?
[04:18] andrewvc cremes, by chance are you around?
[04:33] dermoth iirc there used to be device config example, looks like they're gone. why?
[04:33] dermoth king of easier than reading the code :)
[04:38] guido_g dermoth: see
[04:38] guido_g maybe thats what you want
[04:40] dermoth guido_g, thanks, looks nice. Actually for now i'm just using the stock devices that come with zeromq. The config quite basic, in xml. I was justt wondering where they gone
[04:41] guido_g afair there going to be moved out of the core distribution
[04:42] dermoth ok... well I kind of like them for the most basic usage... I could replace the config with command-line arguments though, imho it would be much easier
[04:43] guido_g dermoth: btw, please send your patch to the ml with a statement that it is licensed under mit/x11, othwerwise it can't be accepted
[04:43] dermoth really?
[04:44] guido_g it's the procedure
[04:44] guido_g has been discussed here and on the ml
[04:46] guido_g dermoth: easiest way is to use format-patch and append that to the mal w/ the license statement
[04:46] dermoth then how about I sen you this?
[04:46] dermoth perl -pi -e 's/ZMQ_DOWNSTREAM/ZMQ_PUSH/; s/ZMQ_UPSTREAM/ZMQ_PULL/;' devices/zmq_streamer/zmq_streamer.cpp
[04:46] guido_g not me
[04:46] guido_g dermoth: see git format-patch
[04:46] dermoth git commit -m'zmq_streamer: Rename DOWNSTREAM/UPSTREAM to PUSH/PULL'
[04:47] dermoth I use git-format-patch... but I'm not even subscrived to the ml
[04:47] guido_g too bad
[04:47] dermoth the two commands abot will get you the same result less the credit
[04:48] guido_g what credit?
[04:48] dermoth the author in commit message
[04:48] guido_g ah
[04:48] dermoth unless you're authorised to do: git commit -m'zmq_streamer: Rename DOWNSTREAM/UPSTREAM to PUSH/PULL' '--author=Thomas Guyot-Sionnest <>'
[04:48] guido_g i'm more concerned about the license statement, w/o it the patch can't go in
[04:49] dermoth it's not a patch it a command that patsh, there shouldn't be any legal issue. if you run the command you have the copyright for the resulting code ;)
[04:50] guido_g i think that is clear
[04:50] guido_g i just stated what's needed to get a patch in, because a pull-req is a patch
[04:54] dermoth I understand the legal basis, but I will likely not send a patch that takes me more time to "send" thna write... Imagine trying that with the Linux Kernel!
[04:54] guido_g as i said, the device apps will be moved out of the core, afaik
[04:55] guido_g so...
[04:55] dermoth so I guess there won't be any trouble getting that in, whatever method you use... on that i'll get back to my testing :)
[04:55] guido_g dermoth: wrong
[04:55] guido_g the license issue is a problem atm
[04:55] guido_g ask pieter about it
[04:56] guido_g and go read the ml (via gmane w/o registration)
[04:56] dermoth what I meant is that it probably take you less time to rename it yourself than using my patch (and I use gmane...)
[04:57] dermoth i wouldn't mind taking the time to post on the ml for a real change... and can'T I put it in the commit message too?
[04:58] guido_g cool, then you can easily find the threads about patches and legal issues
[04:58] guido_g if it were in the commit message, even better
[04:58] dermoth except that i'd rather catch some much needed sleep
[04:58] dermoth ok
[05:04] dermoth here you go... next time i'll know :)
[05:12] guido_g hey, i didn't invent the laws and such, just repeating the handbook
[05:13] dermoth Any PECL developers here? IMHO the PHP api should have a cb that is called on persistent socket creation, so we could create the socket and connect in one shot
[05:39] sleeperbot anybody know an alternative driver to for node.js?
[05:40] sleeperbot this driver takes up a ton of CPU
[05:42] sleeperbot the way that it is implemented =/
[05:48] yrashk sleeperbot afair there's none
[05:48] yrashk but supposedly I will need one soon-ish
[05:48] yrashk so might invest some time in getting it done properly
[05:51] sleeperbot i see, thanks for the info
[05:52] yrashk yw
[07:02] sustrik psino: how would it work for recv calls?
[07:03] sustrik dermoth: it's a problem with OpenPGM, which will be solved soon; what i was saying is that in most cases you have just one context so you never encounter the problem
[07:04] sustrik chowmeined: no, you can't
[08:49] ivan-korobkov Hi, I use zmq 2.0.9 + epgm + the latest python bindings. Sometimes it crashes with Assertion failed: *tmpbuf > 0 (zmq_decoder.cpp:60). Does anybody know why it happens? Is it a bug and I should create an issue?
[09:08] guido_g try to isolate it in a small test case, so others can verify it
[09:12] ivan-korobkov OK, I will try, but it happens rather sporadically.
[09:13] guido_g will be no problem, if we have a program that can run
[09:31] psino sustrik: In the case of a pull socket, I was thinking it could disconnect from the push socket and still be able to recv() on the messages the push socket has sent to it
[09:40] sustrik ivan-korobkov: create an issue
[09:41] sustrik psino: any idea of how the API should look like?
[09:46] psino sustrik: from the end users point of view, much like now, except that after calling socket.close(), a poller wouldn't put the socket in exceptional state before the messages that has been sent to it from the other side has been recv()'ed
[10:32] CIA-20 zeromq2: 03Martin Sustrik 07master * r85aa25e 10/ src/req.cpp : bug in REQ+multipart fixed -
[10:58] CIA-20 zeromq2: 03Martin Sustrik 07master * rc7b9ba3 10/ (src/command.hpp src/session.cpp): reconnection process fixed when failure occurs during init phase -
[20:02] bgranger sustrik: question about msg ref counting...
[20:11] keffo I think they're all off work :)
[20:11] keffo starting to think I'm the only one around anymore :)
[20:12] keffo lol, last time anyone even spoke in here was about 11h ago :)
[20:12] andrewvc and you just broke the record
[20:12] keffo no, bgranger did 10 mins ago :)
[20:14] andrewvc oh damn, hehe, that's what I get for sleeping my laptop, missing all the action
[20:16] keffo :)
[20:17] keffo Does anyone know best-practices for transferring large amounts of data? Simply to split into multipart?
[20:17] andrewvc I remember there were benchmarks on the zeromq site with various message sizes, ranging up to something huge
[20:18] keffo right, but simply sending one huge message just sounds like mem hog to me
[20:32] sustrik i am back
[20:32] sustrik bgranger: what's the question?
[20:34] sustrik keffo: so is multipart message, you have to break it into sequence of messages
[20:34] sustrik then you have to define your own transfer protocol etc.
[20:34] sustrik it's not easy
[20:34] keffo mm
[20:35] sustrik what do you want to do?
[20:35] keffo To be honest I'm leaning towards an accompanying torrent lib for larger transfers
[20:35] keffo I solved both my deadlock and garbled data btw..
[20:35] keffo as expected it was all me :)
[20:36] keffo CRT mismatch actually
[20:36] sustrik :)
[20:36] sustrik yeah, makes sense, you would have to implement torrent kind of behaiour yourself on top of 0mq
[20:36] bgranger sustrik: we have a very subtle question about ref counting of msgs...
[20:37] keffo sustrik, I was thinking more along the lines of using the transmission codebase, or libtorrent..
[20:37] sustrik bgranger: yes?
[20:38] bgranger Hi
[20:38] sustrik keffo: sure
[20:38] sustrik hi
[20:38] bgranger We are trying to understand the refcounting
[20:38] bgranger We see that zmq_msg_close is the place where the ffn is called if the ref count is 0
[20:39] bgranger What exactly does zmq_send do with the ref count. What is zmq_close is called *before* a msg is actually send?
[20:39] bgranger Does send somehow end up calling zmq_msg_close?
[20:42] sustrik yes, send takes ownership of your message
[20:43] bgranger OK, we just saw that at the end of xsend there is
[20:43] sustrik your message object will be empty after return from the send call
[20:43] bgranger int rc = zmq_msg_init (msg_);
[20:43] sustrik yes, that initialises your message to empty
[20:43] bgranger So if you then call zmq_msg_close you are not closing the original msg, but a new empty one.
[20:43] bgranger Great!
[20:43] sustrik exactly
[20:43] bgranger We are getting python to know when all copies of a message are done being sent.
[20:44] sustrik intercepting the free function?
[20:44] bgranger There is a subtle issue with non-copy sends. The user has to be able to know when zmq is done with it and they can edit the buffer again.
[20:44] bgranger Yes, we use the ffn.
[20:45] bgranger Otherwise the user might accidentally start the edit the buffer they sent before zmq has send it...
[20:45] bgranger non-copy sends are super subtle...
[20:45] sustrik so you are basically writing your own allocator
[20:46] bgranger We use Pythons garbage collection mechanism along with our own ref counts that allow us to track the underlying zmq ref counts.
[20:46] sustrik what's the goal?
[20:47] bgranger In Python you might have a mutable array of numbers that you want to send using non-copy.
[20:47] bgranger You send it, and maybe then you want to modify it.
[20:47] sustrik i see
[20:47] bgranger But if you modify it before zmq sends it, the wrong thing gets sent
[20:47] bgranger This is common in Python with numerical arrays (matrices, etc.)
[20:48] bgranger For large static data it is not a problem, but in numerical/scientific computing, most data structures are mutable...
[20:49] bgranger sustrik: I think we understand the refcounting better now though. Thanks!
[20:50] sustrik np
[20:51] sustrik a question though:
[20:51] sustrik doesn't it turn the messaging into sync model?
[20:51] sustrik one-message-a-time?
[21:08] bgranger sustrik: back from lunch...
[21:09] bgranger no the messaging is still fully async
[21:09] minrk sustrik: I'm not sure what your question means. All we did was add an object that tracks whether a message no longer has references in zmq
[21:09] bgranger You can then query that object to learn if the message has fully been sent.
[21:10] bgranger send does not block any more than it used to
[21:10] minrk or if there are any message objects pointing to it at all
[21:11] bgranger minrk: it would be nice if we could have a method of MessageTracker that would block until the msg is not pending though.
[21:12] minrk that's easy
[21:13] bgranger minrk: I am ready when you are....just give me a call on skype
[21:13] minrk okay
[21:14] minrk give me a few minutes
[21:15] bgranger no problem...
[21:20] minrk okay
[21:22] mrossi anyone know who is on the sending side of the signaler instance inside of the app_thread class?
[21:24] mrossi have an app that occasionally gets hung at line 88 of app_thread.cpp doing a signaler.recv()
[21:38] dbudworth question from a newbie zeromq user. I'd like to implement a socket server type system where the server is stateful for each client connection. Is there a zeromq socket type for this? it appears as though the server doesn't have any context about which client it's sending to at any one time
[22:51] ModusPwnens Hmm, can anyone take a look at some results from benchmarking tests?
[22:52] ModusPwnens I am seeing some strange results, but maybe they are normal
[22:52] ModusPwnens