Wednesday April 28, 2010

[Time] NameMessage
[00:28] mike8901 Is it possible to do a read and a write simultaneously to a socket without breaking anything?
[00:31] mike8901 I am doing nonblocking reads, if that makes any difference.
[02:00] CIA-15 pyzmq: 03Brian E. Granger 07master * rb0352c8 10/ (zmq/_zmq.c zmq/_zmq.pyx):
[02:00] CIA-15 pyzmq: Fixing ctypedef of int64_t.
[02:00] CIA-15 pyzmq: Thanks to Lisandro Dalcin for the tip. -
[02:03] CIA-15 pyzmq: 03Fernando Perez 07master * r811389d 10/ (examples/pubsub/ examples/pubsub/ New example for publish/subscribe sockets with multiple topics. -
[02:12] rmancy Hi, does anyone know of a c++ example using ZMQ_REP ?
[02:13] rmancy I was able to get the ZMQ_REP/ZMQ_REQ example using C/C++ fine, but looking for a C++ example using ZMQ_REP so I can find out what I'm doing wrong
[02:31] mike8901 What's the best way to have a vector of zmq::socket_t's? Just have pointers to the vectors?
[02:31] mike8901 **pointers to the sockets
[02:31] mike8901 (or smart pointers)
[03:47] CIA-15 pyzmq: 03Brian E. Granger 07master * r792a621 10/ (zmq/_zmq.c zmq/_zmq.pyx): Added support for getsockopt and multipart messages. -
[05:42] sustrik re
[05:43] sustrik mike8901: simultaneous read & write? each socket can be used only from a single thread
[05:43] mike8901 okay
[05:53] sustrik mike8901: as for storing zmq::socket_t's in vector
[05:53] sustrik they actually _are_ smart pointers
[05:53] sustrik they cannot be stored in a vector because they are not assignable
[05:54] sustrik i.e. they have no copy constructor and assignment operator
[05:54] mike8901 yes, I figured that out
[05:54] sustrik if you add those two, i'll accept the patch to c++ binding
[05:55] mike8901 from my understanding, smart pointers aren't _SUPPOSED_ to be stored in a vector
[05:55] sustrik what you can do is add reference count
[05:55] sustrik so when socket_t is copied reference count increases by 1
[05:55] mike8901 that's not my main concern
[05:56] mike8901 I ended up just storing pointers to zmq::socket_t's in the vector
[05:56] sustrik yes, can be done that way as well
[05:56] sustrik although you have to delete them manually afterwards
[05:57] mike8901 yes, of course
[05:57] mike8901 C++ would be so much nicer if it had -retain and -release like Obj-C
[05:57] sustrik that's what refcounting can do
[05:57] mike8901 yeah, it would be nice if refcounting was builtin
[05:58] mike8901 btw, are there any plans to make zeromq BSD/MIT licensed?
[05:58] sustrik well, it can at least be done for socket_t
[05:59] sustrik in theory, once it's established enough to make sure that contributions are flowing back to the project automatically
[05:59] sustrik at the present the fear is that people would patch it and don't contribute back
[06:00] sustrik why do you need that?
[06:00] mike8901 I'm currently using zeromq for a project with clang(basically implementing distcc at a layer tightly coupled with the compiler)
[06:00] mike8901 I'm using zeromq because it's easy to use and I'm running up against a tight deadline
[06:00] mike8901 (plus it's fast)
[06:00] mike8901 but clang won't accept my patch if it uses an LGPL library
[06:00] mike8901 everything has to be BSD licensed
[06:01] sustrik static linking?
[06:01] mike8901 I think they won't even allow dynamic linking
[06:01] mike8901 (I'm dynamically linking now)
[06:01] mike8901 basically, their policy is BSD only
[06:01] sustrik there's a clause explicitly allowing for static linking in 0mq license text
[06:02] sustrik anyway, if it's their policy, there's little to do at the moment
[06:02] sustrik you'll have to keep the patch out of the tree
[06:02] mike8901 ok, I guess I'll stick with zmq now, and hopefully by the time to patch back to mainline comes, zmq will be licensed for the process :)
[06:02] mike8901 *licensed using BSD
[06:03] sustrik hm, btw, what good is messaging layer in a compiler?
[06:05] rmancy I'm sure this is a silly Q. Can I make zmq libs from the git repo? I checked it out the other day, but didn't have much luck.
[06:06] sustrik rmancy_: what went wrong?
[06:06] rmancy sustrik, first off I'm not too ofey with C Makefiles. It's Been a while. Well there didn't seem to be a configure script, so I just did autoconfigure. Is this right ?
[06:07] sustrik yes
[06:07] sustrik the script is generated so it's not stored in git
[06:07] sustrik ./
[06:07] sustrik ./configure
[06:07] sustrik make
[06:08] rmancy sustrik, excellent. Thanks so much.
[06:08] sustrik np
[06:31] rmancy sustrik, I've been having a problem with a very simple ZMQ_REP/ZMQ_REQ app. Can you recommend a good ZMQ_REP example in c++ (I just converted the Server example from the 0mq site that was in C) ?
[06:32] sustrik what was the problem?
[06:33] rmancy Well, the ZMQ_REP (server) gets the recv() and then _appears_ to send(), but the ZMQ_REQ (client) does not get the response. the client code works fine with the example written in C.
[06:34] sustrik send the test program to the mailing list
[06:37] rmancy sustrik, ok cheers. I'm sure it's an obvious problem to most other people. I'm brand new to C++ and it's been over 4 years since I've even done C.
[07:02] mike8901 sustrik: I'm building a distributed compiler
[07:02] mike8901 similar to distcc, but much more efficient
[07:10] sustrik aha, interesting
[07:10] sustrik the point is make it compile faster?
[07:12] sustrik by running compilation of several machines at the same time?
[13:46] b0gg1e Hi
[13:47] b0gg1e I pushed some very early bindings for zeromq to go on github
[13:49] b0gg1e Currently barely works there seems to happen some weirdness between go's scheduler and zmq which I havent managed to understand yet, anyways feel free to peek or help
[13:52] sustrik b0ggle: great, thanks
[13:53] sustrik do you want to be referenced from website?
[13:59] b0gg1e its not actually usable right now, so that doesnt seem apropriate
[14:00] sustrik well, at least people would know about it
[14:00] b0gg1e more looking for help actually and am a bit lost whom to ask.. go people or zeromq people :-)
[14:00] sustrik you can place a big "EXPERIMENTAL" tag on the top of the page
[14:01] b0gg1e more like (BROKEN, NEEDS HELP)
[14:01] sustrik yes, sure
[14:01] b0gg1e ill do that on github
[14:01] sustrik as you wish
[14:01] b0gg1e maybe put a Go (EXPERIMENTAL) on the page if you like to
[14:02] sustrik sure
[14:02] sustrik do you want a wiki page on
[14:02] sustrik or should i link to your page somewhere?
[14:04] b0gg1e A wiki page on would be nice
[14:04] sustrik ok, do this: open and create an account (top right of the page)
[14:05] sustrik let me know your login then and i'll give you write access
[14:09] b0gg1e recovering wikidot account...
[14:29] b0gg1e account name is "splant"
[14:31] sustrik ok, you are invited
[14:31] sustrik i've created
[14:31] sustrik put the python binding wiki page content there to give you some template
[14:31] sustrik feel free to edit it as you see fit
[14:32] sustrik b0ggle: let me know when it's ready to be linked from the front page
[14:38] b0gg1e k thanks might take a while... the go folks are changing their gc and i might sit this out unless i have a good fast idea on how to fix things
[14:39] sustrik b0ggle: sure, it's up to you
[14:41] b0gg1e i cant edit the page / dont see an edit link
[15:22] sustrik b0ggle: have you accepted the invitation?
[15:25] sustrik it doesn't look so, i don't see you among members
[15:29] b0gg1e ah
[15:29] b0gg1e :-)
[15:29] b0gg1e can edit now, thanks
[15:32] sustrik :)
[15:59] b0gg1e contexts are bound to the thread in which they are created. is it possible to relax that to "contexts should only ever be used from a single thread at atime"?
[15:59] b0gg1e systems like erlang/go etc schedule lightweight processes on thread pools meaning the executing thread can change underneath you stackframe
[16:06] sustrik b0ggle: no, context are thread-safe and can be used from any thread
[16:06] sustrik the restricvtion you are speaking about applies to sockets
[16:07] sustrik and if you meant sockets, then no, it cannot be relaxed
[16:07] b0gg1e ah i ment sockets sure
[16:08] b0gg1e hm guess that has to do with the lock-free algos. too bad, now i have to find out why go's LockOSThread isnt working ;-(
[16:08] b0gg1e or what else is blocking.
[16:09] sustrik yes, it's about lock-free queues
[16:12] b0gg1e that also means one cannot move contexts between threads which might also be nice for loadbalancing purposes
[17:05] mike8901 sustrik: Yup, the idea is to preproc on the master, do everything through object code generation on slave, and link together on master.
[17:07] mike8901 (in a much more efficient manner than distcc)
[18:00] CIA-15 pyzmq: 03Brian E. Granger 07master * r1ffcf0e 10/ (zmq/_zmq.c zmq/_zmq.pyx): Removing SUBSCRIBE/UNSUBSCRIBE from the options in getsockopt. -
[19:28] sustrik mike8901: aha, looks like good match for 0mq; pitty there's the licensing problem
[20:26] CIA-15 pyzmq: 03Brian E. Granger 07master * r7a9d487 10/ (zmq/_zmq.c zmq/_zmq.pyx zmq/zmq_compat.h):
[20:26] CIA-15 pyzmq: Created zmq_compat.h for a compatibility header file.
[20:26] CIA-15 pyzmq: This will hopefully fix the int64_t issue. -
[20:43] mike8901 sustrik: yeah, zmq has worked out pretty nicely so far... you're welcome to follow along at
[20:43] mike8901 comments on (mis-)use of zeromq are greatly appriciated... there are probably some ways to use it better
[20:44] mike8901 right now, we're just doing basic point-to-point communication between nodes
[20:45] mike8901 it might be better to do a multicast setup so work-stealing can be done without rebroadcasting the data(i.e. slaves maintain a cache of files sent out to other slaves so they can steal work from another slave without asking for the data to be sent out again)
[20:46] mike8901 It's a shame we have a deadline in 2.5 weeks, so we're basocally racing to finish this, and can't do too many interesting optimizations
[20:46] mike8901 *basically
[20:51] CIA-15 pyzmq: 03Brian E. Granger 07master * r6ff6da2 10/ README.rst : Edits to the README.rst. -
[21:27] mike8901 sustrik: I ran into an issue where error_t was being defined by one(I don't know which) of the headers included... I had to #undef error_t to use zmq.hpp.
[21:27] mike8901 Is that a known issue?
[21:42] bgranger sustrik: I am trying out the XREP/XREQ support that you added yesterday