Friday April 8, 2011

[Time] NameMessage
[05:25] kevin can anyone help with installation of python bindings for zeromq on mac os x snowleopard?
[05:26] kevin "OSError: dlopen(zmq/, 10): image not found"
[05:31] kevin anyone? "-_-
[05:32] sustrik try later on when more people are online
[05:32] sustrik or, presumably, send an email to the mailing list
[05:33] kevin ah alrighty
[05:33] kevin thanks
[08:19] pieterh sustrik__: I think that was just "forgot to also build zeromq"
[08:19] pieterh a lot of people have trouble getting a language stack working
[08:22] guido_g hi all
[08:22] guido_g pieterh: any new ideas wrt the meeting in brussels?
[08:22] pieterh guido_g: meeting is on, for sure
[08:23] guido_g ok
[08:23] pieterh sustrik will be there
[08:23] guido_g i saw it (tracking the page :)
[08:24] pieterh I've also emailed the list about a larger conference in the US later this year
[08:25] guido_g hmmm... i need to catch up on the list mails...
[08:27] pieterh cyl, I'm off to do some shopping
[08:27] guido_g have fun!
[08:28] joelr good day!
[08:30] guido_g howdy!
[08:37] joelr guido_g: do you by chance know the project that holds zmsg_* functions
[08:37] guido_g not w/o searching, sorry
[08:38] guido_g looks like one of the projects of pieterh
[08:43] joelr i think it's the libzapi project
[08:43] joelr well, no, it is the libzapi project
[08:43] guido_g :)
[10:23] Intellect can anyone help me with zeromq on windows?
[10:24] Intellect I used the msi installer from pyzmq's git page
[10:26] Intellect importi zmq gives me "WindowsError: [Error 14001] The application has failed to start because its side-by-side configuration is incorrect"
[10:51] pieterh Intellect: have you installed ZeroMQ?
[10:52] Intellect pieterh: that msi installs libzmq along with it
[10:52] Intellect I am trying to install from git now
[10:53] pieterh please don't install 0MQ from git
[10:53] pieterh there is a stable release package, 2.1.4, which goes with pyzmq 2.1.4
[10:54] Intellect pieterh: I unzipped that package, and built it in visual studio
[10:55] Intellect now I am trying "python configure --zmq=../libzmq"
[10:55] Intellect but it is giving me an error
[10:57] pieterh Intellect: I'm not familiar with the pyzmq setup process, sorry... the folks who make that are mostly in California and probably waking up soon
[10:58] Intellect pieterh: how exactly do I install zeromq?
[10:58] Intellect I unzipped the package and built it in VS
[10:59] pieterh Intellect: well, you build the project and then put the library and header files somewhere
[10:59] Intellect that just gives a dll on the lib folder right
[10:59] pieterh put the DLL on the path
[10:59] pieterh I usually copy the DLL to a directory that is on my PATH
[11:00] Intellect what about the header files?
[11:00] pieterh any project that needs them has to point to them
[11:00] pieterh I don't think there's any standard for Windows
[11:01] pieterh in msvc you can define 'additional include directories' that refer to random places
[11:02] Intellect pieterh: the msi installer should work. It made a zmq directory inside python side packages which has the dll
[11:02] Intellect it says "On Windows, libzmq.dll will be copied into the zmq directory, and installed along with pyzmq, so you shouldn't need to edit your PATH"
[11:02] pieterh right
[11:02] Intellect but importing zmq gave me that error
[11:03] Intellect :(
[11:03] Intellect thanks anyway
[11:22] Intellect pieterh: I installed it from the msi again, but this time I replaced the dll with a one I built myself
[11:22] Intellect seems to work
[11:22] Intellect :)
[11:23] pieterh Intellect: great! you may want to tell the authors, and/or mention it on the pyzmq binding page
[11:24] Intellect pieterh: I think it is because I am on x64
[11:24] Intellect not sure though
[11:24] pieterh ah, probably
[11:24] Intellect they mention that the msi is dodgy on the git page
[11:24] Intellect so they probably know
[12:18] Intellect pieterh: still around?
[12:28] pieterh Intellect: sorry, went out for a while, back now
[12:28] Intellect pieterh: I was just trying out my first program. Have a PUB sending message and a SUB receiving it
[12:29] Intellect the send script seems to work, but nothing is received
[12:29] pieterh right
[12:29] pieterh that's normal
[12:29] Intellect do I need to run anything else?
[12:29] Intellect how do I receive what I sent?
[12:29] pieterh well, you need to read the Guide
[12:30] pieterh SUB sockets don't capture the beginning of a PUB stream accurately
[12:30] pieterh it's always fuzzy
[12:30] pieterh try sending 1M messages
[12:31] pieterh you may also enjoy this:
[12:31] Intellect pieterh: it should work proper in REQ REP ?
[12:31] pieterh yes, also push/pull
[12:31] Intellect ok, let me try that instead then
[12:31] pieterh please do read the guide, it is explained in detail
[12:32] Intellect yea I should do that. I was reading this -
[13:24] mikko pieterh: wikidot limits api usage to pro users
[13:24] mikko noticed last night
[13:25] pieterh mikko: I'll get your account upgraded
[13:25] mikko it used to be free-for-all ?
[13:25] mikko i think so
[13:26] pieterh the API used to be on request, in beta
[13:27] mikko i sent rpm spec files for openpgm to steve
[13:27] pieterh mikko: ok, upgrade requested, it'll take an hour or two
[13:27] mikko hopefully openpgm RPMs available soon for rpm based distros
[13:28] pieterh neat
[13:28] Intellect it works :D
[13:28] mikko one step closed to distro packaged zeromq/openpgm
[13:29] mikko pieterh: congrats on libzapi
[13:29] pieterh mikko: thanks! it's really useful for C work
[13:29] pieterh I've ported all the Guide examples, am using it in a couple of new ones
[13:29] pieterh mainly it lets me 'fix' all the flaws in the 0MQ api :)
[13:30] mikko what is the difference of libzapi and libzfl?
[13:30] mikko zapi is 0mq related and zfl is generic?
[13:30] pieterh scope, I guess... a lot of libzfl went into libzapi but libzfl will grow larger
[13:30] pieterh whereas libzapi should remain a pure binding
[13:31] pieterh e.g. I have a bunch of classes for resilience, they will go into libzfl
[13:31] pieterh libzfl depends on libzapi & libzmq
[13:34] mikko cool
[13:34] mikko hmm
[15:05] desjardins Having a little trouble building erlzmq2 on OSX...
[15:06] desjardins Any ideas? Not sure how that script could be missing, considering the build finds other files that live in that same dir.
[15:11] desjardins Ah, figured it out - glibtoolize depends on gsed but doesn't declare it as a dependency.
[15:26] evax hi, it's already the third time I have to deal with an issue related to the fact that zmq_recv can return before the corresponding zmq_send for inproc sockets. could this point be documented in the man page ?
[18:34] zedas sustrik: hey, so with the ZMQ_FD that's returned, is it triggered only on a read event or should it be polled on read/write like with a normal socket.
[18:38] MrGoodbyte hello. I just read that "To begin, instead of being stream (TCP), or datagram (UDP) oriented, ZeroMQ communication is message-oriented. This means that if a client socket sends a 150kb message, then the server socket will receive a complete, identical message on the other end without having to implement any explicit buffering or framing."
[18:39] MrGoodbyte what if my client sends a message larger than 150K?
[18:39] MrGoodbyte what that means?
[18:41] zedas MrGoodbyte: it'll be that size. what they mean is, unlike with a socket, when you request to read a message, you get the entire message, not whatever happens to be in the socket buffer.
[18:42] sustrik zedas: you should poll only on IN
[18:42] sustrik getting IN event means "something happened" to the socket
[18:42] zedas sustrik: rock thanks.
[18:42] sustrik then you should check EVENTS
[18:42] zedas right then events has the real POLLIN/POLLOUT
[18:42] sustrik yes
[18:43] sustrik it's pain in ass, but cannot be done simpler on top of POSIX OS
[18:43] sustrik there's no way to simulated real FDs in user space
[18:45] MrGoodbyte zedas: thanks for your response. I'm thinkin of that porting my task server application from gevent to 0mq. but some of my task clients works with 400K data avg. what happens if clients pass that data to server? server gets only the part sized 150k of whole bunch?
[18:52] zedas MrGoodbyte: first, gevent can do 0mq, so you might be able to port real easy. 2nd, that's too big, you should be storing that somewhere, then sending references to it.
[18:52] zedas MrGoodbyte: 3rd you are reading wayyyy to much into this. If you do this: x = "0" * (400 * 1024) ; sock.send(x); sock.recv(x); print len(x)
[18:53] zedas then it'll print 409600.
[18:53] zedas additionally, you could probably have gone and written a small sample server to test this limitation by now :-)
[19:03] MrGoodbyte zedas: it was just an example. in fact, I have kinda shared memory task in my task server application and I store data on them as memcached. that s why I told u about working with larger data. It s probably not 400k in avg but I m sure there are some objects that big (:
[19:04] achin zedas, can you expand on "that's too big"?
[19:04] achin if i want to send a 5Mb file to a worker, i should use zmq to send task info, but send the big data using another channel?
[19:29] eyeris The docs for pyzmq's zmq.core.poll.Poller.poll doesn't explain the return value. It's a list of 2-tuples which contain a socket and an integer
[19:29] Toba could be errnos
[19:29] eyeris I'm assuming that the integer is a code for read vs write vs exception, but I can't confirm that in the source code
[19:29] Toba or, not
[19:34] eyeris It seems to be 1=data to read, 2=ready for write.
[20:13] Toba what if it's... both?
[20:13] Toba 3?
[20:25] zedas achin: generally i don't try to send giant messages around a set of processes. i'd send it to one, store it, then have the others go get it
[20:31] eyeris Toba: REQ and REP sockets are stateful.
[20:31] eyeris You cannot call send or receive twice in a row
[20:32] eyeris send(); send() will raise an error, becuase it expects a recv() call for every send() call for a REQ socket and the reverse pattern for a REP socket
[20:34] eyeris thus there is no way a socket can be in both the readable and writable states
[20:42] achin zedas, what if each process needs a different large message? seems like pushing that large message down a ZMQ socket isn't too terrible
[20:44] Toba huh
[20:44] Toba so I must always do reply() even if the request related to send() failed utterly
[20:44] Toba noted
[21:19] zedas achin: nope you can shove tons of crap around, but if it's the same thing then sending it over and over is kind of dumb
[21:49] lt_schmidt_jr quick question about what is expected if a socket is registered with a poller after messages have queued up for it - will the poller report immediately that it has messages?
[22:20] achin zedas, ok. in the application in my mind, you'd be sending large and unique data to each worker
[22:29] kwag Hi. Anyone can point me in the right direction? Does zmq has something similar to Pyro, where objects are "registered" on a name server, so in this case, remote message queues can be found on the network by name? instead of forced IP adddresses? Thanks.
[23:24] lt_schmidt_jr kwag: no that is something you would have to build yourself on top of 0mq
[23:26] kwag tnx, lt_schmidt_jr