ZeroMq IRC Log

Friday April 8, 2011

[Time] Name	Message
[05:25] kevin	can anyone help with installation of python bindings for zeromq on mac os x snowleopard?
[05:26] kevin	"OSError: dlopen(zmq/_zeromq.so, 10): image not found"
[05:31] kevin	anyone? "-_-
[05:32] sustrik	try later on when more people are online
[05:32] sustrik	or, presumably, send an email to the mailing list
[05:33] kevin	ah alrighty
[05:33] kevin	thanks
[08:19] pieterh	sustrik__: I think that was just "forgot to also build zeromq"
[08:19] pieterh	a lot of people have trouble getting a language stack working
[08:22] guido_g	hi all
[08:22] guido_g	pieterh: any new ideas wrt the meeting in brussels?
[08:22] pieterh	guido_g: meeting is on, for sure
[08:23] guido_g	ok
[08:23] pieterh	sustrik will be there
[08:23] guido_g	i saw it (tracking the page :)
[08:24] pieterh	I've also emailed the list about a larger conference in the US later this year
[08:25] guido_g	hmmm... i need to catch up on the list mails...
[08:27] pieterh	cyl, I'm off to do some shopping
[08:27] guido_g	have fun!
[08:28] joelr	good day!
[08:30] guido_g	howdy!
[08:37] joelr	guido_g: do you by chance know the project that holds zmsg_* functions
[08:37] guido_g	not w/o searching, sorry
[08:38] guido_g	looks like one of the projects of pieterh
[08:43] joelr	i think it's the libzapi project
[08:43] joelr	well, no, it is the libzapi project
[08:43] guido_g	:)
[10:23] Intellect	can anyone help me with zeromq on windows?
[10:24] Intellect	I used the msi installer from pyzmq's git page
[10:26] Intellect	importi zmq gives me "WindowsError: [Error 14001] The application has failed to start because its side-by-side configuration is incorrect"
[10:51] pieterh	Intellect: have you installed ZeroMQ?
[10:52] Intellect	pieterh: that msi installs libzmq along with it
[10:52] Intellect	I am trying to install from git now
[10:53] pieterh	please don't install 0MQ from git
[10:53] pieterh	there is a stable release package, 2.1.4, which goes with pyzmq 2.1.4
[10:54] Intellect	pieterh: I unzipped that package, and built it in visual studio
[10:55] Intellect	now I am trying "python setup.py configure --zmq=../libzmq"
[10:55] Intellect	but it is giving me an error
[10:57] pieterh	Intellect: I'm not familiar with the pyzmq setup process, sorry... the folks who make that are mostly in California and probably waking up soon
[10:58] Intellect	pieterh: how exactly do I install zeromq?
[10:58] Intellect	I unzipped the package and built it in VS
[10:59] pieterh	Intellect: well, you build the project and then put the library and header files somewhere
[10:59] Intellect	that just gives a dll on the lib folder right
[10:59] pieterh	put the DLL on the path
[10:59] pieterh	I usually copy the DLL to a directory that is on my PATH
[11:00] Intellect	what about the header files?
[11:00] pieterh	any project that needs them has to point to them
[11:00] pieterh	I don't think there's any standard for Windows
[11:01] pieterh	in msvc you can define 'additional include directories' that refer to random places
[11:02] Intellect	pieterh: the msi installer should work. It made a zmq directory inside python side packages which has the dll
[11:02] Intellect	it says "On Windows, libzmq.dll will be copied into the zmq directory, and installed along with pyzmq, so you shouldn't need to edit your PATH"
[11:02] pieterh	right
[11:02] Intellect	but importing zmq gave me that error
[11:03] Intellect	:(
[11:03] Intellect	thanks anyway
[11:22] Intellect	pieterh: I installed it from the msi again, but this time I replaced the dll with a one I built myself
[11:22] Intellect	seems to work
[11:22] Intellect	:)
[11:23] pieterh	Intellect: great! you may want to tell the authors, and/or mention it on the pyzmq binding page
[11:24] Intellect	pieterh: I think it is because I am on x64
[11:24] Intellect	not sure though
[11:24] pieterh	ah, probably
[11:24] Intellect	they mention that the msi is dodgy on the git page
[11:24] Intellect	so they probably know
[12:18] Intellect	pieterh: still around?
[12:28] pieterh	Intellect: sorry, went out for a while, back now
[12:28] Intellect	pieterh: I was just trying out my first program. Have a PUB sending message and a SUB receiving it
[12:29] Intellect	the send script seems to work, but nothing is received
[12:29] pieterh	right
[12:29] pieterh	that's normal
[12:29] Intellect	do I need to run anything else?
[12:29] Intellect	how do I receive what I sent?
[12:29] pieterh	well, you need to read the Guide
[12:30] pieterh	SUB sockets don't capture the beginning of a PUB stream accurately
[12:30] pieterh	it's always fuzzy
[12:30] pieterh	try sending 1M messages
[12:31] pieterh	you may also enjoy this: http://zguide.zeromq.org/page:all#Missing-Message-Problem-Solver
[12:31] Intellect	pieterh: it should work proper in REQ REP ?
[12:31] pieterh	yes, also push/pull
[12:31] Intellect	ok, let me try that instead then
[12:31] pieterh	please do read the guide, it is explained in detail
[12:32] Intellect	yea I should do that. I was reading this - http://nichol.as/zeromq-an-introduction
[13:24] mikko	pieterh: wikidot limits api usage to pro users
[13:24] mikko	noticed last night
[13:25] pieterh	mikko: I'll get your account upgraded
[13:25] mikko	it used to be free-for-all ?
[13:25] mikko	i think so
[13:26] pieterh	the API used to be on request, in beta
[13:27] mikko	i sent rpm spec files for openpgm to steve
[13:27] pieterh	mikko: ok, upgrade requested, it'll take an hour or two
[13:27] mikko	hopefully openpgm RPMs available soon for rpm based distros
[13:28] pieterh	neat
[13:28] Intellect	it works :D
[13:28] mikko	one step closed to distro packaged zeromq/openpgm
[13:29] mikko	pieterh: congrats on libzapi
[13:29] pieterh	mikko: thanks! it's really useful for C work
[13:29] pieterh	I've ported all the Guide examples, am using it in a couple of new ones
[13:29] pieterh	mainly it lets me 'fix' all the flaws in the 0MQ api :)
[13:30] mikko	what is the difference of libzapi and libzfl?
[13:30] mikko	zapi is 0mq related and zfl is generic?
[13:30] pieterh	scope, I guess... a lot of libzfl went into libzapi but libzfl will grow larger
[13:30] pieterh	whereas libzapi should remain a pure binding
[13:31] pieterh	e.g. I have a bunch of classes for resilience, they will go into libzfl
[13:31] pieterh	libzfl depends on libzapi & libzmq
[13:34] mikko	cool
[13:34] mikko	hmm
[15:05] desjardins	Having a little trouble building erlzmq2 on OSX... https://gist.github.com/e244de338b24ada74b0e
[15:06] desjardins	Any ideas? Not sure how that script could be missing, considering the build finds other files that live in that same dir.
[15:11] desjardins	Ah, figured it out - glibtoolize depends on gsed but doesn't declare it as a dependency.
[15:26] evax	hi, it's already the third time I have to deal with an issue related to the fact that zmq_recv can return before the corresponding zmq_send for inproc sockets. could this point be documented in the man page ?
[18:34] zedas	sustrik: hey, so with the ZMQ_FD that's returned, is it triggered only on a read event or should it be polled on read/write like with a normal socket.
[18:38] MrGoodbyte	hello. I just read that "To begin, instead of being stream (TCP), or datagram (UDP) oriented, ZeroMQ communication is message-oriented. This means that if a client socket sends a 150kb message, then the server socket will receive a complete, identical message on the other end without having to implement any explicit buffering or framing."
[18:39] MrGoodbyte	what if my client sends a message larger than 150K?
[18:39] MrGoodbyte	what that means?
[18:41] zedas	MrGoodbyte: it'll be that size. what they mean is, unlike with a socket, when you request to read a message, you get the entire message, not whatever happens to be in the socket buffer.
[18:42] sustrik	zedas: you should poll only on IN
[18:42] sustrik	getting IN event means "something happened" to the socket
[18:42] zedas	sustrik: rock thanks.
[18:42] sustrik	then you should check EVENTS
[18:42] zedas	right then events has the real POLLIN/POLLOUT
[18:42] sustrik	yes
[18:43] sustrik	it's pain in ass, but cannot be done simpler on top of POSIX OS
[18:43] sustrik	there's no way to simulated real FDs in user space
[18:45] MrGoodbyte	zedas: thanks for your response. I'm thinkin of that porting my task server application from gevent to 0mq. but some of my task clients works with 400K data avg. what happens if clients pass that data to server? server gets only the part sized 150k of whole bunch?
[18:52] zedas	MrGoodbyte: first, gevent can do 0mq, so you might be able to port real easy. 2nd, that's too big, you should be storing that somewhere, then sending references to it.
[18:52] zedas	MrGoodbyte: 3rd you are reading wayyyy to much into this. If you do this: x = "0" * (400 * 1024) ; sock.send(x); sock.recv(x); print len(x)
[18:53] zedas	then it'll print 409600.
[18:53] zedas	additionally, you could probably have gone and written a small sample server to test this limitation by now :-)
[19:03] MrGoodbyte	zedas: it was just an example. in fact, I have kinda shared memory task in my task server application and I store data on them as memcached. that s why I told u about working with larger data. It s probably not 400k in avg but I m sure there are some objects that big (:
[19:04] achin	zedas, can you expand on "that's too big"?
[19:04] achin	if i want to send a 5Mb file to a worker, i should use zmq to send task info, but send the big data using another channel?
[19:29] eyeris	The docs for pyzmq's zmq.core.poll.Poller.poll doesn't explain the return value. It's a list of 2-tuples which contain a socket and an integer
[19:29] Toba	could be errnos
[19:29] eyeris	I'm assuming that the integer is a code for read vs write vs exception, but I can't confirm that in the source code
[19:29] Toba	or, not
[19:34] eyeris	It seems to be 1=data to read, 2=ready for write.
[20:13] Toba	what if it's... both?
[20:13] Toba	3?
[20:25] zedas	achin: generally i don't try to send giant messages around a set of processes. i'd send it to one, store it, then have the others go get it
[20:31] eyeris	Toba: REQ and REP sockets are stateful.
[20:31] eyeris	You cannot call send or receive twice in a row
[20:32] eyeris	send(); send() will raise an error, becuase it expects a recv() call for every send() call for a REQ socket and the reverse pattern for a REP socket
[20:34] eyeris	thus there is no way a socket can be in both the readable and writable states
[20:42] achin	zedas, what if each process needs a different large message? seems like pushing that large message down a ZMQ socket isn't too terrible
[20:44] Toba	huh
[20:44] Toba	so I must always do reply() even if the request related to send() failed utterly
[20:44] Toba	noted
[21:19] zedas	achin: nope you can shove tons of crap around, but if it's the same thing then sending it over and over is kind of dumb
[21:49] lt_schmidt_jr	quick question about what is expected if a socket is registered with a poller after messages have queued up for it - will the poller report immediately that it has messages?
[22:20] achin	zedas, ok. in the application in my mind, you'd be sending large and unique data to each worker
[22:29] kwag	Hi. Anyone can point me in the right direction? Does zmq has something similar to Pyro, where objects are "registered" on a name server, so in this case, remote message queues can be found on the network by name? instead of forced IP adddresses? Thanks.
[23:24] lt_schmidt_jr	kwag: no that is something you would have to build yourself on top of 0mq
[23:26] kwag	tnx, lt_schmidt_jr