ZeroMq IRC Log

Saturday February 26, 2011

[Time] Name	Message
[00:02] apexi200sx	kool
[01:46] cordor	what is multipart message?
[02:31] tlockley	Are there any install guides for ZeroMQ on OS X?
[07:44] CIA-21	zeromq2: 03Martin Sustrik 07master * r67b1f14 10/ (src/xpub.cpp src/xpub.hpp src/xsub.cpp src/xsub.hpp):
[07:44] CIA-21	zeromq2: Memory leak in PUB/XPUB sockets fixed.
[07:44] CIA-21	zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/ibEgDD
[09:41] cordor	zmq_init (int app_threads, int io_threads, int flags);
[09:42] cordor	am i looking at really old header file?
[09:50] sustrik	2.0.x
[09:50] sustrik	actually, 2.0.6 or older
[10:06] pieterh	sustrik: next week I'll fork off 2.0 into its own repository
[10:07] sustrik	great
[10:07] pieterh	we need some way to direct people from the source repo to these release repos
[10:07] pieterh	for random historical interest it might be fun to put up a 1.x repo as well
[10:08] pieterh	there are still people using 1.x in production afaik
[10:10] cordor	launchpad
[10:13] pieterh	cordor: you mean https://launchpad.net/? but we're already using github...
[10:18] cordor	hummmm, i thought github is for ppl who develop zmq, launchpad is for ppl who use libzmq for development
[10:18] pieterh	we don't use launchpad anywhere afaik, except maybe to distribute Ubuntu packages
[10:19] pieterh	but yes, github is mainly for those developing 0MQ
[10:20] pieterh	most users (should) use the released packages
[11:07] pieterh	sustrik: ping
[11:12] pieterh	it's normal that zmq_term() hangs forever if there are outgoing messages waiting to be sent (on req socket)?
[11:17] mikko	pieterh: yes
[11:17] mikko	pieterh: you need to set ZMQ_LINGER
[11:17] pieterh	does default of "wait forever" make sense?
[11:18] pieterh	it does seem to be a major change in semantics
[12:29] Evet	is it reasonable to use zmq instead of fastcgi?
[12:34] PiotrSikora	Evet: zmq isn't replacement of fastcgi
[12:34] PiotrSikora	those are two different layers (transport vs application)
[12:36] Evet	PiotrSikora: indeed. but, what if you use zmq on both front-end http handler and back-end application?
[12:50] mikko	Evet: that is about what mongrel2 does
[12:53] Evet	hmm
[13:07] PiotrSikora	Evet: then you're replacing TCP with ZMQ and FastCGI with HTTP
[13:07] PiotrSikora	it's apples to oranges
[13:08] Evet	nope
[13:08] PiotrSikora	yes
[13:08] Evet	im replacing fastcgi protocol between front-end http handler and back-end application
[13:08] PiotrSikora	with what?
[13:09] Evet	with zeromq's interprocess protocol
[13:09] PiotrSikora	you don't read what I write, do you?
[13:10] Evet	im not touching http
[13:13] PiotrSikora	Evet: but you're not replacing FastCGI with ZMQ, you're replacing FastCGI with custom logic nad TCP with ZMQ
[13:14] Evet	yes
[18:36] apexi200sx	ok so there is the zeromq os specific client libraries
[18:37] apexi200sx	and then there are language specific bindings to the zeromq client library
[18:37] apexi200sx	when I say use python to send messages using zeromq, what mechanism is used by the zeromq client libraries
[18:37] apexi200sx	to send the messages
[18:38] apexi200sx	what is happening under the hood when I do a socket.send(msg) from python
[18:39] apexi200sx	does the zeromq libraries perform the actual network IO in a separate thread transparent to python
[18:47] pieterh	apexi200sx: yes
[18:47] pieterh	that's to your last question
[18:47] pieterh	no, that's to your first assertion
[18:48] pieterh	there is the core, the C++ library, and then there are language bindings that wrap it
[18:48] apexi200sx	ok
[18:49] apexi200sx	ok, so a separate thread is used, and to make it possible to make connections to lots of other zeromq nodes, this thread uses non-blobking IO
[18:50] apexi200sx	so, for example would use epoll on Linux
[18:51] apexi200sx	What IO mechanism is used on Windows ?
[18:52] apexi200sx	From what I can gatther the epoll performance equivalent on Windows is IOCP
[18:55] apexi200sx	is this used
[20:30] Guthur	apexi200sx, IOCP is not used currently on windows
[20:30] Guthur	which is part of the reason there is no IPC on the win platform
[20:32] Guthur	there has been some preliminary research into bring IOCP to zmq which would allow the use of named pipes and so bring IPC
[20:32] Guthur	part of the problem is that IOCP does not directly provide all the functionality of epoll like mechanisms
[20:33] apexi200sx	Guthur: I see, so, Windows is probably not the best choice of platform for zeromq at this time
[20:33] Guthur	I hope to look into this problem more in the future, my workload does not allow for it at the moment though
[20:34] Guthur	apexi200sx, IPC is a pretty large whole, that is true
[20:34] Guthur	but it's still reasonable in other regards
[20:38] apexi200sx	I could still have windows subscribers, but for large volume forwarding "devices" with many connected devices its probably best to use a *nix type os
[20:38] apexi200sx	?
[20:44] Guthur	if you want to have them communicating between processes (IPC) then yeah, inproc is still ok though
[20:45] Guthur	so if those forwarding devices are all connecting to workers in the same process it is not a problem
[20:46] apexi200sx	Guthur: do you have much experience with Python
[20:46] Guthur	limited experience unfortunately
[20:46] mikko	good evening
[20:47] apexi200sx	mikko: good evening :)
[20:49] Guthur	apexi200sx, I suppose the GIL would mean you want separate processes with python, correct?
[20:49] Guthur	or is it not really a factor
[20:53] mikko	pieterh: there?
[21:05] apexi200sx	Guthur: Yes, the GIL is an issue for CPU bound Python applications. Only 1 thread at a time can be executing python code at any time. A CPU bound python application actually runs slower if you try to split the work over 2 or more threads than a single thread
[21:06] apexi200sx	The GIL i released though when a thread akes an IO request, so having an IO thread and a processing thread is still useful
[21:07] apexi200sx	so yeah processes are the way to go in Python
[22:00] mikko	sustrik: got a compile warning on centos 5