ZeroMq IRC Log

Saturday January 1, 2011

[Time] Name	Message
[15:10] dos000	howdy
[15:10] dos000	and happy new year
[15:10] dos000	is sustrik here ?
[15:12] mikko	he was in yesterday
[15:12] mikko	havent seen him talking today
[15:15] dos000	ah!
[17:34] sustrik	dos000: hi
[17:36] sustrik	happy new year everyone!
[18:16] pythonirc1011	how much work is it to get a installer ready for windows7 for zeromq+python bindings. Where can one get started on that?
[18:33] mikko	pythonirc1011: probably it's just building the installer
[18:33] mikko	pythonirc1011: not sure how python extensions are installed on windows or any platform really
[18:41] dos000	sustrik, howdy !
[18:42] dos000	sustrik, happy new year to you too
[18:47] sustrik	pythonirc1011: building a win installer would be very helpful
[18:47] sustrik	i would start with the core part
[18:47] pythonirc1011	mikko: so i need a installation script language + compilation of a zeromq on my win box i guess...
[18:47] sustrik	it's just a lib and a header file so it should be easy
[18:48] pythonirc1011	i'm more after building the python bindings. I need to use it from python
[18:48] sustrik	sure, but those are dependent on the lib
[18:48] pythonirc1011	true
[18:48] pythonirc1011	perhaps ship the library pre-compiler on windows
[18:48] pythonirc1011	compiler
[18:49] pythonirc1011	compiled i meant
[18:49] sustrik	exactly
[18:49] sustrik	what you need is 2 libraties
[18:49] sustrik	libzmq
[18:49] sustrik	and libpyzmq
[18:49] sustrik	you need to compile them
[18:49] sustrik	and build an installer
[18:49] pythonirc1011	k, that should be doable...are they dlls?
[18:49] sustrik	i am not sure whether python needs anythig else aside of those libs
[18:50] sustrik	yes, i think so
[18:50] pythonirc1011	I'll look into it soon. Thanks.
[18:55] dos000	sustrik, does zma uses lock free ring buffers ? sometalk i listened mentioned it being musch faster than using queues becose of contention
[18:55] dos000	s/zma/0mq
[18:56] sustrik	yes, the queues are lock-free
[18:56] dos000	sustrik, is it ringbuffer based ?
[18:57] sustrik	messages are allocated in blocks
[18:57] sustrik	(256 per block)
[18:57] dos000	sustrik, here is the talk http://www.infoq.com/presentations/LMAX jump to 36:00 for the real meat.
[18:57] sustrik	one block is recycled to eliminate excessive allocation/deallocation
[18:58] dos000	its kinda java oriented but i think the same issues apply in c++
[18:58] sustrik	however, if the number of messages in the queue drops, only 1 block is recycled, any subsequent blocvks are dealloc'd
[18:58] dos000	ah!
[18:59] dos000	thats seems smarter .. they allocate 2min off hand to counter java gc
[18:59] dos000	2 millions
[18:59] sustrik	well, depends on the use case
[18:59] sustrik	it's something that's not good to do on a sensor or an embedded system
[19:00] sustrik	0mq tries to be generic
[19:00] dos000	indeed .. for a server its much different
[19:00] sustrik	you can of course tune it by hand if you want to
[19:00] sustrik	see src/config.hpp
[19:01] dos000	sustrik, so its configurable by source changes only then
[19:02] sustrik	there are 2 mechanisms there
[19:02] sustrik	config.hpp are compile-time options
[19:02] dos000	sustrik, it will apply to all queues then
[19:02] sustrik	socket options can be used in run-time
[19:02] dos000	ah!
[19:03] sustrik	it's a matter to consider whether specific tuning parameter
[19:03] dos000	socket options allow to you bigger/smaller buffers ?
[19:03] sustrik	should be a socket option or a compile-time param
[19:03] sustrik	socket option allow to set the maximal buffer size
[19:03] dos000	i cant use compile time options
[19:03] sustrik	what are you trying to do?
[19:05] dos000	i am writing queues with different bechmarking options
[19:05] dos000	so i want to see effect of the changes
[19:06] sustrik	writing queues?
[19:06] sustrik	you mean you are implementing the queues yourself?
[19:06] dos000	so far i have 0mq and fastflow ( http://calvados.di.unipi.it/dokuwiki/doku.php?id=ffnamespace:about ) on my list
[19:06] sustrik	ok, i see
[19:07] sustrik	and what exactly are you missing?
[19:08] dos000	sustrik, i only have one issue so far ...
[19:09] dos000	i have to receive the events and process them in java ...
[19:10] dos000	i either reimplement the lockfree ring buffers in java .. or use jzmq
[19:10] dos000	it looks deceptively easy to write it in java !
[19:11] sustrik	i still don't see what you are missing :)
[19:11] sustrik	0mq is lock-free
[19:11] dos000	yes ... i am not missing anything ...
[19:11] dos000	i just have options !
[19:12] sustrik	implementing the lock-free queeu is not hard
[19:12] sustrik	it may become a problem as you want to add more features
[19:12] dos000	sustrik, there is locking when you call c++ functions from java
[19:12] sustrik	like limiting the number of messages in queue
[19:12] sustrik	or somesuch
[19:13] sustrik	is there, i am not familiar with how jzmq works atm
[19:13] dos000	its inherent in jni
[19:13] sustrik	i see
[19:13] sustrik	then you have to reimplement it yourself
[19:13] dos000	another option would be to receive the events via sockets in java
[19:13] sustrik	what sockets?
[19:14] dos000	instead of passing the messages via jni pass it via a the (local) netwrk interface
[19:15] dos000	of course that would be slower i imagine
[19:15] dos000	1) evtsource -> zmq --> jzmq --> java app
[19:16] dos000	eventsource -> zmq --> javanio --> javaapp
[19:17] dos000	anyway ...
[19:17] dos000	i will keep you updated !
[19:17] dos000	thanks a lot
[19:20] sustrik	you are welcome
[19:36] mikko	happy new year all
[20:12] andrewvc	cremes: If you're around I have a question about error_check_nonblock if you have a chance
[20:33] andrewvc	So, here's a nonblocking question
[20:36] andrewvc	so, if I get an errno 95 doing a recv NOBLOCK
[20:37] andrewvc	what is the usual cause for that
[20:37] andrewvc	I was expecting an EAGAIN
[20:37] andrewvc	and retrying later my recv did work
[21:27] sustrik	what's errno 95?
[21:59] andrewvc	sustrik, #define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[21:59] andrewvc	basically, I get a notification of activity via select for a FD
[22:00] andrewvc	so I'm calling a recv ZMQ::NOBLOCK on it to see if I can read off a message
[22:00] andrewvc	and I get that
[22:00] andrewvc	i mean if I just pretend that's an EAGAIN it works alright
[22:00] andrewvc	but just not what I expected
[23:23] neopallium	andrewvc: if you are calling recv() on a REQ/REP socket during the wrong state, you will get that error.
[23:24] neopallium	REQ/REP sockets switch back and forth from send-only to recv-only states.