Saturday January 1, 2011

[Time] NameMessage
[15:10] dos000 howdy
[15:10] dos000 and happy new year
[15:10] dos000 is sustrik here ?
[15:12] mikko he was in yesterday
[15:12] mikko havent seen him talking today
[15:15] dos000 ah!
[17:34] sustrik dos000: hi
[17:36] sustrik happy new year everyone!
[18:16] pythonirc1011 how much work is it to get a installer ready for windows7 for zeromq+python bindings. Where can one get started on that?
[18:33] mikko pythonirc1011: probably it's just building the installer
[18:33] mikko pythonirc1011: not sure how python extensions are installed on windows or any platform really
[18:41] dos000 sustrik, howdy !
[18:42] dos000 sustrik, happy new year to you too
[18:47] sustrik pythonirc1011: building a win installer would be very helpful
[18:47] sustrik i would start with the core part
[18:47] pythonirc1011 mikko: so i need a installation script language + compilation of a zeromq on my win box i guess...
[18:47] sustrik it's just a lib and a header file so it should be easy
[18:48] pythonirc1011 i'm more after building the python bindings. I need to use it from python
[18:48] sustrik sure, but those are dependent on the lib
[18:48] pythonirc1011 true
[18:48] pythonirc1011 perhaps ship the library pre-compiler on windows
[18:48] pythonirc1011 compiler
[18:49] pythonirc1011 compiled i meant
[18:49] sustrik exactly
[18:49] sustrik what you need is 2 libraties
[18:49] sustrik libzmq
[18:49] sustrik and libpyzmq
[18:49] sustrik you need to compile them
[18:49] sustrik and build an installer
[18:49] pythonirc1011 k, that should be doable...are they dlls?
[18:49] sustrik i am not sure whether python needs anythig else aside of those libs
[18:50] sustrik yes, i think so
[18:50] pythonirc1011 I'll look into it soon. Thanks.
[18:55] dos000 sustrik, does zma uses lock free ring buffers ? sometalk i listened mentioned it being musch faster than using queues becose of contention
[18:55] dos000 s/zma/0mq
[18:56] sustrik yes, the queues are lock-free
[18:56] dos000 sustrik, is it ringbuffer based ?
[18:57] sustrik messages are allocated in blocks
[18:57] sustrik (256 per block)
[18:57] dos000 sustrik, here is the talk jump to 36:00 for the real meat.
[18:57] sustrik one block is recycled to eliminate excessive allocation/deallocation
[18:58] dos000 its kinda java oriented but i think the same issues apply in c++
[18:58] sustrik however, if the number of messages in the queue drops, only 1 block is recycled, any subsequent blocvks are dealloc'd
[18:58] dos000 ah!
[18:59] dos000 thats seems smarter .. they allocate 2min off hand to counter java gc
[18:59] dos000 2 millions
[18:59] sustrik well, depends on the use case
[18:59] sustrik it's something that's not good to do on a sensor or an embedded system
[19:00] sustrik 0mq tries to be generic
[19:00] dos000 indeed .. for a server its much different
[19:00] sustrik you can of course tune it by hand if you want to
[19:00] sustrik see src/config.hpp
[19:01] dos000 sustrik, so its configurable by source changes only then
[19:02] sustrik there are 2 mechanisms there
[19:02] sustrik config.hpp are compile-time options
[19:02] dos000 sustrik, it will apply to all queues then
[19:02] sustrik socket options can be used in run-time
[19:02] dos000 ah!
[19:03] sustrik it's a matter to consider whether specific tuning parameter
[19:03] dos000 socket options allow to you bigger/smaller buffers ?
[19:03] sustrik should be a socket option or a compile-time param
[19:03] sustrik socket option allow to set the maximal buffer size
[19:03] dos000 i cant use compile time options
[19:03] sustrik what are you trying to do?
[19:05] dos000 i am writing queues with different bechmarking options
[19:05] dos000 so i want to see effect of the changes
[19:06] sustrik writing queues?
[19:06] sustrik you mean you are implementing the queues yourself?
[19:06] dos000 so far i have 0mq and fastflow ( ) on my list
[19:06] sustrik ok, i see
[19:07] sustrik and what exactly are you missing?
[19:08] dos000 sustrik, i only have one issue so far ...
[19:09] dos000 i have to receive the events and process them in java ...
[19:10] dos000 i either reimplement the lockfree ring buffers in java .. or use jzmq
[19:10] dos000 it looks deceptively easy to write it in java !
[19:11] sustrik i still don't see what you are missing :)
[19:11] sustrik 0mq is lock-free
[19:11] dos000 yes ... i am not missing anything ...
[19:11] dos000 i just have options !
[19:12] sustrik implementing the lock-free queeu is not hard
[19:12] sustrik it may become a problem as you want to add more features
[19:12] dos000 sustrik, there is locking when you call c++ functions from java
[19:12] sustrik like limiting the number of messages in queue
[19:12] sustrik or somesuch
[19:13] sustrik is there, i am not familiar with how jzmq works atm
[19:13] dos000 its inherent in jni
[19:13] sustrik i see
[19:13] sustrik then you have to reimplement it yourself
[19:13] dos000 another option would be to receive the events via sockets in java
[19:13] sustrik what sockets?
[19:14] dos000 instead of passing the messages via jni pass it via a the (local) netwrk interface
[19:15] dos000 of course that would be slower i imagine
[19:15] dos000 1) evtsource -> zmq --> jzmq --> java app
[19:16] dos000 eventsource -> zmq --> javanio --> javaapp
[19:17] dos000 anyway ...
[19:17] dos000 i will keep you updated !
[19:17] dos000 thanks a lot
[19:20] sustrik you are welcome
[19:36] mikko happy new year all
[20:12] andrewvc cremes: If you're around I have a question about error_check_nonblock if you have a chance
[20:33] andrewvc So, here's a nonblocking question
[20:36] andrewvc so, if I get an errno 95 doing a recv NOBLOCK
[20:37] andrewvc what is the usual cause for that
[20:37] andrewvc I was expecting an EAGAIN
[20:37] andrewvc and retrying later my recv did work
[21:27] sustrik what's errno 95?
[21:59] andrewvc sustrik, #define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */
[21:59] andrewvc basically, I get a notification of activity via select for a FD
[22:00] andrewvc so I'm calling a recv ZMQ::NOBLOCK on it to see if I can read off a message
[22:00] andrewvc and I get that
[22:00] andrewvc i mean if I just pretend that's an EAGAIN it works alright
[22:00] andrewvc but just not what I expected
[23:23] neopallium andrewvc: if you are calling recv() on a REQ/REP socket during the wrong state, you will get that error.
[23:24] neopallium REQ/REP sockets switch back and forth from send-only to recv-only states.