IRC Log

Friday October 21, 2011

[Time] NameMessage
[04:21] cremes espeed: that is the purpose of the ZMQ_LINGER flag; set it higher than 0 (which means "close immediately and drop pending messages")
[12:54] mikko _o/
[13:38] Kobolog hi, anyone able to answer some questions here?
[13:39] mikko Kobolog: yes
[13:39] Kobolog the question is, since zmq 2.1.10 i'm unable to use tcp transport at all, since all the bind/connect functions return EINVAL on simple endpoints like tcp://*:5000
[13:40] Kobolog it works fine with zmq =< 2.1.9
[13:40] Kobolog and broken on every new version for me
[13:40] mikko tcp://*:5000 for bind?
[13:40] Kobolog yeah
[13:41] Kobolog socket.bind("tcp://*:5000")
[13:41] Kobolog with works just fine in 2.1.9
[13:41] Kobolog with=which
[13:42] mikko hmm, i haven't noticed the bind to fail
[13:42] mikko sec
[13:45] Kobolog mikko: i use the official 2.1.10 tarball, and the sequence of actions is pretty simple: create a context, create a ROUTER socket, set an identity, then bind on 'tcp://*:5000'
[13:46] Kobolog results in EINVAL :/
[13:47] Kobolog well, any other port fails as well
[13:47] Kobolog ipc transport still works, for example
[13:48] mikko Kobolog: seems to be fine here
[13:48] mikko Kobolog: does bind fail if you use ip address explicitly?
[13:49] Kobolog i'll try, sec
[13:50] mikko and does tcp://0.0.0.0:5555 bind work if you test instead of *
[13:55] Kobolog okay, i took the 2.1.10 tarball right from the site, ./configure --with-pgm --prefix=/usr && make && tests/test_pair_tcp
[13:55] Kobolog EINVAL again
[13:56] Kobolog and in test, it binds on tcp://127.0.0.1:5560
[13:57] mikko Kobolog: have you got any sort of security mode going on ?
[13:57] mikko that prevents bind
[13:58] mikko Kobolog: since they seems to succeed here on linux, solaris and freebsd
[13:58] Kobolog but it works just fine if i install 2.1.9 on the same machine
[13:59] Kobolog its lucid 64bit, 2.6.24-23-openvz
[13:59] Kobolog might it be openvz problems?
[13:59] mikko no, unlikely
[13:59] mikko https://zeromq.jira.com/browse/LIBZMQ-207
[13:59] mikko this could be related
[14:00] mikko this was changed in 2.1.10
[14:00] mikko changed=integrated
[14:02] Kobolog is there a link somewhere to the actual commit with introduced these changes?
[14:03] Kobolog so, basically, what fails is hostname resolution?
[14:04] Kobolog with=which, meh
[14:04] mikko https://github.com/zeromq/zeromq2-1/commit/a41271a0c1840729f4a439b0bd240ca78af04f02
[14:04] mikko Kobolog: looks like so
[14:04] mikko although with tcp://127.0.0.1:5555 it should nt fail
[14:05] Kobolog yet it still fails
[14:07] mikko Kobolog: can you check.. hmm
[14:07] mikko sec
[14:07] Kobolog btw, connect() works just fine
[14:08] mikko can you add printf ("getaddrinfo: %s\n", gai_strerror (rc));
[14:08] mikko ip.cpp:336
[14:08] mikko inside the if (rc) condition
[14:08] mikko or break gdb there and print rc value
[14:10] Kobolog yeah sec
[14:13] mikko Kobolog: just want to see if thats where it fails
[14:13] mikko and why
[14:18] Kobolog no, it doesn't fail there
[14:20] mikko hmm
[14:20] mikko interesting
[14:21] Kobolog also, i stuffed logging into every place in ip.cpp where EINVAL presents, and its not there at all
[14:23] mikko Kobolog: ca you break in zmq::tcp_listener_t::set_address
[14:23] mikko tcp_listener.cpp
[14:23] mikko that's where it should fail
[14:26] Kobolog okay i'm at zmq::zmq_listener_t::set_address (this=0x609390, protocol_=0x607608 "tcp", addr_=0x607f58 "127.0.0.1:5560")
[14:27] mikko finish <enter>
[14:27] mikko and check the value it returns with
[14:27] mikko if it's -1 its there
[14:29] Kobolog yeah
[14:30] Kobolog set_address returned -1
[14:30] Kobolog Run till exit from #0 zmq::tcp_listener_t::set_address (this=0x609460, protocol_=<value optimized out>, addr_=<value optimized out>, backlog_=100) at tcp_listener.cpp:175 zmq::socket_base_t::bind (this=0x6076d0, addr_=<value optimized out>) at socket_base.cpp:337 337 alloc_assert (listener); Value returned is $2 = -1
[14:31] mikko ok
[14:31] mikko so it's tcp_listener.cpp
[14:31] mikko maybe restart debugging and step the method (zmq::tcp_listener_t::set_address)
[14:31] mikko to see whats the root cause
[14:32] Kobolog okay
[14:36] Kobolog well
[14:36] Kobolog its here s = open_socket (addr.ss_family, SOCK_STREAM, IPPROTO_TCP);
[14:36] Kobolog s == -1
[14:36] Kobolog tcp_listener.cpp:62
[14:37] mikko if you look at ip.cpp for int zmq::open_socket (int domain_, int type_, int protocol_)
[14:37] mikko it looks very simple method
[14:50] Kobolog it int s = socket (domain_, type_, protocol_); which fails
[14:50] Kobolog okay
[14:50] Kobolog its CLOEXEC
[14:50] Kobolog which fails
[14:50] Kobolog i removed the line and it worked fine
[14:51] mikko interesting
[14:51] mikko because i dont think that line is new
[14:51] Kobolog because we're running the 2.6.24-23 kernel
[14:52] Kobolog and CLOEXEC appeared on 2.6.27
[14:52] Kobolog it is
[14:52] Kobolog i mean, there's something in the changelog about it
[14:52] Kobolog in 2.1.10
[14:52] mikko hmm
[14:52] mikko interesting
[14:53] Kobolog i mean, this line will never work on kernels < 2.6.27
[14:53] mikko so SOCK_CLOEXEC is still defined on your platform?
[14:53] Kobolog i think the problem is
[14:53] Kobolog we're running lucid
[14:53] Kobolog which by default comes with 2.6.32
[14:53] Kobolog but we're running 2.6.24-openvz
[14:53] Kobolog so we have new libc
[14:53] Kobolog but old kernel
[14:53] Kobolog oh god ;/
[14:54] mikko Kobolog: i think that is fairly easy to detect in the build system
[14:54] Kobolog thanks for the help though =)
[14:54] mikko i can wrap up a patch when i get home
[14:55] mikko in the meanwhile, can you open this as an issue in http://zeromq.jira.com ?
[14:55] Kobolog yeah sure
[14:55] mikko that would make getting the fix in slightly faster
[14:55] mikko you can assign it to me (mikko koppanen)
[14:56] mikko i'll try to create a vm tonight and roll a patch for it
[14:56] Kobolog yeah, will do in a couple of minutes
[14:56] mikko thanks
[15:09] mikko Kobolog: i think i got vm ready actually
[15:10] mikko my debian happened to have 2.6.26 image lying around
[15:30] Kobolog so i don't need to file a bug?
[15:30] mikko Kobolog: it would be nice if you can
[15:30] mikko because there needs to be an issue for all fixes to 2.1
[15:30] mikko also, allows tracking progress more easily
[15:32] Kobolog okay
[15:39] Kobolog here's the issue https://zeromq.jira.com/browse/LIBZMQ-273
[15:39] Kobolog i cant assign it to anyone as i don't have the rights, so you gotta reassign it yourself
[15:39] Kobolog 2mikko
[15:43] mikko thanks
[15:48] Guest63712 Hi folks. I'm hoping someone can tell me what I'm doing wrong. I'm using zmq's C interface and I'm running out of FDs even though I'm closing them. This is on 2.1.10.
[15:49] mikko Guest63712: are you using a lot of FDs ?
[15:49] Guest63712 I start a thread and run a device on it, then I have several threads writing to that device thread via inproc
[15:52] Guest63712 mikko: there are a lot of things using FDs, and I have my FD limit set to 100K. All was good until I started with this portion of the project and it's zmq_socket that reports it's can't open a new FD because "too many open files"
[15:53] Guest63712 This client portion does zmq_socket, zmq_connect, zmq_send, and zmq_close in that order.
[15:54] Guest63712 the device thread does a bind on the frontend and a connect on the backend. That's working fine.
[15:56] mikko Guest63712: are the clients very short lived?
[15:56] Guest63712 yes
[16:03] Guest63712 strange part is that the lack of FDs are only affecting zmq and nothing else
[16:05] mikko Guest63712: interesting
[16:05] mikko i need to do my commute in a min but will be back soon(ish)
[16:24] Guest63712 I'm going to try setting LINGER to 0 on the client side of the inproc.
[16:44] Guest63712 Setting LINGER to 0 seemed to do it. Thanks.
[16:47] mvolkov hi, Background: Using 0mq with Python in fan out schema to distribute work among workers. However, workers due to work intensity can't process signals fast enough.
[16:48] mvolkov Q: What options 0mq gives on creating feedback loop to throttle the signal intensity?
[16:56] mvolkov Basically signals get lost and only fraction is been processed on the worker side.
[16:56] mvolkov it is PUSH …. PULL scenario
[16:58] mvolkov So I am looking for "on demand PULL" -- give me signal when I am ready.
[16:58] mvolkov or may be I am climbing the wrong tree here? :-)
[16:59] mvolkov thanks in advance
[17:00] cremes mvolkov: you need to put your own "flow control" on top; the 0mq queues, kernel buffers, messages in flight, etc
[17:01] cremes all make it impossible to let the library handle flow control directly
[17:01] cremes look at the guide and search for flow control
[17:01] mvolkov cremes: thank you for the reply
[17:02] mvolkov would sockets influence any possible choices in this regard? or it doesn't really matter?
[17:02] cremes let the guide be your guide :)
[17:02] cremes brb
[17:02] mvolkov :-) fair enoght
[17:02] mvolkov thank you
[17:20] przemek So I've built zmq and jzmq using VS2010 successfully. I know the libzmq.dll & jzmq.dll need to be in the system path but that's about all I'm able to figure out in terms of getting things setup so that I can start using zmq with java bindings
[17:20] przemek can someone point me to some documentation on where I should be putting all these files that where generated during my build
[17:20] przemek s
[17:22] przemek from what I understand teh files generated in the zeromq/lib & jzmq/lib directories are the important files
[17:22] przemek but I'm not sure how/where I'm supposed to include these files in my java project
[17:38] przemek ok so I think i have figure it out. I ran the unit tests that come with jzmq in the jzmq/test/src/org/zeromq folder and 2/3 of the tests pass
[17:39] przemek the third one testReqRep just sits there
[17:39] przemek it doesn't fail
[17:39] przemek just sits there
[17:40] przemek either way. I assume that since I can atleast run the test java files that means that zmq and jzmq are setup correctly on my system.
[18:15] aroman hi, i'm trying to build an as-simple-as-possible example of zmq's xreq + xrep pairing in PyZMQ
[18:15] aroman the example from the official tutorial works fine, but it uses threads instead of actual separate processes for the convenience of one file
[18:16] aroman i'm trying to split it into two, yet it's not working.
[18:16] aroman here's a quick paste: http://paste.pocoo.org/show/496197/
[18:18] aroman any ideas as to what i'm doing wrong?
[18:28] cremes przemek_: it would be wonderful if you could edit the wiki to clarify any steps you had to take to get the java bindings working
[18:29] cremes aroman: the line that you commented out that sets IDENTITY is necessary
[18:29] cremes as are the lines that send 'A'
[18:29] cremes don't comment out anything you don't understand, pls :)
[18:43] aroman cremes: i commented it out because it wasn't working *with* it
[18:43] aroman (which it still isn't, I just tried again)
[18:45] aroman that is, http://paste.pocoo.org/show/496210/
[19:08] aroman ahh figured it out
[19:08] aroman was a race condition
[19:08] aroman ahh figured it out. turns out the comment on the first sleep() call was a lie
[19:08] aroman it's not only used to give time for the threads to initiatilize, but also for zmq's socket to initialize.
[19:28] gbucher Hi there ! Is there a way to have a non-blocking ZMQ_PUSH socket ? (when reaching HWM)
[19:30] cremes gbucher: zmq_send() with the ZMQ_NOBLOCK flag
[19:30] gbucher cremes: thanks. I was looking at a general option on the socket and did not look there.
[20:07] Rotham hey..
[20:09] Rotham are there any python libraries similar to celery built ontop of zeromq?
[20:11] minrk Salt, maybe
[20:12] minrk IPython.parallel also does some task farming with zeromq, but not targeted at the same use cases as Celery, I think
[20:15] Rotham thanks
[20:31] aroman anyone have an idea why my nodeJS router socket only gets the socket IDENTITY of dealers connecting to it?
[20:43] aroman nevermind, I figured it out
[20:44] aroman (It passes each message part as an argument to the function, not as keys in an Object)
[20:44] aroman (or indicies in an array)
[22:29] CIA-79 rbzmq: 03James Ladd 07 * r710e98b 10/ rbzmq.c : Don't reference string length twice. - http://git.io/sQDmHw
[22:29] CIA-79 rbzmq: 03Brian Buchanan 07 * r179e71c 10/ rbzmq.c : Merge pull request #15 from jamesladd/master ...
[23:04] zerodefect some 'newcomer' zeromq questions. I would like to create a start network topolgy where node A, B, D, and E are all connected to node C.
[23:04] zerodefect is it possible, for example, for node E to receive node A's messages?
[23:22] minrk @zerodefect, I recommend reading the guide, so you get a good sense of what zeromq socket types match your message pattern.
[23:24] zerodefect @minrk. Thanks for your response. I'll have another go at reading the different message patterns.
[23:24] minrk The short answer is yes, but without much more information, it's impossible to say how
[23:25] minrk The guide is really great, give it a few passes, and you will get a better understanding of just what zeromq is and how to use it.
[23:25] zerodefect @minrk I might have to dip my hands into some of the code to get a good feel, I reckon. Great guide indeed!
[23:26] minrk And you will have better vocabulary to bring here or to the list, for a more concrete question people can help with
[23:27] zerodefect thanks again
[23:28] minrk sure thing
[23:46] beobal 4~4~

Web site design and content is copyright © 2010-2011 Peter Alexander, ProDataLab. Site content licensed under the Creative Commons Attribution-Share Alike 3.0 License. ØMQ is copyright © Copyright © 2007-2011 iMatix Corporation and Contributors. Contact us for professional support. ØMQ is free software licensed under the LGPL. ØMQ, ZeroMQ, and 0MQ are trademarks of iMatix Corporation. Terms of Use ? Privacy Policy