IRC Log

Monday October 17, 2011

[Time] NameMessage
[06:58] longlong who use xpub/xsub
[06:59] sustrik hi, i've coded it :)
[06:59] longlong oh, you can display your demo
[07:00] longlong my code don't work
[07:00] longlong pub/sub is ok
[07:04] longlong sustrik: here?
[07:07] sustrik yu[
[07:07] sustrik create a minimal test case and post it to gist, i'll have a look
[07:11] longlong sustrik: zmq::socket_t socket (context, ZMQ_XSUB); socket.setsockopt(ZMQ_SUBSCRIBE, "", 0); the second is error:what(): Invalid argument
[07:11] longlong but use ZMQ_SUB is ok, why?
[07:12] sustrik XSUB treats the subscriptions as upstream messages
[07:12] sustrik have a look here:
[07:12] sustrik http://www.250bpm.com/pubsub
[07:12] longlong yes
[07:12] longlong thanks
[07:13] sustrik "Subscription messages are composed of a single byte, 0 for unsubscription and 1 for subscription, followed by the subscription itself, ie. the prefix we want to check the messages for."
[07:31] longlong sustrik: in other word, xsub isn't set ZMQ_SUBSCRIBE?
[07:33] sustrik ?
[07:33] sustrik you should send the subscription message by hand if you want to use XSUB
[07:34] sustrik feel free to use SUB though
[07:34] sustrik there's no difference except for this mechanism of sending subscriptions
[07:35] longlong i am chinese, my English is bad, sorry!
[07:37] longlong i want know, why "socket.setsockopt(ZMQ_SUBSCRIBE, "", 0);" is error, how make it rigth?
[07:41] sustrik use ZMQ_SUB
[07:42] longlong can not use ZMQ_XSUB ?
[07:42] sustrik you can, but why?
[07:43] longlong i send subscription message to xsub, it will give me a message that i want get?
[07:44] sustrik yes
[07:45] longlong it implement by zmq? in other word, xsub reply me auto ?
[07:49] longlong xpub/xsub is designed for midle device ?
[07:56] sustrik yes, it's for middle device
[07:56] sustrik if you are at the endpoint simply use SUB
[07:57] mikko xsub doesn't send subscription message on setsockopt (ZMQ_SUBSCRIBE ?
[07:58] sustrik nope, xsub is meant for intermediate devices
[07:59] sustrik so it treats subscriptions as simple messages
[07:59] sustrik going upstream
[07:59] sustrik that way the boilerplace device code works ok for subscription forwarding
[08:07] longlong sustrik: thank you very much
[09:36] longlong how sent message to xpub?
[09:37] sustrik zmq_send()
[09:38] longlong but xpub can not receive
[09:39] longlong it is special that send to xpub ?
[09:43] sustrik the subscriptions act as messages going in the opposite direction
[09:43] sustrik however
[09:43] sustrik why don't you simply use PUB/SUB?
[09:46] longlong now, i studying
[09:46] longlong i am studying
[09:51] longlong oh, cann't use zmq_recv on xpub, because zmq has recv the message(from subscriptions)?
[09:52] sustrik why are you using xpub/xsub in the first place?
[09:52] sustrik pub/sub should work ok
[09:54] longlong i'am studying zeromq, i want make the smallest demo. so i use xpub/xsub in endpoint. :)
[10:02] sustrik why?
[10:03] sustrik the endpoints should use pub/sub
[10:09] longlong by your means, i must create midle device. i need simple, so i try.
[10:10] longlong how i receive from xpub?
[10:14] sustrik zmq_recv()
[10:14] longlong message form?
[10:14] longlong i try receive, but block
[10:17] sustrik which presumably means there's no subscription message to receive
[10:20] longlong xpub receive message from xsub(it send subscription message)?
[10:51] sustrik yes
[11:07] mikko sustrik: there?
[11:12] ptrb if I destroy a ZMQ_PUB socket and recreate it (same endpoint) without a sleep(1), it fails. what gives?
[11:14] sustrik mikko: hi
[11:14] sustrik ptrb: fails with what error?
[11:14] mikko sustrik: https://github.com/zeromq/libzmq/blob/master/src/zmq.cpp#L606
[11:15] sustrik mikko: yes?
[11:15] mikko if (unlikely (rc == -1) {
[11:15] mikko (( )
[11:15] sustrik ugh
[11:15] sustrik let me fix that
[11:16] mikko was on cppcheck results
[11:16] sustrik btw, how it's going with the infrastructure?
[11:16] sustrik still broken?
[11:16] mikko src/zmq.cpp606syntaxErrorerrorInvalid number of character (() when these macros are defined: 'ZMQ_FORCE_SELECT'.
[11:16] mikko it's on stable(r) hardware now
[11:17] mikko the blade refuses to recognize hardware extension card
[11:17] ptrb sustrik: not 100% sure; probably "address in use". the point is, zmq_close() isn't blocking. is that right?
[11:17] sustrik i see
[11:17] mikko so need to juggle a bit more at some point
[11:17] mikko but should be pretty stable now
[11:17] sustrik ptrb: no, it's not
[11:17] mikko i pushed a couple more snaphots to http://snapshot.zero.mq/
[11:17] mikko windows and php on windows
[11:17] sustrik great
[11:18] mikko pieter wants that to be moved to downloads. at some point
[11:18] sustrik it's up to you
[11:19] sustrik you can of course use your builds to promote yourself, thus keeping the packages on your site
[11:19] sustrik or maybe you don't care
[11:20] ptrb sustrik: so zmq_close() and immediately zmq_bind(), same endpoint, should work?
[11:20] sustrik btw, sorry for long reponse times lately. i'm writting a chapter for aosabook.org and the deadline is in couple of days
[11:21] sustrik ptrb: not necessarily
[11:21] sustrik shutdown is done asynchronously
[11:21] mikko sustrik: doesn't really matter
[11:21] ptrb ...right, hence zmq_close() isn't blocking. right?
[11:21] mikko sustrik: those are not much additional hassle now
[11:21] ptrb can I poll for status?
[11:21] mikko sustrik: i might convert them at some point
[11:22] sustrik ok
[11:22] sustrik ptrb: no, the handle is invalid after you call zmq_close()
[11:23] ptrb sustrik: then how might I accomplish my use case?
[11:23] sustrik in theory, the shutdown mechanism can be altered to wait for listerenrs to un-bind
[11:24] sustrik do you want to give it a try?
[11:25] ptrb Well, I could. It would (apparently) change the semantics of close(). Is that desirable?
[11:55] sustrik ptrb: yes, but you have to differentiate
[11:55] sustrik close listeners synchronously
[11:56] sustrik however, send pending outbound messages asynchronously
[13:51] cremes sustrik: over the weekend mikko was getting my ruby bindings working with the CI server
[13:52] cremes part of that work is getting my specs/tests running for 2.1 and 3.0
[13:52] cremes for the 3.0 run, there are quite a few failures using ZMQ_FD
[13:52] cremes http://build.zero.mq/view/ffi-rzmq/job/ffi-rzmq-master_libzmq3-0_GCC-debian/4/console
[13:52] cremes have those code paths been changed since the 2.1 release?
[13:53] cremes (my overall point is that maybe these spec failures indicate *real* bugs in 3.0)
[13:54] jsia hi, I'm using the C api for zmq
[13:54] jsia when I try to rcv multipart mesasge
[13:55] jsia the first part that I'm receiving is always blank
[13:55] jsia why is that so?
[13:55] cremes jsia: are you receiving on a XREP socket? if not, what socket type are you receiving on?
[13:55] jsia the one that is sending is XREQ
[13:56] jsia and the one receiving is using poll
[13:56] cremes jsia: what kind of socket is registered with poll?
[13:56] jsia ZMQ_ROUTER
[13:56] cremes ok, and you are on 2.1.x presumably
[13:56] jsia yup 2.1.10
[13:57] jsia when I did this in python Im receiving the message on all parts
[13:57] cremes read two pages: http://www.zeromq.org/tutorials:xreq-and-xrep
[13:57] cremes and
[13:57] cremes http://www.zeromq.org/sandbox:dealer
[13:57] cremes in 2.1.x, XREP is ROUTER is XREP
[13:58] cremes jsia: i recommend you go through the guide again (zero.mq/zg)
[13:58] cremes this is all covered in detail
[13:58] jsia ok btw here is a code snippet of what I use to receive the message from a multipart
[13:58] jsia emcpy (string, zmq_msg_data (&message), size);
[13:58] jsia memcpy (string, zmq_msg_data (&message), size);
[13:59] cremes i hope you are getting size from zmq_msg_size()
[14:00] jsia yup
[14:00] jsia it works on the second part of the message without the rcvmore
[14:00] jsia I got the second message
[14:10] mikko cremes: do you get these FD failures as well?
[14:10] cremes mikko: when testing with 3.02 i do get them
[14:10] mikko cremes: what is the test testing?
[14:11] cremes the spec is confirming that the FD returned by zmq_getsockopt(ZMQ_FD) is a valid FD
[14:11] mikko how does it confirm?
[14:12] cremes it tests that by using getsockopt() from libc
[14:12] cremes it makes a getsockopt call and verifies the return code is 0
[14:12] cremes e.g. it tries to retrieve the size of SO_RCVBUF
[14:12] cremes works great with 2.1.x, fails with 3.x
[14:14] mikko cremes: hmm, interesting
[14:14] mikko cremes: can you reproduce this with raw c code?
[14:14] cremes if i get some free time later today, i'll write a C version
[14:14] cremes :)
[14:15] mikko cremes: does the test call connect before getting the fd?
[14:15] cremes no
[14:16] cremes it just creates a socket and calls getsockopt on it
[14:16] cremes you know how to read ruby code, right?
[14:16] cremes that might be easiest... ;)
[14:17] cremes https://github.com/chuckremes/ffi-rzmq/blob/master/spec/socket_spec.rb#L445
[14:25] mikko cremes: https://gist.github.com/974d39d431e3896bffa4
[14:25] mikko succeeds
[14:25] cremes hmmm...
[14:26] cremes wtf...
[14:27] mikko let me try raw c in a min
[18:11] pieterh cremes: you around?
[18:11] cremes pieterh: yessir
[18:12] pieterh any suggestions for the meetup? I mean, places?
[18:13] cremes good question
[18:13] pieterh I see you guys are already in winter mode...
[18:13] pieterh well, autumn at least
[18:13] cremes i was going to post to the ML today and ask if any company wanted to sponsor or host a meetup
[18:13] pieterh nice idea
[18:13] cremes otherwise we just need to pick a bar that is *not* too busy on a thursday night and take it over :)
[18:14] pieterh my flight gets in at 4.30 so I guess 5.30pm or later is doable
[18:14] cremes ok
[18:14] cremes i'll use that as a guide
[18:14] pieterh I'll leave this in your hands then... I made a wiki page already
[18:15] cremes i'll be sure to put a link to that page in my post
[18:15] pieterh nice, looking forward to it
[18:37] cremes chicago meetup details posted
[18:37] cremes wiki updated
[18:42] mikko cremes: https://gist.github.com/37ac92d3619fa5e9fa36
[18:42] mikko succeeds here
[18:48] cremes <sigh>
[19:02] jond mikko, cremes: the fd returned is that of the mailbox which ultimately comes from socketpair in 2.1 , in 3.0 it might be created by eventfd depending on platform. what's errno?
[19:03] cremes jond: ENOTSOCK
[19:03] jond what platform?
[19:03] cremes linux
[19:03] jond that'll be it then
[19:03] mikko does it succeed on mac?
[19:04] jond mac will use socketpair
[19:04] mikko i thought mac fails as well
[19:04] mikko let me run linux C test
[19:04] mikko C++ one succeeded on linux
[19:04] jond ok maybe, does mac have eventfd?
[19:05] mikko C one fails on linux
[19:05] jond ie : int eventfd(unsigned int initval, int flags);
[19:05] jond on 3.0
[19:05] mikko i guess it's fair breakage
[19:06] mikko as the socket fd is strictly for polling
[19:06] jond i suspect the fd from eventfd is not a socket.
[19:06] jond agreed
[19:06] jond it's for polling then, calling ZMQ_EVENTS stuff
[19:06] cremes 3.0 FD stuff succeeds on osx
[19:07] mikko cremes: so there was some use in adding it to CI :)
[19:07] jond yes, because it uses socketpair
[19:07] cremes mikko: ha!
[19:07] cremes jond: so how would you recommend testing that zmq_getsockopt(ZMQ_FD) is returning a valid socket FD?
[19:07] cremes for 3.0 on linux
[19:08] mikko cremes: >0
[19:08] jond cremes : i am reading the eventfd manpage.......
[19:08] cremes that's it? fd > 0 ?
[19:08] jond mikko : good call
[19:09] cremes looks like FDs returned from eventfd() can have read/write/poll/select/close called on them
[19:09] cremes which explains why getsockopt() isn't working :)
[19:10] jond exactly
[19:11] jond good catch though!
[19:12] cremes so, is the recommendation to just test >0 for 3.0 on linux?
[19:12] cremes or should i try to write/poll or something instead?
[19:12] mikko cremes: not sure if your extension needs to test more than > 0
[19:13] mikko because i guess the test should test whether the value is passed back from zeromq properly
[19:13] cremes mikko: my extension doesn't care... i'm really testing the underlying 0mq library
[19:13] cremes is *it* returning the right thing
[19:13] mikko in that case poll () might be ok
[19:14] jond you mustnt try to write to it. I'm just wondering whether there isnt an upper bound on the fd as well. Is this just a test case?
[19:14] cremes jond: yes, this is a test case
[19:14] mikko just poll with minimal timeout and check that there are no errors
[19:14] cremes mikko: i like it
[19:15] mikko cremes: the ffi-rzmq is built morning and evening GMT
[19:15] mikko around 5am and 5pm iirc
[19:15] mikko assuming libzmq build succeeds
[19:15] cremes ok
[19:15] mikko it kicks off the rest
[19:15] mikko master of linzmq doesn't currently succeed due to hanging test
[19:15] mikko but 2.1 and 3.0 should build
[19:16] cremes mikko: i promise to get these specs running for 2.1 & 3.0 on both osx & linux
[19:16] cremes :)
[19:16] cremes but those other failures for 3.0 i think are *real* bugs
[19:16] mikko bugs in zeromq?
[19:16] cremes yes
[19:17] cremes the PUB & bind spec is a real failure
[19:17] cremes there's an open issue for it
[19:17] mikko thats fine
[19:17] cremes will a few failing specs prevent libzmq from succeeding?
[19:17] mikko no
[19:17] cremes ok
[19:17] mikko only ffi-rzmq
[19:18] mikko http://build.zero.mq/view/ffi-rzmq/
[19:18] mikko the 4.0 hasn't built because of libzmq not testing atm
[19:30] cremes mikko: i have the spec fixed to use poll() for 2.1 & 3.0
[19:30] cremes succeeds on 2.1/3.0 for both osx & linux
[19:30] cremes i'll commit this shortly
[19:50] mikko cremes: looks better now
[19:51] cremes good
[21:58] Saladino hiya
[22:02] mikko Saladino: hi
[22:12] whack I'm seeing some strange behavior with libev and zmq where my ev callback is never invoked
[22:12] whack https://gist.github.com/28d5c870a307766fc1c3
[22:13] mikko whack: what kind of socket is rpc->socket
[22:13] whack from the strace, it seems like the order of events is backwards. Something calls recvfrom() on fd 19 before epoll is told to watch it, so the first message to this event seems to be dropped
[22:13] whack mikko: void *
[22:13] mikko no, not the c type
[22:14] mikko but the socket type
[22:14] whack oh, silly me
[22:14] whack its req/rep, this side is the REQ
[22:14] whack I can verify the remote end sees the request and replies, but the reply is only seen if I invoke rpc_Call_poll myself
[22:14] whack (outside libev)
[22:14] mikko rpc->socket is REQ ?
[22:14] whack I have other code using req/rep (though on the REP side) that works
[22:15] whack rpc->socket = zmq_socket(rpc->zmq, ZMQ_REQ);
[22:15] mikko on REQ read cannot be invoked before write
[22:15] whack the code snippet I showed doesn't do any reading
[22:15] mikko do you get a reply back there?
[22:16] whack I get a reply if I do a sleep then invoke rpc_call_poll explicitly
[22:16] whack what's intended to happen is teh REQ goes out, it sets up an event handler for the reply, and goes about other business
[22:17] mikko do you call ev_loop after that ?
[22:17] whack yup
[22:18] whack well, ev_run technically
[22:18] mikko does ev_run run once?
[22:18] whack nope
[22:18] mikko or does it block until there are events?
[22:19] whack it's used to block forever, handling events that are regsitered accordingly
[22:19] whack this same program uses libev (the same EV loop) with REP to accept rpc calls and reply with results
[22:20] whack I'm implementing calls out of this program, and it seems like libev is not seeing the "ready to read" event
[22:20] whack lemme read over the code and see if there's anything sensitive here and post the full rpc and server implementation
[22:20] mikko are they running on different threads?
[22:20] whack nope
[22:20] whack 1 thread for the main program with libev
[22:20] mikko does it block on ev_run eternally?
[22:21] mikko so just to recap
[22:21] mikko you zmq_send there, your REP socket callback gets called
[22:21] mikko it sends a response
[22:21] mikko and rpc_call_poll doesn't get invoked?
[22:21] mikko what does rpc_call_poll do?
[22:22] whack mikko: ev_run blocks eternally
[22:22] whack https://github.com/jordansissel/experiments/tree/master/c/libev/rpc
[22:22] whack in rpc.c if I uncomment lines 79-83, I am able to read the response.
[22:24] whack rpc_server.c has an example of this rpc stuff functioning, but the receiver instead of the caller
[22:25] mikko where do you call ev_run ?
[22:25] whack elsewhere in the program
[22:25] whack rpc and rpc_server are just ways to register rpc handlers or call rpc methods
[22:26] whack ev_run is not likely related. strace shows libev listening for that event
[22:27] whack https://gist.github.com/28d5c870a307766fc1c3 now includes a piece of the main method if you really need to see the ev_run call
[22:27] whack libev is functioning 100% normally for the rest of the program (which has thousands of event handlers regsitered)
[22:32] mikko hmm
[22:33] mikko it's a handful to read
[22:33] whack C isn't the most compact of languages ;)
[22:34] mikko so you see EV_READ being invoked on REP side?
[22:34] mikko but not on REQ side?
[22:34] whack the request is sent properly, handled, and replied-to
[22:35] whack the reply is not observed unless I do it manually (vs letting libev do it)
[22:35] whack I think the cause is shown by the strace in the gist I posted
[22:35] whack something is calling 'poll' and reads from fd 19 before libev gets to check it
[22:37] whack https://gist.github.com/28d5c870a307766fc1c3#file_strace
[22:38] whack https://gist.github.com/14e28767fbdc9a91e061
[22:39] whack maybe related, but zmq_send seems to check and read from the fd I get from zmq_getsockopt(..., ZMQ_FD)
[22:40] whack probably a red herring I suppose
[22:40] mikko do the client and service run in the same process?
[22:40] whack nope
[22:40] whack one's actually
[22:40] mikko ok, can you test this on REP side
[22:40] whack one's just a javascript tool that handles rpc calls
[22:40] mikko add sleep (1);
[22:40] mikko just to see if there is a race going on
[22:40] whack a race?
[22:40] mikko yes
[22:41] whack I've cut the program down to just the rpc call out, the remote process replies
[22:41] mikko when you send the request, it might've been received before you get to run the ev_run
[22:41] whack what would race with it?
[22:41] whack hmm
[22:41] whack I'll add a delayed send
[22:41] mikko and as the ZMQ_FD is edge triggered rather than level triggered it might mess up
[22:42] mikko there is a small window of race where the state change gets triggered before your event loop starts
[22:42] mikko i guess
[22:43] whack that makes sense actually
[22:43] whack I'll put the rpc call after ev_run happens
[22:43] mikko but ev_run blocks?
[22:43] mikko it's been a while since i've used libeve
[22:43] whack with timers
[22:43] mikko -e
[22:44] whack libev has a timer you can use to prepare a callback to happen after some time, after ev_run starts
[22:44] mikko cool, it would at least exclude the possible race condition there
[22:44] whack in the long term, I think the right case is probably to always do the zmq_send on next tick
[22:44] whack that way libev gets a chance to listen, zmq_send then sends, reply comes, and libev gets notified
[22:46] mikko yes, sounds like a sensible approach
[22:46] mikko i gotta head to bed
[22:47] mikko good night
[22:48] whack later, thanks for the second set of eyes :)
[22:55] whack that was the ticket. I added a zero-delay timer to ensure zmq_send occurs after the start of the next libev iteration allowing it to watch for zmq messages
[23:09] pyth0n hey there
[23:10] pyth0n i keep on getting a "Socket operation on non-socket" error when using zmq_poll
[23:10] pyth0n in c and c++ api
[23:10] pyth0n socket is set up correctly
[23:11] pyth0n i have on poll item
[23:11] pyth0n item[0].socket = mysocket (of type zmq::socket_t)
[23:12] pyth0n item[0].fd = 0; item[0].events = ZMQ_POLLIN; item[0].revents =0;
[23:13] pyth0n socket is bound to a tcp port, as ZMQ_PULL
[23:13] pyth0n it works correctly though when using zmq_poll i get an error
[23:14] michelp pyth0n, it would be most useful if you pasted a snippet of code to a pastebin that reproduced your problem
[23:17] pyth0n http://pastebin.com/tU51MwLR
[23:20] pyth0n the snippet works fine, when i use blocking io
[23:20] pyth0n but with polling i get this error
[23:23] pyth0n any idea?
[23:28] pyth0n uups
[23:29] pyth0n sorry i am stupid
[23:30] pyth0n i had pointer on socket_t .. damn
[23:38] michelp pyth0n, well, I don't work with a lot of C++, unfortunately. Mostly python. The first argument to poll() looks a little weird though, is &_pollitem[0] correct?
[23:39] michelp i don't know the api, but that looks weird referencing the zeroth item

Web site design and content is copyright © 2010-2011 Peter Alexander, ProDataLab. Site content licensed under the Creative Commons Attribution-Share Alike 3.0 License. ØMQ is copyright © Copyright © 2007-2011 iMatix Corporation and Contributors. Contact us for professional support. ØMQ is free software licensed under the LGPL. ØMQ, ZeroMQ, and 0MQ are trademarks of iMatix Corporation. Terms of Use ? Privacy Policy