[Time] Name | Message |
[05:34] sustrik
|
mike8901: no
|
[05:35] sustrik
|
can you check out where's the other error_t included?
|
[05:35] sustrik
|
defined*
|
[05:38] sustrik
|
if it was a type, it wouldn't be a problem - 0MQ error_t is defined within namespace zmq - the define is worse
|
[05:39] sustrik
|
bgranger: it's almost untested
|
[05:39] sustrik
|
any problems with it?
|
[09:36] CIA-15
|
zeromq2: 03Steven McCoy 07master * r7cb076e 10/ (3 files): Defer NetBSD atomic ops to GCC builtins. Revert Sun atomic ops #define. - http://bit.ly/bJ10EV
|
[15:20] CIA-15
|
zeromq2: 03Martin Sustrik 07master * rc193fd1 10/ (18 files in 3 dirs): lock-free polling removed; ZMQ_POLL flag removed - http://bit.ly/csD6bs
|
[15:31] CIA-15
|
zeromq2: 03Martin Sustrik 07master * r37128b7 10/ (10 files): fd_signaler_t renamed to signaler_t - http://bit.ly/d8kupZ
|
[15:46] bgranger
|
sustrik: I am back, I might have found a bug with the new XREP/XREQ stuff, but I am not sure.
|
[15:46] sustrik
|
hi
|
[15:46] sustrik
|
what bug?
|
[15:46] bgranger
|
But I want to make sure I understand what it should be doing
|
[15:46] bgranger
|
The identity string should still be there, as the first part of a multipart message?
|
[15:47] sustrik
|
in REQ/REP
|
[15:47] sustrik
|
yes
|
[15:47] bgranger
|
In XREP/XREQ
|
[15:47] sustrik
|
well, it works in this way:
|
[15:47] sustrik
|
each node identifies itself when the connection is created to its peer
|
[15:48] sustrik
|
when REQ sends a request, the only prefix is empty message part
|
[15:48] bgranger
|
Is this the same as the ZMQ_IDENTITY
|
[15:48] sustrik
|
no, it's just a delimiter
|
[15:48] bgranger
|
OK
|
[15:48] sustrik
|
then on the XREP side XREP sockets adds new prefix
|
[15:49] sustrik
|
the identity of the sender
|
[15:49] bgranger
|
Right
|
[15:49] sustrik
|
XREQ socket sends the message further with no change
|
[15:49] sustrik
|
at the end of the chain there's a REP socket
|
[15:49] sustrik
|
which extracts all the prefixes from the message
|
[15:49] sustrik
|
until it finds an empty message part (the delimiter)
|
[15:50] sustrik
|
the rest is passed to the user
|
[15:50] bgranger
|
OK
|
[15:50] sustrik
|
when user send a reply
|
[15:50] bgranger
|
I am using a single XREP socket, so when I do a recv, I see the identity string
|
[15:50] sustrik
|
REP socket attaches all the prefixes from the request to the reply and sends it further
|
[15:50] sustrik
|
let me finish
|
[15:50] bgranger
|
OK
|
[15:51] sustrik
|
XREQ socket passes the reply further unchanged
|
[15:51] sustrik
|
XREP socket strips the first prefix and uses it to forward the reply to the right connection
|
[15:51] sustrik
|
finally REQ socket strips the delimiter and returns the reply to the user
|
[15:51] sustrik
|
that's all
|
[15:51] bgranger
|
This is in a configuration where an XREP forwards to an XREQ.
|
[15:52] sustrik
|
right, standard shared queue thing
|
[15:52] sustrik
|
now you have XREQ/XREP setting, right?
|
[15:53] bgranger
|
Right
|
[15:53] sustrik
|
so this is what is expected:
|
[15:53] bgranger
|
I am not forwarding the XREP onto an XREQ
|
[15:53] sustrik
|
you send a request to XREQ socket
|
[15:53] sustrik
|
retrieve is from XREP socket
|
[15:54] sustrik
|
it should be prefixed by XREQ's identity
|
[15:54] bgranger
|
Yes
|
[15:54] sustrik
|
however, you shouldn't see the identity on the wire
|
[15:54] bgranger
|
Right
|
[15:54] sustrik
|
(it's passed on connection initiation only)
|
[15:54] sustrik
|
when you send a reply
|
[15:54] sustrik
|
first prefix is used to route the message to the right XREQ socket
|
[15:55] sustrik
|
if there's no such peer, the reply is dropped
|
[15:55] bgranger
|
Right, we had that working fine before the multipart message stuff.
|
[15:55] sustrik
|
once again, the prefix should not be visible on the wire
|
[15:55] sustrik
|
also, you should receive the reply _w/o_ the prefix
|
[15:55] sustrik
|
that's it
|
[15:55] bgranger
|
Right
|
[15:56] bgranger
|
Here is what we are seeing now:
|
[15:56] bgranger
|
s = c.socket(zmq.XREP)
|
[15:56] bgranger
|
s.bind('tcp://127.0.0.1')
|
[15:56] bgranger
|
s.recv()
|
[15:56] bgranger
|
'\x00\xba\xd2T\x96\xd6\x8eL\xab\x84\x19\xa9\xa5\xe6\x1f\x95\xeb'
|
[15:56] bgranger
|
s.rcvmore()
|
[15:56] bgranger
|
False
|
[15:57] bgranger
|
s.recv()
|
[15:57] bgranger
|
'themessage'
|
[15:57] bgranger
|
Is this correct? I was expecting rcvmore() to return True (more is coming).
|
[15:57] sustrik
|
yeah
|
[15:57] sustrik
|
looks like a bug
|
[15:58] bgranger
|
So, the identity should show up as the first part of a multipart message in this type of situation?
|
[15:58] sustrik
|
yes
|
[15:58] sustrik
|
and it's there AFAIU
|
[15:58] sustrik
|
(auto-generated UUID)
|
[15:59] sustrik
|
have you set identity manually on the XREQ socket?
|
[15:59] bgranger
|
No, not in this case
|
[15:59] bgranger
|
Should I
|
[15:59] sustrik
|
then it's ok
|
[15:59] sustrik
|
the False is bug though
|
[15:59] sustrik
|
i'll have a look at it shortly...
|
[15:59] bgranger
|
How is the auto-generated UUID related to the ZMQ_IDENTITY if it is set?
|
[16:00] sustrik
|
UUID is generated only if there's no identity
|
[16:00] bgranger
|
OK, so I should see the ZMQ_IDENTITY there if it has been set
|
[16:00] sustrik
|
right
|
[16:00] bgranger
|
Also, what do you think of the socket.rcvmore() API?
|
[16:00] bgranger
|
I also wrapped getsockopt as well.
|
[16:01] sustrik
|
it's shorthand for getsockopt (ZMQ_MORE) ?
|
[16:01] bgranger
|
Yes
|
[16:01] sustrik
|
yes, seems reasonable
|
[16:01] bgranger
|
Isn't it ZMQ_RCVMORE though?
|
[16:01] sustrik
|
:)
|
[16:01] sustrik
|
lemme see
|
[16:01] bgranger
|
Cool, thanks!
|
[16:02] sustrik
|
right RCVMORE :)
|
[16:02] bgranger
|
The multipart stuff is fantastic though - will *really* help us
|
[16:03] bgranger
|
I will write a test suite for the multipart stuff to help test it all
|
[16:03] CIA-15
|
zeromq2: 03Martin Sustrik 07master * r1ffc6dd 10/ (configure.in src/signaler.cpp src/signaler.hpp): eventfd-style signaling removed - http://bit.ly/9Jk5gv
|
[16:09] sustrik
|
bgranger: great, test suite will be of much help
|
[17:44] sustrik
|
bgranger: i don't seem to have an option to close the issue
|
[17:45] sustrik
|
presumably only the original create of the issue can close it
|
[17:45] bgranger
|
Which issue?
|
[17:45] bgranger
|
I just close the one about ZMQ_IDENTITY a second ago
|
[17:45] sustrik
|
ah, good
|
[17:45] sustrik
|
thanks
|
[17:46] bgranger
|
sustrik: so is ZMQ_POLL gone for good?
|
[17:47] sustrik
|
yes
|
[17:47] sustrik
|
well, it's still there to retain backward compatibility
|
[17:47] sustrik
|
but it does nothing
|
[17:48] sustrik
|
zmq_poll can be used by default now
|
[17:48] CIA-15
|
pyzmq: 03Brian E. Granger 07master * r4bf8945 10/ (zmq/_zmq.c zmq/_zmq.pyx): Fixing but in Python bindings to getsockopt with IDENTITY. - http://bit.ly/atHUvJ
|
[17:48] bgranger
|
Nice, wasn't there a small performance issue before?
|
[17:49] sustrik
|
it's still there
|
[17:49] sustrik
|
the impact is ~2us on the latency side
|
[17:49] sustrik
|
on Linux
|
[17:50] sustrik
|
but the advantages seem worth of it
|
[17:50] sustrik
|
1. no annoying ZMQ_POLL flag
|
[17:50] sustrik
|
2. any number of threads can be used! even >64
|
[17:51] sustrik
|
3. significant codebase simplification
|
[17:59] bgranger
|
Seems like a good trade off. I like simplicity :)
|
[18:34] CIA-15
|
zeromq2: 03Martin Sustrik 07master * rae93ed3 10/ (12 files): signaler rewritten in such a way that any number (>64) of threads can be used - http://bit.ly/chon4a
|
[18:53] CIA-15
|
zeromq2: 03Martin Sustrik 07master * rbeb4da3 10/ (builds/msvc/libzmq/libzmq.vcproj src/signaler.cpp): windows port fixed - http://bit.ly/9BogHN
|