[Time] Name | Message |
[06:42] umesh
|
Hi
|
[06:42] umesh
|
I am newbie to zeromq
|
[06:43] umesh
|
I have a simple question that ... how we can use the zeromq with the XML-RPC ?
|
[06:43] umesh
|
means any pointers to simple example will help
|
[06:53] sustrik
|
umesh: you have to pair xml-rpc serialisation library with 0mq library
|
[06:55] sustrik
|
zedas: i had a look at the leak you've reported; it seems the debug info is missing and presumably, optimiser messed with the code; can you do the same with -g -O0 built code?
|
[06:55] sustrik
|
zedas: as an option, do you have a test program to reproduce the problem?
|
[06:55] zedas
|
sustrik: that is with -g -O0, it's valgrind on OSX and the retarded dylibs OSX uses
|
[06:56] sustrik
|
hm, strange, the filenames and linenumber seem missing
|
[06:56] zedas
|
sustrik: i do have a test program, but it's my project, so probably take you too long to build it and get it working. http://mulletdb.com/
|
[06:57] zedas
|
i'll run the test on linux and get you line numbers
|
[06:57] sustrik
|
great, thanks
|
[06:57] zedas
|
in the mean time, does your rbzmq actually build? it fails miserably for me because of the test for zmq_init not including the right libraries or header files.
|
[06:58] sustrik
|
not sure, what version are you using 2.0.6?
|
[06:58] sustrik
|
i mean 0mq version
|
[06:59] sustrik
|
unfortunatrly, rbzmq is orphaned at the moment
|
[06:59] sustrik
|
i am taking care of it a bit, but i am not a ruby person...
|
[07:01] zedas
|
latest from git for both 0mq and rbzmq
|
[07:02] zedas
|
alright i'll see what's up and give you a patch
|
[07:02] zedas
|
i'm not a ruby person anymore, but seems i get asked to fix ruby stuff :-)
|
[07:02] sustrik
|
that would be great, thanks :)
|
[07:02] umesh
|
sustrik: Is there any some real time example which demostrate xml-rpc and zeromq
|
[07:03] sustrik
|
umesh: i am not aware anyone had done that yet
|
[07:03] sustrik
|
you may ask on the mailing list
|
[07:03] zedas
|
sustrik: http://dpaste.de/odLV/ there ya go
|
[07:04] zedas
|
that might be a slightly older git checkout than is current, but i can rebuild if that doesn't pinpoint the spot
|
[07:04] sustrik
|
zedas: interesting, is that the only leak there?
|
[07:05] zedas
|
that's the only one i can find
|
[07:05] sustrik
|
ok
|
[07:05] zedas
|
the others are stock sets of ram for say arc4, threads, etc.
|
[07:05] zedas
|
this spot is the only one that grows perpetually and causes server death
|
[07:05] zedas
|
oom killer
|
[07:06] sustrik
|
one leak for each connection created, right?
|
[07:06] zedas
|
no, i think one for each message passed
|
[07:07] sustrik
|
hm
|
[07:07] zedas
|
so i get 2MB of leaks from about that much data transmitted, but only a few hundred connections
|
[07:07] zedas
|
but, i could be wrong. maybe the unit tests are doing more than i think.
|
[07:07] zedas
|
or, 0mq is doing more connections behind the scenes
|
[07:07] zedas
|
let me get you the code that does this, it's pretty simple
|
[07:08] sustrik
|
ok
|
[07:08] zedas
|
http://dpaste.de/SHso/
|
[07:08] zedas
|
that's it
|
[07:08] zedas
|
nothing else in the system does 0mq, and that's the entire 0mq usage
|
[07:09] zedas
|
and all the tests are over the REQ/REP socket, not the pub/sub socket
|
[07:09] zedas
|
so lines 84-86, which calls perform_request
|
[07:10] zedas
|
ohhhhh hey, see how i'm passing in the same socket as an in and out socket? could that be causing it?
|
[07:10] zedas
|
perform_request(in_socket, in_socket)
|
[07:10] zedas
|
they're by ref (&) so not sure, but i'm not clear how the memory handling is done
|
[07:11] sustrik
|
the strange thing that the leak, as reported by valgrind
|
[07:11] sustrik
|
is tied to connection creation
|
[07:11] sustrik
|
how many client do you have
|
[07:11] sustrik
|
just one?
|
[07:14] zedas
|
yep.
|
[07:14] zedas
|
i can show you the python code for the client, it's not much
|
[07:15] sustrik
|
wait a sec
|
[07:16] sustrik
|
no luck :(
|
[07:16] zedas
|
http://dpaste.de/RyrE/
|
[07:16] sustrik
|
thx
|
[07:16] zedas
|
that's what makes the connection
|
[07:17] zedas
|
and i only make one for the whole test code run
|
[07:17] zedas
|
so, i can try putting in an explicit close, but iirc there isn't one in the python library, similar to the c++ library
|
[07:18] sustrik
|
client is perfectly ok
|
[07:18] sustrik
|
the most basic use case
|
[07:18] zedas
|
yep, super vanilla
|
[07:18] sustrik
|
i'll try to run a code similar to one in your server
|
[07:18] sustrik
|
we'll see...
|
[07:18] zedas
|
and the server isn't doing much either. it gets the data out, and then hands it to another class or two
|
[07:18] sustrik
|
but the leak is in server, right?
|
[07:19] sustrik
|
sure it is
|
[07:28] zedas
|
yep
|
[07:28] zedas
|
although i could try running a fast loop of requests in a client
|
[07:31] sustrik
|
you get tghe leak with req/rep or pub/sub style?
|
[07:31] zedas
|
req/rep
|
[07:31] sustrik
|
ok
|
[07:37] sustrik
|
the client sends one message, receive a reply and exits
|
[07:37] sustrik
|
or does it run in a loop?
|
[07:38] sjampoo
|
sustrik: interesting plans for the future
|
[07:39] sjampoo
|
btw i just benched the nocopy version of pyzmq http://nichol.as/wp-content/pyzmq-lat.png
|
[07:41] sustrik
|
sjampoo: nice, what about putting in on the website?
|
[07:41] sustrik
|
it's us not ns btw
|
[07:42] sjampoo
|
hah, good catch
|
[07:42] sjampoo
|
I think it would be nice to put it on the website when the nocopy version is release ready
|
[07:43] sustrik
|
you should speak to brian about this
|
[07:44] sustrik
|
i am not sure what's the right place for it
|
[07:44] sjampoo
|
i will
|
[07:44] sustrik
|
feel free to use zeromq.org's "performance" section if you have no better place
|
[07:46] sustrik
|
zedas: i'm running your code but can't produce the leak here :(
|
[07:47] zedas
|
sustrik: client creates the connection, then does a bunch of req/rep tests, then exits
|
[07:48] zedas
|
the server is a daemon that just processes in that loop
|
[07:48] sustrik
|
yes, i've tried both scenarios
|
[07:48] sustrik
|
not luck though
|
[07:48] zedas
|
ok can i see your code. maybe i can root out what's in mine that causes it
|
[07:48] sustrik
|
can you check whether all sockets and the context are properly shut down?
|
[07:49] sustrik
|
http://dpaste.de/WOCm/
|
[07:50] zedas
|
sockets shutdown on the client or server side
|
[07:51] zedas
|
also, this wouldn't be a shutdown leak because the server crashes after repeated hits
|
[07:51] sustrik
|
on the server side i mean
|
[07:51] zedas
|
ok let me do something to this sample you wrote sustrik
|
[07:52] sustrik
|
sure
|
[08:07] zedas
|
sustrik: looks like it's on connect
|
[08:07] sustrik
|
?
|
[08:10] zedas
|
ok this client produces the leak: http://dpaste.de/BcJV/ i'll post my server code next
|
[08:11] zedas
|
http://dpaste.de/zqtK/
|
[08:11] zedas
|
sustrik: not much different from your own except i'm sending back a more reliable message, and freeing it in a free method
|
[08:11] zedas
|
but, if you hit that server with tons of messages, no leak
|
[08:12] zedas
|
it's only if you run the client in the shell with:
|
[08:12] zedas
|
while ./testcli ; do ps aux | grep test | grep -v grep ; done
|
[08:12] zedas
|
then you start to leak ram from each connection
|
[08:13] sustrik
|
let me try
|
[08:13] sustrik
|
ugh, i have to leave for an hour
|
[08:13] sustrik
|
will continue afterwards
|
[08:17] zedas
|
sustrik: no problem.
|
[08:17] zedas
|
i'll just make fewer connetions :-)
|
[08:17] zedas
|
but i think that narrows it down pretty good.
|
[09:03] pieterh
|
@zedas: mulletdb looks pretty neat, do you have use cases for it already?
|
[09:26] zedas
|
pieterh: i'm using it on some dinky projects, but mostly just for fun
|
[09:27] pieterh
|
@zedas: would you like to be listed on the zeromq.org site? I'd like to collect projects using 0MQ
|
[09:29] zedas
|
pieterh: yeah sure, if you like. it's fairly new but i'm contributing it regularly.
|
[09:29] pieterh
|
yes, I read your initial blog
|
[09:30] pieterh
|
well, we can wait a while to see if mulletdb becomes famous :-)
|
[09:31] zedas
|
it's famous now
|
[09:31] pieterh
|
:-)
|
[09:31] jugg
|
I'm using lua bindings, and calling the equivelant of zmq_getsockopt with ZMQ_RCVMORE and the call if failing. Any known issues with that?
|
[09:31] zedas
|
well, famous for being another of my weirdo projects :-)
|
[09:32] pieterh
|
@zedas: do you have a pretty logo yet?
|
[09:42] jugg
|
nevermind, its a bug in the lua bindings.
|
[10:09] sustrik
|
jugg: the point is that getsockopt is a new function
|
[10:09] sustrik
|
so it's probably not yet available via lua binding
|
[10:09] jugg
|
it is, it was incorrectly implemented.
|
[10:09] jugg
|
I've patched it.
|
[10:09] sustrik
|
ah
|
[10:10] sustrik
|
do move the patch upstream
|
[10:10] sustrik
|
bindings are separate products though
|
[10:10] jugg
|
in the middle of doing so... :)
|
[10:14] jugg
|
http://github.com/iamaleksey/lua-zmq/issues#issue/1
|
[10:16] sustrik
|
!
|
[10:34] sustrik
|
zedas: ok, i am runnig your test
|
[10:34] sustrik
|
how do you terminate it?
|
[10:46] sustrik
|
ok, i see
|
[11:55] cremes
|
zedas: for ruby you may want to check out the ruby FFI bindings (it's my project) at http://github.com/chuckremes/ffi-rzmq
|
[11:55] cremes
|
they are getting a lot more love than the official bindings at the moment
|
[11:56] cremes
|
i'm adding specs over the next few days to give some confidence in the release quality
|
[11:57] cremes
|
also, the bindings are not idiomatic ruby; they map to the C functions pretty closely; i am working on a higher-level library to add ruby sugar back in
|
[11:57] cremes
|
i know you're not a ruby guy anymore so pass this info on to whoever needed the help with rbzmq
|
[12:01] cremes
|
oh, and the ffi bindings allow 0mq to work with ruby runtimes other than MRI
|
[13:06] jugg
|
I've set up a REQ/REP pattern, where the REQ socket is connecting, and the REP socket is binding. Is there a way for the REQ socket to determine if a binding exists? Because the REQ socket send must subsequently block on recv waiting for the reply, I'd rather not have start sending messages through the REQ socket until the REP socket is ready to go.
|
[13:06] jugg
|
s/not have/not/
|
[13:07] sustrik
|
jugg: that's done automatically
|
[13:07] sustrik
|
when there's no connection, messages are queued and sent after the server is available
|
[13:08] jugg
|
yes, this is the problem. My app is now hung on recv waiting for a reply that is sometime to come yet.
|
[13:08] sustrik
|
ok, what would you want to do instead?
|
[13:08] sustrik
|
check periodically?
|
[13:09] jugg
|
I'd like to detect if this is the case (no server available) or just have the send go to oblivion and the recv be a nop.
|
[13:09] sustrik
|
you can do a non-blocking send
|
[13:09] jugg
|
sure, but I still have to wait for the recv.
|
[13:09] sustrik
|
zmq_send (s, msg, ZMQ_NOBLOCK)
|
[13:09] sustrik
|
no, because the message is not send in that case
|
[13:10] jugg
|
ok, so without a server on the other side, it is send that is blocking?
|
[13:10] sustrik
|
it just says EAGAIN
|
[13:10] sustrik
|
yes
|
[13:10] jugg
|
well, I *do* want to block on recv if my message is actually sent.
|
[13:11] sustrik
|
yes
|
[13:11] sustrik
|
psudocode:
|
[13:11] sustrik
|
s = socket (REQ)
|
[13:11] sustrik
|
s.send (request, NOBLOCK)
|
[13:11] sustrik
|
if (EAGAIN)
|
[13:11] sustrik
|
do someting else
|
[13:11] sustrik
|
reply = s.recv ()
|
[13:13] jugg
|
ok, so NOBLOCK will cause send to fail if no server is available? May there be another reason that send would fail because NOBLOCK is set even if there was a server?
|
[13:14] jugg
|
ie. are there other side affects of NOBLOCK
|
[13:14] sustrik
|
no, this is the only reason
|
[13:14] sustrik
|
ah
|
[13:14] sustrik
|
ok, one more reason
|
[13:14] sustrik
|
if you set high watermarks on all sockets
|
[13:15] sustrik
|
it may happen (if server is slow to respond)
|
[13:15] sustrik
|
that all buffers space will be eqhausted
|
[13:15] sustrik
|
by pending requests
|
[13:15] jugg
|
ok, but without messing with ZMQ_HWM this is not an issue?
|
[13:15] sustrik
|
then NOBLOCK would return EAGAIN
|
[13:15] sustrik
|
no
|
[13:16] jugg
|
great, thanks.
|
[13:16] sustrik
|
you are welcome
|
[13:26] jugg
|
well, either another bug in the lua bindings, or NOBLOCK isn't working...
|
[13:26] jugg
|
send is returning success
|
[13:26] sustrik
|
REQ socket?
|
[13:26] jugg
|
yes
|
[13:26] sustrik
|
let me see
|
[13:27] jugg
|
the binding looks good
|
[13:27] sustrik
|
ah, right, you are connecting it
|
[13:27] jugg
|
?
|
[13:27] sustrik
|
that creates the buffer for messages right away
|
[13:28] jugg
|
oh
|
[13:28] sustrik
|
forget what i said before
|
[13:28] jugg
|
unfortunate, because your confusion is what I wanted... :/
|
[13:28] sustrik
|
you were right
|
[13:29] sustrik
|
you send a message and it'll be delivered eventually
|
[13:29] sustrik
|
what about checking for request in non-blocking manner and doing other work if there's no reply yet?
|
[13:29] sustrik
|
checking for reply
|
[13:32] jugg
|
that doesn't fit the existing structure that I'm plugging into :/ The current implementation requires that it is known whether the send was received by the server. But blocking on on the send until the server is available isn't acceptable. It needs to simply drop the message and keep going in that case.
|
[13:33] sustrik
|
then you need a roundtrip to server to ensure that, right?
|
[13:34] jugg
|
well, the round trip is to ensure the message was processed correctly when it actually gets processed. But it is ok to drop the message into oblivion if the server isn't available.
|
[13:34] sustrik
|
pub/sub sockets work that way
|
[13:34] sustrik
|
deliver is peer is available
|
[13:35] sustrik
|
drop message if it is not
|
[13:35] jugg
|
sure, but the peer doesn't reply back in that pattern.
|
[13:35] sustrik
|
it doesn not
|
[13:36] sustrik
|
basically, what you need is a sync connection rather than message queueing system
|
[13:36] sustrik
|
that's somehow out of scope of 0mq
|
[13:37] sustrik
|
why not use standard BSD sockets for such a trivial scenario?
|
[13:37] jugg
|
I think it is simply an issue of the REQ/REP pattern, there needs to be some way to check if a REQ can be made, if it can, then certainly block on recv for the reply. But if a request can't be made, don't sit there for eternity waiting to make the request.
|
[13:38] sustrik
|
it's a design issue
|
[13:38] jugg
|
this is only the first part of the message chain. Once it gets inserted here, it needs to be ensured from there out.
|
[13:38] sustrik
|
mq systems are expected to queue the message and let the user continue trusting the message will be delivered
|
[13:42] sustrik
|
it's similar to TCP in a way
|
[13:42] sustrik
|
you send data
|
[13:42] sustrik
|
and trust the underlying stack to deliver it eventually
|
[13:46] jugg
|
This is a frontend delivery to the messaging system. Once the message is in the system, then that message is ensured to be processed, but if the system isn't available there is no point in having the messages queued by the requesting application. That application needs to be told the system isn't available and try again later when it tries to send a message.
|
[13:47] sustrik
|
i would use simple TCP connection in such case
|
[13:48] sustrik
|
other option would be to timout
|
[13:48] sustrik
|
if no response arrives in N seconds the system is assumed to be unavailable
|
[13:48] sustrik
|
not sure whether that would work for you
|
[13:50] sustrik
|
actually, there were already some discussion about providing sync access to 0mq network
|
[13:50] sustrik
|
say, having a simple library that would speak 0mq wire protocol
|
[13:51] sustrik
|
but would handle connections same way as TCP does
|
[13:51] sustrik
|
but that kind of thing doesn't exist yet
|
[13:53] cremes
|
jugg: i think sustrik's suggestion of sending the message async and timing out on the reply is the way to go if you want to stay within the 0mq framework
|
[13:53] cremes
|
otherwise what you are trying to do isn't a good fit for 0mq
|
[13:53] sustrik
|
cremes: there's a point in what jugg says
|
[13:53] sustrik
|
the applications that handle exactly 1 connection
|
[13:53] sustrik
|
would want that kind of thing
|
[13:54] sustrik
|
say client/server application
|
[13:54] cremes
|
but don't you agree that it is already possible today without a change to the library?
|
[13:54] sustrik
|
server does obviously need 0mq framework as it is today
|
[13:55] sustrik
|
the connections are hidden to the user
|
[13:55] sustrik
|
which makes perfect sense if there's arbitrary number of them
|
[13:55] sustrik
|
like in server handling 10,000's of connections
|
[13:55] cremes
|
i agree
|
[13:55] sustrik
|
with the client the case is a bit different
|
[13:56] sustrik
|
you want to open exactly one connection
|
[13:56] sustrik
|
send/recv messages
|
[13:56] sustrik
|
and get notified if the connection breaks
|
[13:56] sustrik
|
technically, it's much easier than what 0mq does at the moment
|
[13:57] sustrik
|
but nobody implemented it yet
|
[13:58] cremes
|
by "nobody implemented it yet" do you mean no one has implemented the 0mq wire format outside of the 0mq framework?
|
[13:58] cremes
|
because that is one way to accomplish that goal
|
[13:59] sustrik
|
witthin or outside, doesn't matter
|
[13:59] sustrik
|
no sync interface as for now
|
[14:02] cremes
|
btw, what is your release plan for 2.0.7? 1 week? a month? the api has changed quite a bit in master but i am waiting to update my bindings until the next release.
|
[14:02] cremes
|
i don't want to chase more api changes.
|
[14:02] sustrik
|
it's not completely up to me
|
[14:02] sustrik
|
let me check with others involved
|
[14:02] cremes
|
oh, i thought you owned it :)
|
[14:03] sustrik
|
i have no idea how the build system works :)
|
[14:03] cremes
|
me neither!
|
[14:04] jugg
|
hmm, the watermark stuff doesn't even make sense in conjuction with REQ/REP anyway, as there can only ever be one message in the queue.
|
[14:05] sustrik
|
jugg: right
|
[14:05] jugg
|
So, in this case, I would think that NOBLOCK should work the way I want it.
|
[14:05] sustrik
|
on send?
|
[14:06] jugg
|
yes, the only point in the queue on send is if the server isn't available.
|
[14:07] sustrik
|
yes, the queue allows you to continue processing even though the peer is not available
|
[14:07] sustrik
|
it might seem an overkill in REQ case
|
[14:07] sustrik
|
but REQ is just a convenience wrapper over XREQ
|
[14:08] sustrik
|
which allows you to send arbitrary number of requests without waiting for answer
|
[14:08] sustrik
|
in that case the queue makes perfect sense
|
[14:08] jugg
|
I guess that one isn't documented...
|
[14:08] sustrik
|
it's not, sorry
|
[14:08] sustrik
|
it works in same way as REQ
|
[14:09] sustrik
|
but you don't have to recv after each send
|
[14:09] sustrik
|
the drawback is that you have to do some bookkeeping by hand then
|
[14:10] jugg
|
yah, it would seem the messages would have to have manual sequence numbers tagged on.
|
[14:10] jugg
|
maybe not in a 1to1 but in a load-balanced topology, certainly.
|
[14:10] sustrik
|
it works this way
|
[14:11] sustrik
|
imagine a complex network of requester and repliers with arbitrary structure of stand-alone queues in the middle
|
[14:11] sustrik
|
each request travels N hops till it gets to service
|
[14:11] sustrik
|
the reply has to be routed back the same way, opposite direction
|
[14:12] sustrik
|
so what happens is that each intermediate node tags the request with it's name
|
[14:12] sustrik
|
when request arrives at the service
|
[14:12] sustrik
|
it can has a stack of node names attached to it
|
[14:13] sustrik
|
what REP socket does
|
[14:13] sustrik
|
is that it cuts off the stack and provides raw message to the service
|
[14:13] sustrik
|
(it stores the stack somewhere in the meantime)
|
[14:14] sustrik
|
when service processes the request and sends the reply, REP socket takes the stored stack and attaches it to the reply
|
[14:14] sustrik
|
then each node routes the reply right way and chops of one name from the stack
|
[14:15] sustrik
|
when reply arrives at the original requester there are no more attached names in the message
|
[14:15] sustrik
|
makes sense?
|
[14:16] jugg
|
yes
|
[14:16] sustrik
|
so, when you use XREQ/XREP instead of REQ/REP
|
[14:16] sustrik
|
the bookkeeping you have to do is:
|
[14:17] sustrik
|
1. in XREP you have to chop and store the stack manually and attach it back the reply manually
|
[14:17] sustrik
|
2. in XREQ you have to attach 'bottom of the stack' message part to the request manually
|
[14:17] sustrik
|
and chop it off from the reply manually
|
[14:17] sustrik
|
'bottom of the stack' message part is just a message part zero bytes long
|
[14:18] sustrik
|
that's all about XREQ/XREP
|
[14:19] mato
|
sustrik: maybe it's time to document XREQ/XREP after all? but mark it prominently as "API is experimental" or something?
|
[14:19] sustrik
|
yes, i think so
|
[14:19] sustrik
|
people are using it
|
[14:20] sustrik
|
so no point in hiding the documentation
|
[14:20] mato
|
ok
|
[14:20] mato
|
do we have a code example that uses it? one of the devices?
|
[14:21] sustrik
|
zmq_queue
|
[14:22] mato
|
ok, good
|
[14:51] jugg
|
well, I still cast my vote that a REQ socket should fail if NOBLOCK is set and the REP socket isn't available. Failing that, I cast a vote for a UNAVAIL flag to be added which would then cause a failure if the REP socket is not available. (available meaning not bound/connected)
|
[14:59] sustrik
|
the problem is that it's a distributed system; say if you have a stand-alone queue in the middle, client would send the request (because queue is available) but it it'll linger in the queue because there are no services connected to it
|
[15:00] sustrik
|
:|
|
[15:01] sustrik
|
one of the design goals for 0mq is to allow for adding middle nodes into the network without changing semantics for the applications
|
[15:01] jugg
|
not at all, if the first hop is available, that is all that matters. The nodes in the middle shouldn't be using NOBLOCK, and it should wait until the message can be processed further.
|
[15:01] jugg
|
The "gateway" received it, therefore the finaly processing should be ensured.
|
[15:02] sustrik
|
can't you think of the queue in the client as a gateway as well?
|
[15:02] jugg
|
no
|
[15:03] sustrik
|
why so?
|
[15:03] sustrik
|
because of client application failing?
|
[15:05] jugg
|
An application knows how better to queue is data than a messaging handler. The application is purely interested in knowing whether it can connect to the "system" or not. If it can, it will send out messages. If it can't, it'll wait until later.
|
[15:10] sustrik
|
makes sense
|
[15:10] sustrik
|
so we have two conflicting user requirements here
|
[15:11] sustrik
|
1. "don't bother me with transport details, accept the message and leave me alone"
|
[15:11] sustrik
|
2. "let me know if the message cannot be sent immediately"
|
[15:13] jugg
|
in the general case, it isn't if it can't be sent immediately (queuing is fine, as long as the queue is being processed - ie something is consuming it). However in the REQ/REP case, where there is only a single message at a time, then yes, it simplifies down to that.
|
[15:13] jugg
|
Seems like the UNAVAIL flag satisfies both needs.
|
[15:13] sustrik
|
jugg: yes, REQ/REP is special in a way
|
[15:13] jugg
|
sustrik: thanks for the input and discussion.
|
[15:14] sustrik
|
I like to think of XREP/XREQ as of IP and REQ/REP as TCP
|
[15:14] sustrik
|
it's as there were two separate layers there
|
[15:14] sustrik
|
with distinct interfaces possibly
|
[15:14] sustrik
|
anyway
|
[15:14] sustrik
|
see you later
|
[15:14] jugg
|
bye
|
[16:20] jugg
|
Hmm, this is odd on a REP socket, I call send(msg, SNDMORE) 20 times, and one more time without SNDMORE. On the REQ socket, I call recv() and check getsockopt(RCVMORE) which only returns 10 times that there are more messages. The 10 successful recv() calls return every other sent message. So, every other message is being lost somewhere.
|
[16:28] jugg
|
meh, this is quite bothersome...
|
[16:29] zedas
|
cremes: thanks.
|
[16:29] zedas
|
sustrik: any luck?
|
[16:45] jugg
|
there certainly seems to be a bug here. I can combine my data set into a single message, and it comes accross just fine. But if I use SNDMORE, every other message is lost.
|
[16:47] cremes
|
jugg: fyi, you are using an experimental feature. if you can reproduce a bug, file a report
|
[17:31] jugg
|
issue filed: http://github.com/sustrik/zeromq2/issues/issue/28
|
[17:31] cremes
|
excellent
|
[20:48] bgranger
|
sustrik: I have a question about if/when zmq_free_fn is called if zmq_send fails...
|
[20:49] CIA-17
|
pyzmq: 03Brian Granger 07nocopy * r4685c6d 10/ (zmq/_zmq.c zmq/_zmq.pyx): Adding comments about when Py_DECREF is called. - http://bit.ly/aW0sUW
|
[20:53] sjampoo
|
bgranger, hi
|
[20:53] bgranger
|
hi
|
[20:54] sjampoo
|
i verified it, it does not call zmq_free_fn on error
|
[20:54] sjampoo
|
not sure if it should though
|
[20:54] bgranger
|
sjampoo: thanks, so I possibly have to clean up myself (I can't rely on zmq_free_fn for that)
|
[20:55] bgranger
|
That is a tough one, because it really depends on why the send fails.
|
[20:56] bgranger
|
As long as 0mq is consistent and never calls it on error, I can handle it.
|
[21:19] CIA-17
|
pyzmq: 03Brian Granger 07nocopy * rc0dce80 10/ (zmq/_zmq.c zmq/_zmq.pyx): Documentation an minor improvements to Message object. - http://bit.ly/cziApz
|
[21:22] mato
|
hi guys, does this work better for a description of the pipeline pattern?
|
[21:22] mato
|
The pipeline pattern is used for distributing data to _nodes_ arranged in
|
[21:22] mato
|
a pipeline. Data always flows *down* the pipeline, and each stage of the
|
[21:22] mato
|
pipeline is connected to at least one _node_. When a pipeline stage is
|
[21:22] mato
|
connected to multiple _nodes_, data shall be processed by all connected _nodes_
|
[21:22] mato
|
in parallel.
|
[21:29] cremes
|
mato: sounds better than what is there
|
[21:29] cremes
|
as usual, code is the best documentation
|
[21:29] mato
|
goodo
|
[22:49] CIA-17
|
pyzmq: 03Brian Granger 07master * r88a07bb 10/ (README.rst setup.cfg.template): Updating README.rst and setup.cfg.template. - http://bit.ly/9dVTj5
|
[22:59] CIA-17
|
zeromq2: 03Martin Lucina 07master * r5219e4c 10/ (doc/zmq.txt doc/zmq_setsockopt.txt doc/zmq_socket.txt): Clarify socket types in documentation, reinstate ZMQ_PAIR - http://bit.ly/9JerN2
|
[22:59] CIA-17
|
zeromq2: 03Mikko Koppanen 07master * r8bd3f74 10/ builds/redhat/zeromq.spec : Import redhat packaging - http://bit.ly/drWjGK
|
[22:59] CIA-17
|
zeromq2: 03Mikko Koppanen 07master * rb4cc7b9 10/ Makefile.am : dist-hook for copying zeromq.spec to top-level - http://bit.ly/bLxodo
|
[22:59] CIA-17
|
zeromq2: 03Martin Lucina 07master * r74f1a4a 10/ builds/redhat/zeromq.spec :
|
[22:59] CIA-17
|
zeromq2: RPM packaging cleanups
|
[22:59] CIA-17
|
zeromq2: - ditch -utils package
|
[22:59] CIA-17
|
zeromq2: - add descriptions from Debian packaging - http://bit.ly/aTC42b
|
[23:39] CIA-17
|
zeromq2: 03Martin Lucina 07master * rda37c45 10/ (doc/zmq_bind.txt doc/zmq_connect.txt):
|
[23:39] CIA-17
|
zeromq2: Clarify zmq_bind/zmq_connect
|
[23:39] CIA-17
|
zeromq2: Use the term 'endpoint' correctly, and drop the nonsense about local/remote addresses which doesn't clearly explain what is going on - http://bit.ly/d3A9wD
|