| [Time] Name | Message |
|
[00:01] Guest4929
|
Got a question: I am using a C# zmq binding. I have a thread that blocks on recv(). At some point I want to call abort() on the thread - catch the exception, and terminate. But Abort() seems to have no effect while thread is in recv(). Also, cant seem to find zmq_term() in C#? Any help is much appreciated!
|
|
[02:03] dcolish
|
i wish pgm did req/reply. I cant really think of another pattern where i can do broadcast style req/reply. does such a pattern exist?
|
|
[02:03] dcolish
|
i'm thinking server broadcasts, clients all reply
|
|
[02:03] dcolish
|
or is it simple enough that you just use a req/reply socket where the clients connect and the server reqs on *:5555?
|
|
[02:53] CIA-79
|
pyzmq: 03MinRK 07master * ra53f687 10/ zmq/core/socket.pyx : move self.close() to __del__ from __dealloc - http://git.io/CpT3VA
|
|
[02:55] CIA-79
|
pyzmq: 03MinRK 07master * ra41a9e9 10/ zmq/core/socket.pyx : move socket.close() to __del__ from __dealloc__ ...
|
|
[09:48] someone-noone
|
hello! If zmq_connect doesn't return real connection state (for tcp) how can it be resolved? I mean how can i know that connection successfully established?
|
|
[12:50] mikko
|
someone-noone: the connection happens async
|
|
[12:51] mikko
|
there is really no concept of connection state
|
|
[12:51] someone-noone
|
mikko: it's bad, i need to handle unsuccesful connection somehow
|
|
[12:51] mikko
|
what sort of pattern are you using?
|
|
[12:51] someone-noone
|
req rep
|
|
[12:52] mikko
|
you can poll the connection whether it becomes writable
|
|
[12:52] mikko
|
and assume failure if timeout exceeds
|
|
[12:52] someone-noone
|
thanks
|
|
[13:34] enleth
|
Hi there. Anyone knows which git commit for php-zmq actually builds with PHP 5.3.8? I'm getting undefined PHP_ZMQ_TIMEOUT and no wonder, it's commented out in php_zmq_private.h
|
|
[13:36] mikko
|
enleth: have you tried the current master?
|
|
[13:36] enleth
|
mikko: I think it is current master - but maybe I should just rm -rf the tree and clone it from scratch, just a moment
|
|
[13:37] enleth
|
It's the same 0a963..., configuring right now
|
|
[13:39] enleth
|
zmq.c:1068:54: error: âPHP_ZMQ_TIMEOUTâ undeclared
|
|
[13:39] enleth
|
And I just noticed it complains about undefined ZMQ_DEALER at zmq.c:1712:2
|
|
[13:40] mikko
|
what version of zeromq?
|
|
[13:40] mikko
|
i assume 2.0.x ?
|
|
[13:40] enleth
|
2.1.9, just installed
|
|
[13:40] mikko
|
that can't be right
|
|
[13:40] mikko
|
let me check
|
|
[13:40] enleth
|
[enleth@heroes] [15:40:25] [php-zmq]$ grep ZMQ_DEALER /usr/include/zmq*
|
|
[13:40] enleth
|
/usr/include/zmq.h:#define ZMQ_DEALER 5
|
|
[13:40] mikko
|
yes
|
|
[13:41] enleth
|
This looks very wrong
|
|
[13:41] mikko
|
have you got /usr/local/include/zmq.h ?
|
|
[13:41] mikko
|
can check what is reported prefix and version for zeromq
|
|
[13:41] enleth
|
No, nothing in local
|
|
[13:41] mikko
|
you should see those in configure output
|
|
[13:41] enleth
|
You mean the zeromq itself?
|
|
[13:41] mikko
|
no, php-zmq
|
|
[13:42] mikko
|
when you run configure it should report the version and i think prefix as well
|
|
[13:42] enleth
|
No, just PHP prefixes
|
|
[13:43] enleth
|
Those are PHP includes only
|
|
[13:43] mikko
|
checking libzmq installation... found version 2.1.10
|
|
[13:43] mikko
|
it should tell you something like that
|
|
[13:44] enleth
|
Uh. That's funny. checking libzmq installation... found version 4.0.0
|
|
[13:45] mikko
|
yes
|
|
[13:45] mikko
|
that explains
|
|
[13:45] enleth
|
I mean, WTF?
|
|
[13:45] mikko
|
do pkg-config libzmq --prefix
|
|
[13:45] mikko
|
did you install libzmq master?
|
|
[13:45] mikko
|
that is not 2.1 series
|
|
[13:45] mikko
|
https://github.com/zeromq/zeromq2-1
|
|
[13:46] enleth
|
That was most certainly built from the 2.1.9 tarball
|
|
[13:46] enleth
|
I've never even looked at the zmq git repo
|
|
[13:47] enleth
|
I'm building an RPM from a spec file I wrote, but it worked just fine for 2.1.4
|
|
[13:47] enleth
|
And there weren't any unpackaged files for that matter
|
|
[13:47] mikko
|
what does the spec file do?
|
|
[13:48] mikko
|
can you update your php-zmq and rerun phpize && ./configure
|
|
[13:48] mikko
|
it should now report the location where it finds libzmq
|
|
[13:48] mikko
|
my assumption is that your spec file clones the git repo (?)
|
|
[13:49] enleth
|
Nope, it's using the tarball
|
|
[13:49] enleth
|
I mean, the spec file for libzmq - php-zmq I'm building manually
|
|
[13:50] mikko
|
did you download the tarball?
|
|
[13:50] mikko
|
for some reason it looks like you have 4.0.0
|
|
[13:50] enleth
|
It's looking for libzmq in /usr/local
|
|
[13:50] mikko
|
let me check the tarballs
|
|
[13:50] mikko
|
maybe you need --with-zmq=/usr
|
|
[13:51] mikko
|
you seem to have 4.x installed under /usr/local
|
|
[13:51] enleth
|
My libzmq is definitely built from http://download.zeromq.org/zeromq-2.1.9.tar.gz
|
|
[13:51] enleth
|
And /usr/include/zmq.h does contain 2.1.9 version defines
|
|
[13:51] enleth
|
I'm trying the --with-zmq
|
|
[13:52] enleth
|
libzmq installation... found version 2.1.9, under /usr
|
|
[13:52] mikko
|
so you have another installation under /usr/local ?
|
|
[13:54] enleth
|
My goodness, the other admin probably did that. It is there, yes. Bah, can't leave the server for a week without anything getting screwed up. But the build still fails, I'll clean up this mess and try again.
|
|
[13:56] enleth
|
It appears that neither /usr/include nor /usr/local/include is passed to cc as -I
|
|
[13:57] tohava
|
I am using zeromq with erlang, I had an erlang process with zeromq terminate, when it restarted it failed to listen on the same socket due to EADDRINUSE
|
|
[13:57] tohava
|
Supposedly there is some way to tell it to reuse address in sockets api, how do I do it in zeromq?
|
|
[13:58] mikko
|
i think we already use SO_REUSEADDR
|
|
[13:58] mikko
|
but the socket might be in TIME_WAIT state (?)
|
|
[13:59] tohava
|
if the socket entered TIME_WAIT, reuse won't work?
|
|
[13:59] enleth
|
Ugh, it does build now. Thanks mikko, sorry for bothering.
|
|
[14:01] mikko
|
tohava: not 100% sure
|
|
[14:02] mikko
|
SO_REUSEADDR allows your server to bind to an address which is in a
|
|
[14:02] mikko
|
TIME_WAIT state.
|
|
[14:02] mikko
|
so yes, it should be able to bind
|
|
[14:02] enleth
|
By the way, what's the status of zmq3 support for PHP and Python?
|
|
[14:02] mikko
|
tohava: if that happens again can you check what state the socket is in?
|
|
[14:02] mikko
|
enleth: i did zmq3 support for php
|
|
[14:02] tohava
|
mikko, I will take a look
|
|
[14:02] mikko
|
the current master supports 2.1.x and 3.x
|
|
[14:03] enleth
|
Is it documented yet?
|
|
[14:03] mikko
|
it should be
|
|
[14:03] mikko
|
not sure if the doc generation script works
|
|
[14:04] mikko
|
i got it running somewhere
|
|
[14:04] mikko
|
the changelog should be accurate
|
|
[14:07] tohava
|
mikko, state is "LISTEN"
|
|
[14:07] tohava
|
mikko, I wonder if this could be a leak in the erlang binding for zeromq
|
|
[14:09] mikko
|
tohava: is the process gone?
|
|
[14:11] tohava
|
mikko, the erlang process is gone, however all erlang processes are ran through a single unix process (virtual machine)
|
|
[14:12] tohava
|
mikko, I think the problem is on our side, thanks anyway, if I find anything of interest on zeromq I'll let you know
|
|
[14:12] mikko
|
cool, thanks
|
|
[14:13] mikko
|
i've been using zeromq so long that bsd socket api is slowly getting erased from my head
|
|
[14:24] enleth
|
Looks like zmq is better than brain bleach
|
|
[15:23] taotetek
|
pieterh_: around?
|
|
[16:36] bitcycle
|
Hey all. Any ideas as to why I'm getting a segfault using pyzmq? http://paste.pound-python.org/show/12842/
|
|
[16:39] Steve-o
|
can you trace in gdb?
|
|
[16:40] bitcycle
|
I'm using the python bindings. The segfault doesn't occur when the pub & sub are in the same process.
|
|
[16:42] Steve-o
|
but you still should be able to catch the dump, gdb python script.py ...
|
|
[16:44] bitcycle
|
test.py is not a core dump: File format not recognized
|
|
[16:59] Steve-o
|
you probably want "gdb `which python`" then "run script.py", difficult to remember off the top of my head
|
|
[17:06] mikko
|
gdb --args python script.py
|
|
[17:07] mikko
|
and hes gone
|
|
[17:08] Steve-o
|
doh, missed by 9 minutes too, :-)
|
|
[17:19] cremes
|
hi, i have a question but i *need* it answered within the next 30 seconds or else i'm leaving
|
|
[17:19] cremes
|
hurry up
|
|
[17:19] cremes
|
anyone?
|
|
[17:20] cremes
|
really? no one can help?
|
|
[17:20] mikko
|
i've seen a few reports of segfaults with pyzmq lately
|
|
[17:20] mikko
|
more or less informal
|
|
[17:20] mikko
|
i wonder if they have a bug
|
|
[17:49] CIA-79
|
pyzmq: 03MinRK 07master * re4c3be6 10/ zmq/core/context.pyx : move ctx.term() to __del__ from __dealloc__ ...
|
|
[17:52] dcolish
|
I'm running into a "resource not available" issue with pub/sub sockets and i'm not too sure how to fix it. I've never had this issue before. Here's a paste of the two simultaneous sessions http://paste.pocoo.org/show/480048/
|
|
[17:54] Steve-o
|
out of fd's? love vague libc errors
|
|
[17:55] dcolish
|
man i hope not, i guess i could be
|
|
[17:55] dcolish
|
i'm not running anything else really, should be fine
|
|
[17:57] Steve-o
|
can you strace / truss it and capture the error code?
|
|
[17:58] dcolish
|
lsof says i'm only using 4094 fds
|
|
[17:58] dcolish
|
i can try stracing it
|
|
[18:00] dcolish
|
oh thats right, osx doesnt have strace :/
|
|
[18:00] Steve-o
|
awesome
|
|
[18:01] Steve-o
|
dtruss
|
|
[18:01] Steve-o
|
they've ported Solaris' truss it seems
|
|
[18:01] Steve-o
|
>> http://humberto.digi.com.br/blog/2008/02/25/strace-on-mac-os-x-leopard/
|
|
[18:02] dcolish
|
ah nice
|
|
[18:02] dcolish
|
http://paste.pocoo.org/show/480053/
|
|
[18:02] dcolish
|
didnt know about that one
|
|
[18:18] Steve-o
|
so that's the connect failure?
|
|
[18:18] Steve-o
|
is that edeadlock I don't have a reliable list to hand
|
|
[18:19] dcolish
|
yeah, its EINPROGRESS
|
|
[18:22] Steve-o
|
that appears to be handled correctly in the code, converted to EAGAIN
|
|
[18:23] dcolish
|
right, i guess i'm not sure why its not connecting at all
|
|
[18:23] dcolish
|
its over localhost so i'd expect it to connect
|
|
[18:23] Steve-o
|
the code means it is connecting though
|
|
[18:24] dcolish
|
hmm, ok but even when i poll i'm not receiving messages
|
|
[18:24] Steve-o
|
but it hasn't finished connecting, so a bit strange
|
|
[18:24] Steve-o
|
it should be a very quick transient state
|
|
[18:25] dcolish
|
right
|
|
[18:27] Steve-o
|
but it is caught somewhere, is it pyzmq?
|
|
[18:27] dcolish
|
nowhere i can tell, maybe there is something internal to pyzmq
|
|
[18:27] dcolish
|
?
|
|
[18:27] Steve-o
|
are you on the latest from upstream?
|
|
[18:27] dcolish
|
nope, 2.1.7
|
|
[18:28] dcolish
|
this has worked for me in the past
|
|
[18:28] Steve-o
|
it might have been fixed already :/
|
|
[18:28] dcolish
|
i'll look
|
|
[18:29] Steve-o
|
but maybe not: https://github.com/zeromq/pyzmq/blob/master/zmq/core/socket.pyx#L531
|
|
[18:30] Steve-o
|
it does not handle -1/EAGAIN on connect
|
|
[18:30] Steve-o
|
file a bug in github and ping the list
|
|
[18:30] Steve-o
|
s/ping/prod/ uggh, I shouldn't use that word
|
|
[18:30] dcolish
|
no ping?
|
|
[18:31] Steve-o
|
prod the list, cf. ping the list
|
|
[18:33] dcolish
|
poke the list?
|
|
[18:33] dcolish
|
*sigh*
|
|
[18:35] Steve-o
|
needs to be "if rc != 0 or zmq_errno() != EAGAIN:"
|
|
[18:36] Steve-o
|
should be easy to test
|
|
[18:50] dcolish
|
hmm so updated to pyzmq HEAD and I get "No such file or directory" when calling connect
|
|
[18:51] dcolish
|
did the zmq api change much between 2.1.7 and 2.1.9, or at least enough to break pyzmq?
|
|
[18:51] pieterh
|
dcolish: what's the connect string?
|
|
[18:51] dcolish
|
"tcp://127.0.0.1:7004"
|
|
[18:51] dcolish
|
i've also tried "tcp://localhost:7004"
|
|
[18:52] pieterh
|
so there was a change that pre-validated the connect string
|
|
[18:52] dcolish
|
i'm binding "tcp://*:7004" on the same host
|
|
[18:52] pieterh
|
allowing zmq_connect to return an error immediately
|
|
[18:52] pieterh
|
does "ping 127.0.0.1" work?
|
|
[18:52] dcolish
|
yes
|
|
[18:52] dcolish
|
so does "ping localhost"
|
|
[18:52] pieterh
|
can you try a simple C test?
|
|
[18:52] dcolish
|
sure
|
|
[18:54] dcolish
|
just to reiterate, PUSH/PULL sockets seemed to work on the same host, its just PUB/SUB which is giving me trouble
|
|
[18:59] pieterh
|
dcolish: that's weird
|
|
[18:59] dcolish
|
aye, i'm stumped
|
|
[19:00] pieterh
|
well, usual problem solving technique applies
|
|
[19:00] pieterh
|
1. disregard the cases where it works, we don't care about those
|
|
[19:00] pieterh
|
2. reproduce the case where it fails using simplest possible code
|
|
[19:00] Steve-o
|
probably easiest to test plain C version, but that return value does look suspect
|
|
[19:00] pieterh
|
3. file issue with test case
|
|
[19:01] pieterh
|
if it's returning EINVAL that's the pre-validation complaining about the connect hostname
|
|
[19:01] Steve-o
|
but "resource not available" is EAGAIN
|
|
[19:02] pieterh
|
Steve-o: hmm
|
|
[19:03] Steve-o
|
unless that is a red herring
|
|
[19:03] dcolish
|
i think it might be
|
|
[19:04] dcolish
|
the EAGAIN behavior looks as i'd expect
|
|
[19:04] jond
|
dcolish: can you strace the binary?
|
|
[19:04] Steve-o
|
output listed above
|
|
[19:05] Steve-o
|
>> http://paste.pocoo.org/show/480053/
|
|
[19:05] Steve-o
|
it's truss or rather dtruss on OSX
|
|
[19:06] Steve-o
|
which as Pieter suggests might be all bogus as the error could come from 0mq's validation routine
|
|
[19:07] dcolish
|
i gotta run out and grab some food, i'll run the C test cases when i get back
|
|
[19:07] pieterh
|
ack
|
|
[19:11] jond
|
in the link posted here the ports are different http://paste.pocoo.org/show/480048/
|
|
[20:01] dcolish
|
jond: yeah that was a mistake, the output is the same when they're the same
|
|
[20:02] jond
|
dcolish: i was just checking.... ;-)
|
|
[20:55] dcolish
|
well here's a C test case for my issue http://paste.pocoo.org/show/480150/. Doesnt seem to be receiving messages at all, but I am not getting errors while calling connect
|
|
[20:57] jond
|
dcolish: you need to subscribe to a pattern
|
|
[20:59] dcolish
|
jond: shoot, you're right. i dont know why I thought it would always subscribe to all by default
|
|
[20:59] dcolish
|
thanks
|
|
[21:00] dcolish
|
although, still doesnt explain those errors in pyzmq
|
|
[21:01] jamescarr
|
how come it's not possible to have a pure java client?
|
|
[21:02] jond
|
dcolish: no prob, get the c example working, then contact the pyzmq people if problem still there.
|
|
[21:06] jond
|
dcolish: actually in the pyzmq does sock.recv(1) mean non-blocking recv? you hadnt subscribed there either in which case EAGAIN might be ok
|
|
[21:06] dcolish
|
no i would expect it to time out, but i'm not sure its reporting the right errno
|
|
[21:10] jond
|
dcolish: pyzmq src doc "If NOBLOCK is set, this method will raise a ZMQError with EAGAIN if a message is not ready"
|
|
[21:11] dcolish
|
right, but when its not set what should happen?
|
|
[21:11] dcolish
|
normally i just use a poller but i wasnt seeing the messages so i ended up trying recv with a timeout
|
|
[21:11] dcolish
|
thats how i got on this path
|
|
[21:12] dcolish
|
turned out that all i needed was a subscribe pattern
|
|
[21:12] GregBowyer
|
I have a zeromq application that is currently in two parts, the top of the application is called "capture-head" and is a program that listens to and reassembles packets from net_ring or libpcap off the network wire. Once he has reassembled a packet (typically http) the http packet is put out onto zero-mq for worker-processes to do with as they will.
|
|
[21:12] GregBowyer
|
currently these worker processes take the (in this case) http request, and replay it against a different server
|
|
[21:13] GregBowyer
|
the upshot of this is I can consume our production traffic, and use that to 24x7 load test new software waiting release in staging
|
|
[21:13] GregBowyer
|
this is done with PUSH/PULL sockets, there is one capture head and many (identical) workers
|
|
[21:14] GregBowyer
|
now I have been reading the documentation, is there a way I could have two sets of workers see these messages and act on the independently
|
|
[21:15] GregBowyer
|
effectivly I would like to copy the request as sent on the PUSH socket and have worker group 1 consume it *and* worker group 2 consume it
|
|
[21:15] Steve-o
|
GregBowyer: so zeromq as a network thread pool, you want pub functionality?
|
|
[21:16] GregBowyer
|
Steve-o not really I guess I should draw a picture as its a little hard to explain (or I am not getting your answer)
|
|
[21:16] Steve-o
|
the obvious seems a broker in each group that has SUB and sends out on PUSH
|
|
[21:18] GregBowyer
|
so the workers connect to a PULL socket in the broker, which connects to the PUB in the capture head
|
|
[21:19] Steve-o
|
yup
|
|
[21:20] GregBowyer
|
.... yeah that could work
|
|
[21:20] Steve-o
|
depends on reliability requirements if you need a more advanced pattern
|
|
[21:21] GregBowyer
|
its quite nice the reliability requirements for this software stack are to just drop things if anything goes wrong
|
|
[21:23] jond
|
dcolish: I don't think that calling pyzmq.recv(1) is passing a timeout, it's a flag meaning don't block. glad yr sorted anyhow.
|
|
[21:24] dcolish
|
crap, you're right
|
|
[21:24] dcolish
|
i probably got mixed up with poll(timeout)
|
|
[21:25] GregBowyer
|
Steve-o, that does mean I end up with an extra process in each worker group, but I guess thats ok since I can abuse the broker to scale up and down a number of subordinates
|
|
[21:26] GregBowyer
|
Steve-o, .... which makes me ask is there a way I can see the size of the message queue for PUSH/PULL sockets
|
|
[21:26] Steve-o
|
GregBowyer: actually I was wondering if you could multi-push
|
|
[21:26] Steve-o
|
like this: http://lists.zeromq.org/pipermail/zeromq-dev/2011-January/008956.html
|
|
[21:29] GregBowyer
|
similer but I dont have to worry about lost messages, in fact I could open multiple PULL sockets and have different worker groups read from different PULL sockets, I do need the load balancing within a worker group
|
|
[21:34] Steve-o
|
two sockets is the cleaner method I guess
|
|
[21:34] Steve-o
|
send twice
|
|
[21:34] GregBowyer
|
ok I will try that
|
|
[21:43] Steve-o
|
or something elaborate like pub-sub ipc to two devices that forward on push, so only one send and scalable
|
|
[21:46] Steve-o
|
you can then dynamically increase or decrease the groups
|
|
[21:46] Steve-o
|
pretty nifty
|
|
[21:50] Steve-o
|
GregBowyer: I think many users prefer using devices to manually hacking distribution with multiple sockets and sends
|
|
[21:53] jond
|
steve-o: yep it's fowarder and streamer you are talking about, but worth noting these are gone in 3.0 onwards from the core library
|
|
[21:53] GregBowyer
|
that seems like a nice approach, the only thing that put me off was the concept of having yet another process on the box fwiw zero-mq works really well, but the box has some quite funny networking requirements (e.g. I have to be careful what I send since I listen to the network at the same time) and its definatly using the CPU in the box (the entire thing cruses at GB's of vmem and fairly constant CPU 100% burn
|
|
[21:54] Steve-o
|
device you can spawn off in another thread, makes it easy
|
|
[21:54] GregBowyer
|
my PUSH/PULL is over unix sockets at the moment, that basically avoids the networking stack and means I dont end up hearing things I send (which was really fun last time I had that happen :D)
|
|
[21:55] Steve-o
|
I think we still don't have ITC sockets (?)
|
|
[21:56] GregBowyer
|
you must have ipc sockets I am using them !
|
|
[21:56] GregBowyer
|
no wait
|
|
[21:56] Steve-o
|
between threads, not processes
|
|
[21:56] GregBowyer
|
sorry my bad parse error
|
|
[21:56] Steve-o
|
so disruptor basically
|
|
[21:56] GregBowyer
|
I read that as IPC
|
|
[21:56] Steve-o
|
stick it on my todo list
|
|
[21:57] GregBowyer
|
you meaning the java distruptor LMAX ?
|
|
[21:57] Steve-o
|
yup
|
|
[21:57] Steve-o
|
really needs proactor pattern which is my, I guess, present outstanding task
|
|
[21:58] Steve-o
|
trying to improve Win32/64 performance by targeting IOCP
|
|
[21:58] GregBowyer
|
I was under the impression that was what zero-mq used under the hood no ?\
|
|
[21:58] Steve-o
|
a big long list of dependencies
|
|
[21:58] Steve-o
|
Win32 is all select() reactor based
|
|
[21:59] Steve-o
|
and IPC is TCP socket based
|
|
[21:59] Steve-o
|
not so great unfortunately
|
|
[21:59] Steve-o
|
but still pretty good numbers
|
|
[21:59] GregBowyer
|
Steve-o, are unix versions picking between kqueue/ epoll and friends
|
|
[21:59] Steve-o
|
yes
|
|
[22:00] Steve-o
|
you have an option at configure time to override
|
|
[22:00] GregBowyer
|
I care less about windows as I could never get the packet capture rate on windows that I get on linux / freebsd, pcap on windows is way to slow, and the native option is no fun
|
|
[22:01] Steve-o
|
which is why we are in the state today, Windows is second citizen
|
|
[22:01] GregBowyer
|
(not that I am saying that windows is not important for zero-mq its just never going to be usedful for this app)
|
|
[22:01] Steve-o
|
I've spent the last few years re-architecting and developing OpenPGM for Windows, headache at times
|
|
[22:01] GregBowyer
|
its interesting on windows you do IPC with TCP, I would have thought namedpipes would have been better (my windows API knowledge is super rusty so I might be way off here)
|
|
[22:02] Steve-o
|
you cannot poll a named pipe
|
|
[22:02] Steve-o
|
so you need the proactor model
|
|
[22:02] Steve-o
|
which I actually have on a local branch
|
|
[22:03] Steve-o
|
and IOCP on another, they all need to integrate a bit cleaner
|
|
[22:03] Steve-o
|
last project was IPv6 though so distracted *a lot*
|
|
[22:03] GregBowyer
|
ah check :D and hence we are reduced to your earlier statements
|
|
[22:03] jhawk28
|
whats a proactor?
|
|
[22:03] Steve-o
|
completion ports
|
|
[22:04] Steve-o
|
basically, notification when something has finished,
|
|
[22:04] GregBowyer
|
out of sheer curiosity what would be your take on sctp and similer protocols
|
|
[22:04] Steve-o
|
because you start the job before you can run it, you can queue up several receives for better performance, and usually zero-copy
|
|
[22:04] Steve-o
|
sctp is in 0mq 1.0
|
|
[22:05] GregBowyer
|
jhawk28, there is a fairly nice (albiet slighty dated and biased) description of it here http://www.cs.wustl.edu/~schmidt/patterns-ace.html
|
|
[22:05] Steve-o
|
I think Martin Sustrik or someone commented on how it looks well on paper but doesn't implement well in practice
|
|
[22:06] Steve-o
|
the framing you see in SCTP is effectively improved with something like SPDY by Google too, to a certain extent
|
|
[22:06] GregBowyer
|
makes sense
|
|
[22:07] GregBowyer
|
just curious because I saw the wiki page but didnt see anything about it anywhere else
|
|
[22:08] Steve-o
|
new protocols have a large mountain to climb to get acceptance anywhere
|
|
[22:09] dcolish
|
wrt my no resource found errors earlier, i think i may have actually run out of fds
|
|
[22:09] dcolish
|
*to
|
|
[22:09] GregBowyer
|
Steve-o, heh well I said zero-mq for this application and that was a big enuff battle, so a brand new layer 4 protocol ......
|
|
[22:09] Steve-o
|
which is why 0mq has UDP/PGM in addition to IP/PGM
|
|
[22:10] Steve-o
|
gigabit wire speed Cisco routers are expensive for IP/PGM
|
|
[22:10] GregBowyer
|
Steve-o, ? do you offer similer atomic guarantees on UDP/PGM ?
|
|
[22:11] Steve-o
|
actually UDP tends to run a bit faster too
|
|
[22:11] GregBowyer
|
(again curious more than intending to use this at the moment, adding endpoints is pretty involved for me)
|
|
[22:11] Steve-o
|
PGM is available for PUB/SUB only
|
|
[22:12] Steve-o
|
makes things easier
|
|
[22:12] Steve-o
|
you could stretch it like TIBCO RV for PUSH/PULL and REQ/REP but it can get overly complicated and slow
|
|
[22:13] Steve-o
|
unfortunately 0mq has no bus socket either, but no so much need so far
|
|
[22:13] GregBowyer
|
I think I would end up using it to spin out multiple brokers on seperate boxes and use PUSH/PULL on those boxes rather than try to warp it
|
|
[22:14] Steve-o
|
it is nice to have the options
|
|
[22:15] GregBowyer
|
so far its been seriously neat stuff I am digging zero-mq for how fast its working although I will have to see if it puts up with the pull production capture one day rather than just a slice of the traffic :D
|
|
[22:16] Steve-o
|
it has a good record so far, so give it a spin
|
|
[22:17] GregBowyer
|
for now my biggest bottle neck is in libpcap, packet reassembly and damnable memory leaks occuring with broken tcp conversations that dont quite reassmble nicly (the same ones that would appear to be FIN_CLOSE forever)
|
|
[22:17] GregBowyer
|
but thats not zero-mq's fault thats my own shoddy tcp state machine
|
|
[22:17] Steve-o
|
you need hardware acceleration?
|
|
[22:17] Steve-o
|
oodles of cards available
|
|
[22:17] GregBowyer
|
what it did was take out all of the bzyantine queuing instantly, I think I dropped about 3000 lines of code outright
|
|
[22:18] GregBowyer
|
Steve-o, has to be commodity hardware only unfortunatly, but I can still get a damn good rate, just screwed up latency between requests
|
|
[22:19] GregBowyer
|
Steve-o, I am more than tempted to get an FPGA and try this on it mind http://www.netfpga.org/
|
|
[22:19] Steve-o
|
does libpcap work with user-space mellanox and solarflare drivers?
|
|
[22:20] GregBowyer
|
no idea
|
|
[22:20] Steve-o
|
otherwise endace, corvil, tsassociates all have capture hardware
|
|
[22:21] GregBowyer
|
the next version may be using PF_RING from here http://www.ntop.org/products/pf_ring/ it seems to work out pretty nice in testing
|
|
[22:22] Steve-o
|
I didn't fancy the hassle of writing to that
|
|
[22:22] Steve-o
|
libpcap can be hardware accelerated transparently which is why it is nice
|
|
[22:22] Steve-o
|
I think openonload with solarflare would be worth a check before the effort required for pf_ring
|
|
[22:23] GregBowyer
|
I had not seen openonload before thanks !
|
|
[22:25] GregBowyer
|
anyhow thanks for the pointers towards PUB/SUB and brokers, I think I will go with that long term
|
|
[22:30] Steve-o
|
mellanox have a similar technology now
|
|
[22:30] Steve-o
|
I think it was 2-3 months ago, mentioned on low-latency.com
|
|
[22:30] Steve-o
|
pass on the name
|
|
[22:32] Steve-o
|
it might be VMA: http://www.mellanox.com/content/pages.php?pg=products_dyn&product_family=106&menu_section=69#tab-one
|
|
[22:32] Steve-o
|
yup, that's right: http://www.mellanox.com/related-docs/prod_acceleration_software/VMA.pdf
|
|
[22:34] jond
|
pieterh_: are you attending this PubSub thing in london....
|
|
[22:34] pieterh
|
jond: nope, I'm in the States right now
|
|
[22:35] pieterh
|
Steve-o: do you think VMA would work well with 0MQ?
|
|
[22:35] jond
|
pieterh_: I'm not going either, but i've emailed sustrik/ian in case they are around later.
|
|
[22:35] GregBowyer
|
Steve-o, that looks quite nice, albeit it seems to be a faster network card right rather than a dedicated capture card
|
|
[22:35] Steve-o
|
should be pretty good, on par with IP-over-IB
|
|
[22:37] Steve-o
|
GregBowyer> libpcap isn't my forte but this would offer performance improvement without code change and no overly custom hardware
|
|
[22:38] Steve-o
|
otherwise ask libpcap or ts-associates for their (recommended) capture hardware
|
|
[22:40] Steve-o
|
I know the ts-associates dev team are written their own capture drivers so, not very flexible from resourcing point of view
|
|
[22:41] GregBowyer
|
I will put it on the list to investigate, for now I get big improvements in small changes so I am not too unhappy at the moment
|
|
[22:41] GregBowyer
|
and the current use of this software it more than meets its requirements, my one itty bitty box turns over a full production workload that is serviced by hundreads of machines in a search engine cluster
|
|
[22:42] GregBowyer
|
but yeah, more people I work with ask to play with this stuff so the requirements and performance is going to become harder, so I might just say fuckit and look at custom hardware
|
|
[22:42] Steve-o
|
the libpcap hardware is popular with analysing mdds networks, so a bit niche
|
|
[22:43] GregBowyer
|
the biggest lift recently was going to linux 3.0 and having native compiled BPF filters
|
|
[22:43] Steve-o
|
linux 3.0 introduces sendmmsg() which is quite interesting
|
|
[22:44] Steve-o
|
IBM posted speed improvements with that and recvmmsg() but I have difficulty reproducing on IBM hardware
|
|
[22:44] Steve-o
|
that's pretty much UDP only concerns though, you can skip all that with pcap
|
|
[22:46] GregBowyer
|
BPF jit here https://github.com/mirrors/linux/commit/0a14842f5a3c0e88a1e59fac5c3025db39721f74
|
|
[22:46] GregBowyer
|
I was just about to mention sendmmsg
|
|
[22:47] GregBowyer
|
I think I can bench comfortably at 1Gbe 10Gbe is a challange, but I dont see that much traffic yet
|
|
[22:48] GregBowyer
|
just thought you might like to know that zero-mq IPC is not a barrier in there :D
|
|
[22:48] Steve-o
|
:-)
|
|
[22:52] GregBowyer
|
anyhow thanks again for the interesting conversation, I have to leave it at that and run off to catch my train
|
|
[22:53] Steve-o
|
no worries, laters
|
