[Time] Name | Message |
[18:30] pieterh
|
PerfDave, sorry, i was kidding around
|
[18:31] pieterh
|
sd88g93, ok, test case is now clear, lemme think about it a sec
|
[19:45] pieterh
|
sd88g93, what is 'qskt' for?
|
[19:45] sd88g93
|
that's the rep socket, in the worker thread, that recieves the messages
|
[19:45] sd88g93
|
from the client
|
[19:45] pieterh
|
there is already a terrible lack of vowels here in Slovakia
|
[19:46] sd88g93
|
tough times, gotta conservee
|
[19:46] pieterh
|
that's kind of like "who is that guy?" answer: "two legs, body, arms, head"
|
[19:47] pieterh
|
cause it does not in fact talk to the client at all
|
[19:47] sd88g93
|
couldnt come up with a more descriptive variable, sorry, i blame my lack of imagination
|
[19:47] sd88g93
|
it doesnt ? it does on mine
|
[19:47] pieterh
|
it talks to the main thread
|
[19:47] pieterh
|
via inproc
|
[19:47] sd88g93
|
yeah, then there's a queue device to the main socket
|
[19:47] sd88g93
|
the client skt
|
[19:48] pieterh
|
why is the client not talking to the worker directly?
|
[19:48] sd88g93
|
if you have more than one thread
|
[19:48] sd88g93
|
use the device , at least that's the scenario
|
[19:48] pieterh
|
well...
|
[19:49] pieterh
|
it
|
[19:49] pieterh
|
the device is really when you have an N-to-N case
|
[19:49] pieterh
|
and don't want all the N's to have to know about each other
|
[19:49] sd88g93
|
yeah, there's just one client now, but there could be more
|
[19:49] pieterh
|
ah... you are in fact developing the MT server model...
|
[19:49] sd88g93
|
yeah
|
[19:49] pieterh
|
i hardly recognized it... :-/
|
[19:49] sd88g93
|
there could be more than one client
|
[19:50] sd88g93
|
well, a diffferent form of it, i guess
|
[19:53] sd88g93
|
zhelpers.h:25 i thought sprintf was "evil", suppose to use snprintf ?
|
[19:54] keffo
|
pieterh, I've done the lru now!
|
[19:54] pieterh
|
keffo: wow!
|
[19:54] pieterh
|
sd88g93, this is what the worker loop becomes:
|
[19:54] pieterh
|
char *command = s_recv (control);
|
[19:54] pieterh
|
char *count = s_recv (control);
|
[19:54] pieterh
|
puts ("Worker got command: %s\n", command);
|
[19:54] pieterh
|
s_send (publisher, "msg.x|");
|
[19:54] pieterh
|
s_sendmore (publisher, command);
|
[19:54] pieterh
|
s_send (publisher, count);
|
[19:54] pieterh
|
free (command);
|
[19:54] pieterh
|
free (count);
|
[19:54] pieterh
|
s_send (control, "OK");
|
[19:54] pieterh
|
there is a bug, i see immediately...
|
[19:55] pieterh
|
in my code, not yours
|
[19:55] ModusPwnens
|
Wait a sec pieter, i was referring to something on the site
|
[19:55] sd88g93
|
oh, in s_send and s_sendmore you do init the message and close it each time,
|
[19:56] sd88g93
|
yeah, that's what i thought, you were just referrring to coding style, eh ? not actuall instructions needed
|
[19:56] sd88g93
|
i get ya now, ok
|
[19:56] pieterh
|
ModusPwnens, even better, join the site and click 'edit'
|
[19:56] pieterh
|
it's a wiki :-)
|
[19:57] ModusPwnens
|
Oh okay, well that's easy to fix then
|
[19:57] keffo
|
pieterh, it seems to be working very well.. I just need to post back the results now, but I'll probably use a new socketlayer for that, with some persistance as well
|
[19:57] pieterh
|
hmm, nice
|
[19:57] pieterh
|
keffo: i still find working with envelopes to be too painful
|
[19:57] pieterh
|
but we'll make abstractions for those later
|
[19:58] keffo
|
I have a Route class for that
|
[19:59] pieterh
|
yeah, right, you told me... nice
|
[19:59] keffo
|
which can construct itself from incoming, and also send it
|
[19:59] keffo
|
yeah
|
[19:59] pieterh
|
were the names in the routing section too weird?
|
[19:59] pieterh
|
i was trying to get mnemonics that worked
|
[19:59] keffo
|
where?
|
[19:59] pieterh
|
ch3
|
[19:59] keffo
|
mama,papa? =)
|
[19:59] pieterh
|
yeah
|
[19:59] jhawk28
|
the mama and papa were hilarious
|
[19:59] keffo
|
papa was a bit confusing, didnt really see the application to be honest, but I was mostly interested in the mama anyway :)
|
[20:00] pieterh
|
well, good, it's hard to make this material even remotely funny
|
[20:00] jhawk28
|
we enjoyed it at work
|
[20:00] psino
|
pieterh: you should move some more reasoning about the naming in ch3 to the place where you introduce them
|
[20:00] keffo
|
pieterh, Another interesting sideffect of the mama topology you presented there is that there is no longer any immediate necessity for a heartbeat system
|
[20:00] pieterh
|
psino: did I get the order wrong?
|
[20:01] pieterh
|
yup, but you need heartbeats for another reason
|
[20:01] pieterh
|
i was going to write that today but got distracted by our friend sd88g93's curious case here
|
[20:01] pieterh
|
also, ISP guys getting lost
|
[20:01] keffo
|
sure, but not in the same way as a topdown design
|
[20:02] psino
|
no, but when you say that REQ is a MAMA socket, making insistent demands, you might want to add something more that helps the reader remember the terminology
|
[20:02] pieterh
|
hmm, lemme think about that for a second...
|
[20:02] pieterh
|
i wrote that text very late on tuesday evening after a glass or two of wine... :-/
|
[20:03] psino
|
you describe the usage of mama pretty good in the "Router-to-Mama"-section
|
[20:03] psino
|
(personal opinion of course)
|
[20:03] pieterh
|
i think the character is based on my friend Fabio' Italian mother
|
[20:03] pieterh
|
ok...
|
[20:03] pieterh
|
opinion is better than silence
|
[20:04] jhawk28
|
I was going through the swap and hwm parts of the user guide and I got an exception after the pub ran for a while (from yquery:108)
|
[20:05] pieterh
|
jhawk28, interesting!
|
[20:05] jhawk28
|
It died when I was using the connect, but not the bind
|
[20:06] jhawk28
|
not sure if it matters, but I was using the Java bindings
|
[20:06] pieterh
|
do you mean yqueue?
|
[20:06] jhawk28
|
yes, it was an assertion that failed
|
[20:06] jhawk28
|
going from memory - so it could be wrong
|
[20:06] pieterh
|
could you please log this on the issue tracker, along with the hardware setup you're using?
|
[20:07] jhawk28
|
I think so
|
[20:07] pieterh
|
this should absolutely never never happen
|
[20:07] jhawk28
|
figured as much
|
[20:07] pieterh
|
there is indeed an assertion on yqueue.hpp:108
|
[20:07] pieterh
|
woa, hang on
|
[20:07] pieterh
|
it's an out of memory condition
|
[20:08] jhawk28
|
hwm was set to 1 and swap was set to 25000000
|
[20:09] pieterh
|
you sure you didn't reverse those? :-)
|
[20:09] pieterh
|
it's worth logging, we'll run that on some big boxes here and see if we can reproduce it
|
[20:09] jhawk28
|
:) was following the guide and then started playing around seeing what it would do
|
[20:09] pieterh
|
thanks
|
[20:15] daggertip
|
so
|
[20:23] pieterh
|
sd88g93, you still there? I award you the "obfuscated 0MQ contest" award :-)
|
[20:24] pieterh
|
you have six sockets and two devices in your minimal test case
|
[20:46] sd88g93
|
i'm always up for an award
|
[20:47] sd88g93
|
think sustrik can have a looksie ?
|
[20:54] pieterh
|
sd88g93, lol, are you trolling me?
|
[20:54] sd88g93
|
trolling ?
|
[20:54] pieterh
|
i've stripped it right down and removing the queue device lets it work
|
[20:54] pieterh
|
:-)
|
[20:54] pieterh
|
gnash, i will find this... gnash
|
[20:55] sd88g93
|
how do you read the messages into the thread ?
|
[20:55] pieterh
|
well, connect worker directly to client
|
[20:55] pieterh
|
next time, develop such code step by step so you can test it gradually
|
[20:56] pieterh
|
and when you add a new piece if it does not work you know exactly what is questionable
|
[20:56] sd88g93
|
i'm suspicious that the messages are being altered in some way, like martin is slipping coded messages in there
|
[20:56] sd88g93
|
yeah, that's what i did , but ....
|
[20:56] pieterh
|
i doubt it
|
[20:56] sd88g93
|
I realized zmq can do more
|
[20:57] sd88g93
|
I had a little different setup before, and then i put the forwarder back in , because i really didnt want to move zmq_msg_t 's around between threads
|
[20:57] pieterh
|
I've cleaned up (rewritten :-() the server now: http://gist.github.com/gists/572417
|
[20:57] pieterh
|
that is a messed up version but it shows you what it looks like
|
[20:57] pieterh
|
see how the queue thread creates its own sockets
|
[20:58] pieterh
|
not necessary to do that in main, kind of pointless and... in fact... illegal
|
[20:58] pieterh
|
you cannot create socket in thread A and use it in thread B
|
[21:05] pieterh
|
ok, sd88g93 that's the problem
|
[21:05] sd88g93
|
oh really
|
[21:06] pieterh
|
dude, you owe me a case of beer
|
[21:06] pieterh
|
corrected code is at http://gist.github.com/572417
|
[21:06] sd88g93
|
yes yes, i do hahaa
|
[21:06] pieterh
|
the publisher socket is not there, you can put it back of course
|
[21:07] pieterh
|
so problem was that your queue thread wasn't working reliably
|
[21:07] pieterh
|
when you test code, don't run it once, run it 10-100 times
|
[21:07] sd88g93
|
so you just create the skts for the device in that thread then,
|
[21:07] pieterh
|
threading issues are nasty like that
|
[21:07] pieterh
|
pls dnt sy skts t mks m brn hrt
|
[21:08] pieterh
|
socks if you really have to save on letters
|
[21:08] pieterh
|
but yes, you do
|
[21:08] sd88g93
|
what's wrong with skts ?
|
[21:08] pieterh
|
what's wrong with the letters 'o' and 'e'?
|
[21:08] pieterh
|
:-)
|
[21:08] pieterh
|
seriously, if you remove essential letters from words you create extra parsing costs for the reader
|
[21:08] sd88g93
|
sorry, that's just how i do variables, i tend to leave out vowels
|
[21:09] pieterh
|
english may look redundant but it's been elegantly evolved for lazy readers
|
[21:09] pieterh
|
if you want a hint for style: make your code readable
|
[21:09] pieterh
|
please compare my versions with yours
|
[21:09] sd88g93
|
those zhelper.h are helpful, but unfortunately, i need it for binary data
|
[21:10] pieterh
|
you seriously cost me several hours just to untangle your code...
|
[21:10] pieterh
|
zhelpers.h is an example, modify/extend it for binary data
|
[21:10] pieterh
|
s_recv already works for binary data
|
[21:10] sd88g93
|
well, i was hoping for zero copy as well
|
[21:10] pieterh
|
add s_send_bin (socket, data, size) and it's done
|
[21:10] sd88g93
|
ok,
|
[21:10] pieterh
|
just make sure to copy the data
|
[21:11] pieterh
|
otherwise you'll get weirdness too
|
[21:11] pieterh
|
unless you are doing a lot of traffic (100K+ / second) or very large messages, zerocopy won't be an advantage
|
[21:11] pieterh
|
but it will make your code harder to write and read and fix
|
[21:11] sd88g93
|
well, there's no way to do zero copy though, when i can find a way to do zerocopy and those functions , then we've got something
|
[21:12] pieterh
|
do not optimize code that isn't working
|
[21:12] pieterh
|
it is really a bad idea
|
[21:12] pieterh
|
actually I need to totally remove all reference to zerocopy from the user guide
|
[21:12] sd88g93
|
well, one message part can be pretty big
|
[21:12] pieterh
|
people should not even know it exists until chapter 4 or so...
|
[21:12] pieterh
|
doesn't matter
|
[21:12] sd88g93
|
like 100k
|
[21:13] pieterh
|
doesn't matter
|
[21:13] sd88g93
|
100KB , easily
|
[21:13] pieterh
|
seriously, do the test yourself
|
[21:13] sd88g93
|
really ? it wont matter ?
|
[21:13] pieterh
|
for (namewithvowels = 0; nwv < 1000; nwv++) { do large copy here }
|
[21:13] sd88g93
|
hmm ok , i'll have to think on that some
|
[21:13] pieterh
|
run, test, prove it to yourself
|
[21:13] pieterh
|
it matters if you are into microsecond latency issues and CPUs are running at 75% full
|
[21:14] pieterh
|
i'll bet you that case of beer x 2 that's not your case
|
[21:14] pieterh
|
premature optimization is one of the many roads to evil and it's not even a particularly fun one
|
[21:15] gebi
|
zero-copy can also easily be more expensive than plain copy if not aligned on cache-lines and between different cpus
|
[21:18] pieterh
|
ok, folk, /me has to go sleepy byes
|