Saturday July 24, 2010

[Time] NameMessage
[01:10] ben_k Can some explain this code ?
[01:10] ben_k size_t zmq_msg_size (zmq_msg_t *msg_)
[01:10] ben_k {
[01:10] ben_k if (msg_->content == (zmq::msg_content_t*) ZMQ_VSM)
[01:10] ben_k return msg_->msgs.vsm.vsm_size;
[01:10] ben_k if (msg_->content == (zmq::msg_content_t*) ZMQ_DELIMITER)
[01:10] ben_k If VSM return VSm
[01:12] ben_k Ifhas Delimiter ( what does this mean ? ) return empty . Its the last line that has me stumped if its full and not a VSM return the size of the VSM size field ( which is limited to 256 bytes)
[01:50] ben_k Ignore that ...
[14:39] mato sustrik: are you there?
[14:39] sustrik here I am
[14:39] mato sustrik: a question
[14:40] mato sustrik: when we talked about the funky pollset manipulation needed when forwarding requests between sockets that have flow control enabled, you said "go look at the queue device, i do it there"
[14:40] mato so i went and looked, and you don't
[14:40] mato :-)
[14:40] sustrik let me see
[14:40] mato is there some other place i should look for an example or do i need to develop the algorithm from scratch myself?
[14:41] sustrik what's missing in queue.cpp?
[14:42] mato imagine send can block
[14:42] mato e.g. insocket has an infinite queue length, outsocket has a low queue length (say 1)
[14:43] sustrik aha, i see
[14:43] sustrik yes, it's missing there
[14:43] mato i will of course do this ad-hoc, i'm just trying to visualize a generic model to see if there is one
[14:43] sustrik you'll have to write it yourself then
[14:44] mato am doing that, can then patch the queue device maybe at some point sice currently it'll break if flow control is involved
[14:45] sustrik right, that would be good
[14:45] sustrik the algo is not that complex imo
[14:45] mato not really, it just has the ugly property of maintaining various bits of state
[14:46] mato which i'm wondering if i can abstract away in some kind of smart poller api
[14:51] sustrik specifically in queue.cpp the state can be kept in the pollitem array
[15:19] CIA-20 zeromq2: 03Martin Hurton 07master * re1c596b 10/ src/pipe.cpp : Make sure lwm > 0 when hwm > 0 -
[15:38] CIA-20 zeromq2: 03Martin Hurton 07master * r10533a5 10/ (src/pipe.cpp src/pipe.hpp src/ypipe.hpp): pipe: check_read() should check for message delimiter -
[16:46] mato sustrik: nice discussion on the mailing list :-)
[16:46] sustrik yes, i like it
[16:46] mato sustrik: i like brian saying "we find that just building stuff on top of 0MQ the right way works best" :-)
[16:46] sustrik professional
[16:46] mato you forgot to mention that pieter seems to want to build a new application the "non-scalable" way :-)
[16:47] mato sustrik: anyway, i think i have my smart poll idea thought out
[16:47] sustrik ok, good
[16:47] mato sustrik: remember when i mentioned "poll with conditions" ?
[16:47] sustrik yes, i do
[16:47] mato sustrik: actually, that's not what i need
[16:48] sustrik so?
[16:48] mato i just need one extra thing besides the usual "can read" or "can write"
[16:48] mato which is "can *forward* from A to B"
[16:49] sustrik isn't it just a conjunction of the two?
[16:49] mato not if flow control is involved
[16:50] sustrik ?
[16:50] mato or maybe i misunderstand what you mean by conjunction
[16:50] sustrik &&
[16:51] mato well yes, basically it means readable(A) && writable(B) but the problem is you can't get that out of a single poll call
[16:51] mato hence the wrapper class
[16:52] mato ?
[16:52] sustrik zmq_poll ({A, POLLIN}, {B, POLLOUT})
[16:52] sustrik what's wrong with that?
[16:52] mato that's || not &&
[16:52] sustrik sure, you have to do && afterwards
[16:53] mato won't work if A has 1M requests queued up and B is slowly consuming with a small HWM
[16:53] mato well, will *work* but will busy loop like mad
[16:53] sustrik ok, i see
[16:53] sustrik so what's your idea?
[16:54] mato oh, nothing too magic, I will just take Brian's Python poller wrapper class and add functionality to allow me to register can_forward(from,to) conditions
[16:54] mato which will be handled smartly internally
[16:55] mato i.e. avoiding busy looping by manipulating the actual pollset
[16:55] mato makes sense, no?
[16:56] mato sustrik: anyway, i'll try this and make some notes how it goes, i think this would be a useful trick to have if we want to invent a more interesting polling api
[16:56] sustrik goodo
[16:57] mato sustrik: since it's really useful for anything device-like
[16:57] sustrik right, having a nice design for that kind of stuff would be usefull
[16:57] mato indeed