ZeroMq IRC Log

Saturday July 24, 2010

[Time] Name	Message
[01:10] ben_k	Can some explain this code ?
[01:10] ben_k	size_t zmq_msg_size (zmq_msg_t *msg_)
[01:10] ben_k	{
[01:10] ben_k	if (msg_->content == (zmq::msg_content_t*) ZMQ_VSM)
[01:10] ben_k	return msg_->msgs.vsm.vsm_size;
[01:10] ben_k	if (msg_->content == (zmq::msg_content_t*) ZMQ_DELIMITER)
[01:10] ben_k	If VSM return VSm
[01:12] ben_k	Ifhas Delimiter ( what does this mean ? ) return empty . Its the last line that has me stumped if its full and not a VSM return the size of the VSM size field ( which is limited to 256 bytes)
[01:50] ben_k	Ignore that ...
[14:39] mato	sustrik: are you there?
[14:39] sustrik	here I am
[14:39] mato	sustrik: a question
[14:40] mato	sustrik: when we talked about the funky pollset manipulation needed when forwarding requests between sockets that have flow control enabled, you said "go look at the queue device, i do it there"
[14:40] mato	so i went and looked, and you don't
[14:40] mato	:-)
[14:40] sustrik	let me see
[14:40] mato	is there some other place i should look for an example or do i need to develop the algorithm from scratch myself?
[14:41] sustrik	what's missing in queue.cpp?
[14:42] mato	imagine send can block
[14:42] mato	e.g. insocket has an infinite queue length, outsocket has a low queue length (say 1)
[14:43] sustrik	aha, i see
[14:43] sustrik	yes, it's missing there
[14:43] mato	i will of course do this ad-hoc, i'm just trying to visualize a generic model to see if there is one
[14:43] sustrik	you'll have to write it yourself then
[14:44] mato	am doing that, can then patch the queue device maybe at some point sice currently it'll break if flow control is involved
[14:45] sustrik	right, that would be good
[14:45] sustrik	the algo is not that complex imo
[14:45] mato	not really, it just has the ugly property of maintaining various bits of state
[14:46] mato	which i'm wondering if i can abstract away in some kind of smart poller api
[14:51] sustrik	specifically in queue.cpp the state can be kept in the pollitem array
[15:19] CIA-20	zeromq2: 03Martin Hurton 07master * re1c596b 10/ src/pipe.cpp : Make sure lwm > 0 when hwm > 0 - http://bit.ly/db7UyA
[15:38] CIA-20	zeromq2: 03Martin Hurton 07master * r10533a5 10/ (src/pipe.cpp src/pipe.hpp src/ypipe.hpp): pipe: check_read() should check for message delimiter - http://bit.ly/cQ2Jij
[16:46] mato	sustrik: nice discussion on the mailing list :-)
[16:46] sustrik	yes, i like it
[16:46] mato	sustrik: i like brian saying "we find that just building stuff on top of 0MQ the right way works best" :-)
[16:46] sustrik	professional
[16:46] mato	you forgot to mention that pieter seems to want to build a new application the "non-scalable" way :-)
[16:47] mato	sustrik: anyway, i think i have my smart poll idea thought out
[16:47] sustrik	ok, good
[16:47] mato	sustrik: remember when i mentioned "poll with conditions" ?
[16:47] sustrik	yes, i do
[16:47] mato	sustrik: actually, that's not what i need
[16:48] sustrik	so?
[16:48] mato	i just need one extra thing besides the usual "can read" or "can write"
[16:48] mato	which is "can forward from A to B"
[16:49] sustrik	isn't it just a conjunction of the two?
[16:49] mato	not if flow control is involved
[16:50] sustrik	?
[16:50] mato	or maybe i misunderstand what you mean by conjunction
[16:50] sustrik	&&
[16:51] mato	well yes, basically it means readable(A) && writable(B) but the problem is you can't get that out of a single poll call
[16:51] mato	hence the wrapper class
[16:52] mato	?
[16:52] sustrik	zmq_poll ({A, POLLIN}, {B, POLLOUT})
[16:52] sustrik	what's wrong with that?
[16:52] mato	that's \|\| not &&
[16:52] sustrik	sure, you have to do && afterwards
[16:53] mato	won't work if A has 1M requests queued up and B is slowly consuming with a small HWM
[16:53] mato	well, will work but will busy loop like mad
[16:53] sustrik	ok, i see
[16:53] sustrik	so what's your idea?
[16:54] mato	oh, nothing too magic, I will just take Brian's Python poller wrapper class and add functionality to allow me to register can_forward(from,to) conditions
[16:54] mato	which will be handled smartly internally
[16:55] mato	i.e. avoiding busy looping by manipulating the actual pollset
[16:55] mato	makes sense, no?
[16:56] mato	sustrik: anyway, i'll try this and make some notes how it goes, i think this would be a useful trick to have if we want to invent a more interesting polling api
[16:56] sustrik	goodo
[16:57] mato	sustrik: since it's really useful for anything device-like
[16:57] sustrik	right, having a nice design for that kind of stuff would be usefull
[16:57] mato	indeed