IRC Log

Friday March 11, 2011

[Time] NameMessage
[06:38] sena hi I have a bit of cleanup problem when using zeromq.
[06:38] sena i am using the c++ classes for coding
[06:39] sena when deleting the object windows gives an run time error which says that the application was shutdown in an unusual way
[06:40] sena before deleting the objects i am deleting the sockets and then contexts
[06:40] phaplt i am newbee zmq
[06:40] sena the delete is fine but when trying to delete the encapsulating class the program gives the error
[06:41] sena any suggestions
[06:41] phaplt anybody help me tutorial
[06:41] sena The version is 2.1 rc2
[06:41] sena all the sockets are using ZMQ_PAIR
[06:42] sena this is all inter thread communication
[06:53] guido_g sena: try to give a minimal working code example and the complete error message, otherwise it's not possible to help you
[06:54] guido_g sena: put both on a paste-bin like https://gist.github.com/
[08:35] sena got the problem solved
[08:49] pieterh g'morning folks
[08:51] guido_g mornin
[09:00] pieterh hi guido_g!
[09:05] sena in PAIR socket pattern can one socket pair with many sockets
[09:18] Guthur sena: From the man: A socket of type ZMQ_PAIR can only be connected to a single peer at any one time. No message routing or filtering is performed on messages sent over a ZMQ_PAIR socket.
[09:18] Guthur http://api.zeromq.org/master:zmq-socket#toc15
[09:23] sena thanx for clarifying.
[09:24] Guthur you're welcome
[09:49] Guthur pieterh: Should the EINTR situation be documented?
[09:49] Guthur or is the mailing list message enough
[09:49] pieterh Guthur: certainly in the C# binding page...
[09:49] pieterh the mailing list message was clear
[09:49] Guthur yeah, it's not specific to C# and Mono though
[09:49] pieterh indeed...
[09:50] Guthur just more apparent due to the runtimes use of signals
[09:50] pieterh I'm still somewhat... surprised... that Linux poll() seems broken this way
[09:50] pieterh i.e. any user signaling at all will cause EINTR
[09:51] Guthur yeah, it's not nice
[09:51] pieterh there is perhaps a workaround
[09:51] pieterh there are two issues. first, trapping a real interrupt, i.e. SIGINT (Ctrl-C)
[09:52] pieterh second, continuing with the poll if it's not a real interrupt
[09:52] Guthur yep
[09:52] pieterh for the first, a normal signal handler will do it, I've an example in zhelpers.h, at the end
[09:52] pieterh for the second, the problem is calculating the time properly so the remaining poll is not longer than expected
[09:53] Guthur hehe, that was just on the tip of my tongue
[09:53] pieterh I believe there is an example of code that does this, since zmq_poll used to have the same issue of exiting randomly
[09:53] pieterh it's on the wiki somewhere, afair, let me check
[09:53] pieterh heh
[09:53] pieterh http://www.zeromq.org/topics:zmq-poll-workaround
[09:55] Guthur I could work that into clrzmq2 at the very least
[09:56] Guthur cheers for the suggestion pieterh
[09:56] pieterh np :-)
[09:56] Guthur this has been a nice adventure, hehe
[09:56] Guthur it would be good to end it with some sort of solution
[09:57] pieterh it's great that we have the binding layer where we can fix up this kind of stuff
[09:58] Guthur yeah I'm happy doing that, my worst case scenario was having to leave it for the user to deal with
[09:59] Guthur would make app code necessarily complex imo
[10:13] Guthur unnecessarily*
[10:16] guido_g unbelievable, c++ socket_t miss a close method
[10:17] guido_g so you can't close a socket in c++
[10:20] pieterh srsly? wow
[10:21] pieterh guido_g: can you provide a patch for that?
[10:21] guido_g on the way
[10:21] guido_g first issue
[10:21] guido_g then ranting
[10:21] guido_g then patch
[10:21] pieterh excellent :-) I'll make sure it gets into 2.1 stable
[10:21] guido_g stick to the process :)
[10:21] pieterh process is everything
[10:21] Guthur guido_g: I thought you were a python hack
[10:22] guido_g Guthur: sure
[10:22] guido_g re-learning c++ atm
[10:23] Guthur you have my utmost sympathies
[10:24] guido_g it's wanted in the industry, so i've close to no choice :/
[10:24] Guthur crumbs, which industry?
[10:24] Guthur broadcast?
[10:24] Guthur media*
[10:27] guido_g it consulting in generall, i'm only little conslutant ,)
[10:28] guido_g pieterh: will a simple pull req on zeromq2/master do?
[10:29] pieterh guido_g: nope, that's the one thing that won't work
[10:29] pieterh sustrik is not accepting pull requests
[10:29] pieterh (yet)
[10:29] guido_g *sigh*
[10:29] pieterh feel free to lobby him to do so, I've tried and failed (hi sustrik!)
[10:30] pieterh simplest is to send a signed patch to zeromq-dev
[10:30] guido_g forgot how to do this...
[10:30] pieterh then I can apply that to zeromq2-1/master (my git for 2.1 releases) and sustrik can do the same for zeromq2/master (source development git)
[10:31] pieterh easy
[10:31] guido_g so reading
[10:31] pieterh yeah, I don't keep it in memory cache either...
[10:31] pieterh i assume this fix can also go into 2.0 stable?
[10:31] guido_g yes, it's extremly small
[10:32] pieterh lovely lovely
[10:32] Guthur I'm sure there is benefits to going with the patch strategy
[10:32] pieterh not really
[10:33] pieterh it's better than nothing but it's weak compared to pull requests
[10:34] pieterh when all our gits are on github there is really no excuse for NOT doing pull requests except "not familiar with it, won't use it, don't trust github". Sigh.
[10:46] guido_g hope that goes through
[10:47] guido_g https://gist.github.com/865705 <- for review
[10:48] pieterh guido_g: this affects just the C++ API?
[10:48] guido_g yes
[10:48] guido_g in zmq.hpp, nothing else
[10:48] pieterh you have a test case?
[10:48] guido_g you're joking?
[10:48] pieterh ok, my question is 'what's your confidence level that this patch is correct?'
[10:49] pieterh assume I don't know the code
[10:50] guido_g this is mostly stolen from the existing destructor code and a the error reporting of other methods.
[10:51] guido_g so, given that all the other mthods are tested well... *cough*
[10:51] guido_g i sure that the patch is ok
[10:51] guido_g btw, will use it myself in a few seconds :)
[10:52] pieterh ok, I don't want to be ungrateful, but since I'd like to put this into two stable version (2.0, 2.1), could you please confirm that it actually works by running a before/after test?
[10:52] pieterh *versions
[10:52] guido_g i'm on my to compile w/ and test w/ the patch
[10:52] guido_g just a few seconds...
[10:52] pieterh ack
[10:53] guido_g oh
[10:53] guido_g this is master...
[10:53] guido_g are thre known issues?
[10:55] pieterh what do you mean?
[10:56] guido_g i'm using zeromq2-1 as development env
[10:56] guido_g normally
[10:56] pieterh sure
[10:57] pieterh I've already pushed your patch to the 2-1 master, at https://github.com/zeromq/zeromq2-1
[10:58] guido_g ok, the close works
[10:58] guido_g but...
[10:59] guido_g the destructor doesn't like it.
[10:59] guido_g next patch
[10:59] pieterh bah, pull requests only work between forks of the same repository... :-/
[11:00] pieterh cumulative patches, guido_g, or one clean one with all changes?
[11:01] pieterh I'll go undo that last patch then
[11:01] guido_g no
[11:01] lestrrat you don't really need a pullreq to do inter-repo merges.
[11:01] lestrrat in git.
[11:01] guido_g i'll send an addition
[11:01] guido_g +inal one
[11:01] pieterh lestrrat: I've used fetching + cherry-picking
[11:01] pieterh is that what you mean?
[11:02] pieterh guido_g: you'll want to submit a single clean patch to the list
[11:02] pieterh for me, fine to patch the patch
[11:03] lestrrat so if guido wants to you to pull what he's got, 1) guido fetch+merge from main repo master, 2) perterh fetches and merges from guido's repo
[11:03] lestrrat this is by far the cleanest way to merge stuff
[11:03] lestrrat cherry-picking is great if you have a really complex patch application, but generally you just want to merge
[11:03] lestrrat just make sure merging to the mainline is done cleanly
[11:03] lestrrat (i.e., fast-forward only)
[11:04] pieterh lestrrat: hang on, you overloaded my git jargon buffer... :-)
[11:04] guido_g ok
[11:04] guido_g unfortunately i send the first one to the list
[11:04] pieterh guido_g: let's try what lestrrat is describing here...
[11:04] guido_g *sent
[11:04] pieterh guido_g: just email again to cancel it...
[11:05] pieterh lestrrat: can you please walk us through this with actual git commands? Afterwards I'll document it, useful stuff
[11:05] Guthur pieterh: does patches not become more trackable
[11:05] lestrrat sure, but let me rephrase what I was saying
[11:05] Guthur making it easier to review and possibly back out
[11:05] pieterh Guthur: on an email list? up to a point, yes
[11:06] lestrrat you basically don't want to make conflicts in master. you want the cleanest possible git merge, which is "fast-forward"
[11:06] pieterh ok
[11:06] lestrrat to do this, the merge target ( guido's ) need to be aligned with the mater's history
[11:06] lestrrat that can only be done by hand -- and guido needs to do this by pulling from master
[11:07] pieterh lestrrat: hang on, let me explain the context a little
[11:07] lestrrat sure
[11:08] pieterh we have three gits, a 2-2 master, a 2-1 master, and a fork of the 2-1 master which guido_g is working on
[11:08] guido_g going to make a clean patch now
[11:08] pieterh we want to get changes from guido_g's git into the two other gits
[11:09] pieterh the 2-2 master and 2-1 master have diverged some time ago
[11:09] lestrrat k
[11:09] pieterh 2-1 cherry-picks specific commits from 2-2, over time
[11:09] pieterh we also have a 2-0 master
[11:09] pieterh :-) that's our arrangement
[11:09] pieterh we're now looking for a simple process everyone can use
[11:10] lestrrat between 2-2 and 2-1 , you'd need to cherry-pick
[11:10] pieterh right
[11:10] lestrrat between 2-1 and guido_g, you shouldn't
[11:10] pieterh we'd have made 2-1 and 2-0 *forks* except github doesn't allow that within one username
[11:11] pieterh ok, so guido_g first pulls from 2-1 master, makes sure his history is correct
[11:11] lestrrat well
[11:11] lestrrat forking doesn't change anything in this context
[11:11] pieterh ok'
[11:12] lestrrat if you want to merge 2-1 and 2-0 (or whatever else), all you really need to do is to let your local git repo know about these repos
[11:12] lestrrat using "git remote"
[11:12] lestrrat and then, git merge would just allow you to merge these.
[11:12] lestrrat (no pullreqs required)
[11:12] pieterh right
[11:12] lestrrat you merge, tweak, and then "git push 2-1 mybranch" or whatever.
[11:13] pieterh the advantage of pullreqs is that they are like issues, trackable, discussion, etc.
[11:13] lestrrat yeah, that's understood, but if somebody wants to make changes to 2-1 and 2-0, you just going to have to
[11:14] lestrrat 1) fork 2-1 2) make changes and send pullreq to 2-1 3) do the above git remove dance, and merge stuff to 2-0 3) push to github 2-0
[11:14] pieterh yeah, and forking isn't an option, so pull reqs are basically out
[11:15] pieterh ok, fast-forward merging, makes sense
[11:15] pieterh thanks, I'll do my reading and make sure I understand that
[11:15] Guthur we have just had the craziest enquiry from one of our clients, a rather large internet based firm, their legal team fears that because of the OCaml and Ruby code generators in the Thrift library their code maybe infected by GPL
[11:15] Guthur we don't even use these code generators, doh
[11:15] Guthur I think they need a new legal team
[11:16] lestrrat on a different project with about 10 devs, they just follow that rule: "all merges to master MUST be fast-forwards (to make sure they apply cleanly)" "Otherwise, create your own branch and do whatever you want to do. When you're done, let us know the name of the branch so we can merge to master"
[11:17] pieterh guido_g: ok, waiting for your new patch
[11:18] lestrrat well, good luck with the merges :)
[11:18] pieterh :-) thanks
[11:19] pieterh lestrrat: good luck, I hope things aren't too bad
[11:19] lestrrat things around my home is sane enough that I can blab about git merges :) but thanks anyways
[11:23] guido_g https://gist.github.com/865763 < -the new and better tested one
[11:24] pieterh "tested", that has a nice ring about it
[11:25] pieterh guido_g: ok, thanks, applying that now to 2-0 and 2-1
[11:25] guido_g tested means compiled and used in a program w/o problems
[11:28] guido_g and now for the ml again
[11:30] pieterh ok, applied to 2-0 and 2-1
[11:30] guido_g and sent to the ml
[11:30] guido_g i should really become more patient
[11:31] pieterh nah, we release maintainers just have to make sure to ask the right questions
[11:33] guido_g btw, why did you disable multicast loop in 2.1?
[11:33] pieterh Steve-o's contribution
[11:35] guido_g should be mentioned at a prominent place in the release notes, because it will break single-box testing
[11:35] pieterh there was iirc some discussion of this on the list, let me check...
[11:38] pieterh there is some discussion about a week ago but tbh I'm not following it totally
[11:39] guido_g yes, but everyone is reading the ml or remembers all topics going on there
[11:39] guido_g *but *not*
[11:39] pieterh yes, I meant just for us, now, to understand this
[11:39] pieterh afaics Steve says multicast loop doesn't work on Linux and lo is lossy
[11:40] guido_g going to read the patch...
[11:44] guido_g seems like the socket optoin for multicast loopback is turned off, nothing to w/ the lo interface
[11:45] pieterh I'd suggest asking Steve, on the ml, where he posted the patch, what the rationale is
[11:46] guido_g ok
[11:51] guido_g time for a break (http://2.bp.blogspot.com/_ELzM_S573r8/SJthznx2u3I/AAAAAAAAGgU/dqPtF2Xr718/s400/computer-pause.jpg)
[11:51] pieterh :-)
[12:12] Eruquen Just one quick question: "When sending messages a ZMQ_XREP socket shall remove the first part of the message [..]". Is that my job or is it what XREP does automatically?
[12:14] pieterh Eruquen: this is how the XREP / ROUTER socket works, read Ch3 of the guide for extensive explanation
[12:14] pieterh the socket gives you an extra frame on recv and expects an extra frame on send
[12:14] pieterh that frame has the identity of the sender, and the recipient, respectively
[12:15] Eruquen great, thank you.. I think that was the only piece of the puzzle I was missing
[13:10] guido_g re
[16:08] Guthur has anyone ever looked at using ZeroMQ as the transport for Thrift RPC stuff?
[16:33] Guthur guido_g: It was more the RPC stuff
[16:33] Guthur Thrift seems to have succumb to the we will do everything project syndrome
[16:34] Guthur It even offers transport and server solutions
[16:34] Guthur guido_g: sorry I should have read closer
[16:35] Guthur it does RPC
[16:35] Guthur msgpack
[16:35] guido_g i know :)
[16:35] guido_g and can read thrift idl
[16:35] guido_g at least they claim that it can
[16:54] guido_g at least, it compiles :)
[17:08] Guthur guido_g: is there more marketing than substance
[17:08] jobytaffey Hi, I'm having a problem with zeromq-2.1.0. I have a program in C, using libev to have one ZMQ REP socket and stdin. I'm compiling against libefence for malloc checking and I'm getting ocassional crashes. But, when I remove libefence I can't reproduce the problem. Could it be a problem with efence's malloc wrappers not being thread safe? Or a 0mq bug? http://pastebin.com/cLdQ8Fms
[17:09] guido_g Guthur: yes, it's all a little shallow
[17:09] Guthur the 4x faster than protocol buffers seems hard to believe
[17:09] Guthur in all but the most contrived test
[17:11] guido_g otoh, i like the dynamic typing feature
[17:17] pieterh jobytaffey: could well be a problem with efence, yes
[17:17] pieterh I'd suggest valgrind instead
[17:18] jobytaffey Thanks, I'll try it.
[17:23] pieterh jobytaffey: http://zguide.zeromq.org/page:all#toc31
[17:24] pieterh all, I've rewritten the contribution tutorial, it was a bit messy
[17:24] jobytaffey Thanks again.
[17:24] pieterh np
[17:41] cremes can someone put the link to the 0mq irc logs up in the channel topic please?
[17:41] cremes i always forget what it is...
[17:43] pieterh cremes: it's travlr who keeps it, afair, perhaps the link should go on this page http://www.zeromq.org/chatroom
[17:43] pieterh we don't seem to have a channel admin
[17:44] cremes indeed, you are correct
[17:45] cremes i'll add a link to travlr's github page to the wiki
[17:48] cremes oh, it's already there
[17:48] pieterh :-)
[17:48] cremes i'll make it more prominent
[17:48] pieterh I didn't even see it...
[17:49] cremes refresh and look
[17:49] cremes it's slightly easier to find
[17:50] pieterh still seems very bland, let me try...
[17:52] pieterh try now, cremes
[17:52] cremes ooh, even better
[17:53] pieterh someday a student of technology history will read these logs and gaze in wonder...
[17:53] pieterh "how did they actually *do* anything, with all that inane chatter...?"
[17:55] michelp now that I know they exist i'll be reading them myself ;)
[17:55] cremes s/inane/insane/
[17:56] michelp BTW freenode policy asks that you inform all channel users of logging in a topic or autojoin notice of some kind.
[17:56] cremes though both fit
[17:56] pieterh cremes: you take insane, I'm better with inane
[17:56] pieterh michelp: true, but we seem to have lost our channel admin somewhere in 2009
[17:57] pieterh michelp: I guess stating this on the IRC page on the wiki is almost good enough
[17:57] michelp ah i see. i bet there are some freenode peoples who can help you with that
[17:58] michelp yeah, as long as you say it somewhere it's better than nowhere ;)
[17:58] michelp is the topic locked down? in other channels I am in anyone can change it
[17:59] michelp yeah :(
[17:59] pieterh op
[17:59] pieterh nope, that didn't work :-)
[18:00] michelp i'll ask in #freenode
[18:00] michelp i bet we can either get the topic unlocked or get someone op'ed
[18:00] michelp or both ;)
[18:01] pieterh unlocked would be simplest, we don't need an op otherwise
[18:04] michelp well there's no staff currently in #freenode, i'll try again in a bit
[18:05] pieterh yeah, no hurry
[18:05] pieterh actually I wanted the topic to say, "Have you read the Guide yet?"
[18:06] michelp i've been reading through the docs myself the past couple of days, 0mq is pretty fascinating
[18:06] michelp that's why i jumped in here was to see what was going on
[18:06] pieterh it's pretty fascinating, indeed... :-)
[18:06] pieterh have you read the Guide yet?
[18:07] michelp we use amqp currently where i work and personally I'm pretty unhappy with it
[18:07] pieterh amqp/0-9-1?
[18:07] michelp skimmed it, i've started with the docs comparing amqp to 0mq mostly to try and get my head around it
[18:09] michelp i'm not sure which protocol version, we use rabbitmq. i didn't build that part of the system, i just inject messages into it and pray they get somewhere
[18:09] pieterh 0-9-1, then
[18:09] michelp i may end up writing the bit that replaces it ;)
[18:10] pieterh it's a good implementation but the protocol is clunkier than it should be
[18:11] michelp are there centralized brokers written on top of 0mq? the docs say that it can be done as one of the many possible patterns but some digging around i didn't find any. preferably one written in python which is our primary shop language
[18:11] michelp i didn't dig to hard or necessarily in the right direction
[18:12] pieterh so... in fact we only recently finished our first protocol spec able to turn into real brokers and clients
[18:12] pieterh http://rfc.zeromq.org/spec:7
[18:13] pieterh there is a small implementation in C, and one in Python by guido_g
[18:13] michelp ah nice
[18:13] michelp i'll check that out
[18:13] pieterh depending on the kind of messaging you're doing, you may want brokers (of various types), or not
[18:14] michelp we might be able to give it some production time, we have been kicking around the idea of having a "shadow" message system that runs alongside rabbit so we can evaluate other technologies
[18:14] michelp we push a few hundred thousand message through per day so it would be a good medium-scale test
[18:14] pieterh 0MQ architects end up making their own mini brokers, highly customized, for the places that need them
[18:14] pieterh this is what 0MQ is really for, a toolkit to make the architectures you need
[18:15] pieterh reliable messages?
[18:15] michelp reliable in the sense that they can't be dropped?
[18:15] pieterh have to survive crashes etc.
[18:15] michelp yeah, they are critical to our business flows
[18:15] michelp lots of our backend is async
[18:16] pieterh so, read the Guide and especially Ch4
[18:16] michelp awesome, thanks for the pointers, i'll check them out this weekend
[18:16] pieterh np
[18:17] michelp i'm sure i'll have some questions so hopefully i don't annoy you too much until it all sinks in my head
[18:17] kdj I've found the examples in the guide really helpful
[18:18] pieterh ideally, read the Guide slowly over a week, try every example, and it'll become clear
[18:18] pieterh kdj: thanks :-)
[18:26] michelp kdj, oh hey Kyle ;) not at pycon?
[18:26] kdj Heh, I wish
[18:26] kdj Are you there?
[18:26] michelp kdj, not that I've seen you in here I'm going to have to bug you about giving a 0mq talk at the next pdxpython meetings
[18:27] michelp nah... i've been to enough pycons. i always end up just hanging out with jaded zope programmers from the good old days and drinking too much whiskey
[18:27] kdj pieterh: Thanks for putting together the guide. :)
[18:27] kdj michelp: Haha
[18:28] pieterh kdj: my pleasure, this will hopefully become a printed book
[18:28] michelp kdj, what do you guys use 0mq for?
[18:29] kdj michelp: A couple pretty simple work broker and task status implementations
[18:30] michelp using python?
[18:30] kdj Python and C#
[18:31] michelp how long did you think you could get away with not giving a presentation at pdxpython on that? ;)
[18:32] kdj Long enough to actually have something worth presenting. :)
[18:32] pieterh Ian Barber raised the 0MQ presentation stakes somewhat
[18:33] michelp is there a video?
[18:34] michelp looks like he's talking at pycon
[18:35] kdj Here are some slides, it looks like? http://www.slideshare.net/IanBarber/zeromq-is-the-answer
[18:36] michelp ah zed is talking too
[18:36] michelp http://us.pycon.org/2011/schedule/presentations/98/
[18:37] kdj Nice
[18:38] michelp now i kinda want to go to pycon just to see that one talk ;)
[18:40] kdj Gah. Yes. How long does it usually take for them to get the videos up? :)
[18:40] michelp days, weeks, never
[18:40] kdj Never is a long time.
[18:41] kdj I hope those presentations happen to get put up
[18:51] michelp so it looks like dermoth sustrik and mato are channel admins
[18:51] michelp according to chanserv
[18:53] michelp so if dermoth|home can op someone else we can get the channel topic unlocked
[19:00] michelp pieterh, so a freenode staffer said if you follow this process they can help unlock the channel http://freenode.net/group_registration.shtml
[19:01] michelp the process is verified by phone, apparently
[19:01] pieterh ok, will take care of it
[19:01] michelp here is a blog post about it http://freenode.net/group_registration.shtml
[19:01] michelp oops
[19:01] michelp http://blog.freenode.net/2010/06/group-registration-form-verifications/
[19:03] michelp the staffer wants me to tell him when it's sent, i can do that or you can contact him directly
[19:05] michelp pieterh, his nick is 'niko' and he says you can contact him directly when it's filed
[19:05] pieterh ok, will do...
[19:05] pieterh thanks a lot michelp
[19:05] michelp yeah no sweat
[19:05] michelp it's the least I can do until i crank out some 0mq using code :)
[19:07] pieterh I'm randomly electing myself "Chair" of the 0MQ community so I can tell them what my role is here
[19:08] pieterh Anyone wants to fight me for chair, please wait until the music stops
[19:09] pieterh michelp: form is submitted, why not ping niko, he'll be happy to hear from you
[19:09] michelp sure thing
[19:10] michelp for future reference the syntax for a private message is "/msg <nick> blah blah blah..."
[19:10] pieterh I'm so happy in my ignorance...
[19:10] michelp irc is the original distributed message system ;)
[19:11] pieterh indeed
[19:15] guido_g pieterh: is it ok to call all the message frames between empty frames a message segment?
[19:16] pieterh 'segment' has no defined meaning, nor do two empty frames have any semantics that I'm aware of
[19:16] pieterh I tend to stick to "envelope" vs. "body"
[19:17] pieterh what's the context?
[19:17] guido_g but we make heavy use of empty frame separated multipart messages
[19:17] guido_g a multipart message class
[19:17] pieterh 'we' is your project, right?
[19:17] pieterh then 'segment' seems fine
[19:17] guido_g we as in mdp protocol
[19:18] guido_g ok
[19:18] pieterh ah, hang on, nope in that case
[19:18] guido_g huh?
[19:18] pieterh we don't, in MDP, give any special meaning to bunches of frames separated by empty frames
[19:18] pieterh do we?
[19:19] pieterh if you add such meaning in your implementation, it can break arbitrarily when we modify the protocol
[19:19] guido_g it's not about the meaning, it's just for the fact that there are logical groupings of frames
[19:19] pieterh the only two cases I see are in REQUEST and REPLY
[19:19] guido_g in mdp, yes
[19:20] guido_g but i'm on a general multipart message class
[19:20] pieterh and there, the first empty part is... meaning free
[19:20] pieterh in MDP what we have are 'envelope' and 'body'
[19:20] guido_g so i came up with the idea of having a method that helps on getting these groups
[19:20] pieterh personally I'd not invent concepts that don't come from actual use
[19:21] pieterh weird semantics that don't quite work
[19:21] guido_g can't follow you
[19:21] pieterh well, in all the examples I've made so far, this notion of groups of frames doesn't occur once
[19:22] pieterh we use frames in MDP because we're too lazy to do encoding
[19:22] pieterh the real semantics, the meaning, in MDP is a stack
[19:22] pieterh the body is the last set of frames
[19:22] pieterh the envelope comes before that
[19:22] pieterh both may contain empty frames
[19:23] pieterh 0MQ REP socket has a similar notion, of envelope ending in empty frame, followed by body
[19:23] pieterh a stack
[19:23] pieterh do I make sense?
[19:24] guido_g nevertheless an empty frame is a very easy and already implemented method of doing this logical grouping
[19:24] pieterh two problems
[19:25] pieterh a. "groups of frames" is not a concept I've seen actually used or useful
[19:25] Bloudermilk Hi all. I was hoping someone could voice their opinion/experience on using 0mq with media streaming. My initial estimate is that it adds significant overhead, but I'm not sure
[19:25] pieterh b. when you could claim "groups" in real cases, they can contain empty frames
[19:25] pieterh Bloudermilk: why would it add overhead? Compared to what?
[19:25] cremes Bloudermilk: i'm using it for disseminating market data which needs low latency and low overhead
[19:25] Bloudermilk pieterh: Comapred to vanilla TCP
[19:25] cremes i wouldn't have chosen 0mq otherwise
[19:26] pieterh Bloudermilk: ... what is your estimate based on?
[19:26] pieterh and how do you define "significant"?
[19:26] Bloudermilk pieterh: Absolutely nothing :) That's why I wanted to ask
[19:27] Bloudermilk pieterh: I'd define significant as a subjective point where the performance overhead outweighs the convenience
[19:27] pieterh ok, no, it does not add significant, or even moderate overhead
[19:27] pieterh not even small overhead
[19:27] Bloudermilk That's great news!
[19:27] pieterh something between "tiny" and "insignificant", I'd say
[19:27] Bloudermilk Haha
[19:28] pieterh it is easy to measure with running code, and you should try
[19:28] pieterh guido_g: go for it, it could work, my initial feeling is it's not a natural 'pattern' for using multipart messages but it could be within some applications
[19:28] guido_g ok
[19:29] pieterh just don't come back and ask me to make MDP work better with 'segments' :-)
[19:29] guido_g nope
[19:29] guido_g its working fine
[19:29] guido_g using it to test my c++ ioloop :)
[19:29] pieterh nice...
[19:30] pieterh do announce your implementation on zeromq-dev and the Labs page so people know about it
[19:30] guido_g https://github.com/guidog/cppzmq/blob/master/work/messages.cpp <- the test program, so far
[19:30] guido_g will do, after some more tests and some *sigh* documentation
[19:31] pieterh guido_g: do you have a dedicated key on your keyboard that prints *sigh*?
[19:31] guido_g not yet
[19:31] pieterh lol
[19:59] mikko howdy
[20:57] troutwine Compiling 2.0.10 from source on a Posix system I find that there's no .pc for pkg-config, than pkg-config does not list libzmq as a known package. I'm doing the standard ./configure&&make&&make install. Am I forgetting something?
[21:02] guido_g get 2.1.x, the build system was improved
[21:02] guido_g and it's more stable
[21:10] troutwine guido_g: More stable?
[21:10] guido_g yes
[21:10] guido_g lots of bugs removed
[21:12] troutwine Thanks.
[21:57] troutwine I'm running wuserver with wuclient from the Guide on 2.1.2. When I netstat -ant I see the following: https://gist.github.com/866653
[21:57] troutwine For each pub/sub connection made to the wuserver, why are two sockets in use?
[22:02] troutwine I am reading netstat's output correctly, yes?
[22:15] guido_g no
[22:15] guido_g read the heading of the netstat output
[22:16] guido_g it says local and foreign address
[22:16] guido_g and a tcp connection has 2 endpoints, so for every endpoint is a local and a foreign view
[22:17] guido_g add the -p flag and see which processes are using the connections
[22:21] troutwine guido_g: Thanks. I need to read up on TCP more, clearly.
[22:23] michelp might I suggest http://www.amazon.com/Internetworking-TCP-IP-Vol-5th/dp/0131876716
[22:34] michelp the guide is pretty mindblowing
[22:35] michelp i got a chuckle that it goes Chapters One, Two, Three and 4

Web site design and content is copyright © 2010-2011 Peter Alexander, ProDataLab. Site content licensed under the Creative Commons Attribution-Share Alike 3.0 License. ØMQ is copyright © Copyright © 2007-2011 iMatix Corporation and Contributors. Contact us for professional support. ØMQ is free software licensed under the LGPL. ØMQ, ZeroMQ, and 0MQ are trademarks of iMatix Corporation. Terms of Use ? Privacy Policy