Wednesday March 2, 2011

[Time] NameMessage
[00:30] skm if a pusher has pushed 100msgd, the first client to call recv then gets all 100 msgs (and has to recv 99 more times to see them) - correct?
[01:22] Guthur skm, according to the docs push will go into an exceptional state if there is nothing downstream to recv
[01:22] Guthur
[01:23] Guthur and it's load balanced across any connected downstream peers
[01:28] skm i want to know what happens though when multiple clients connect with 100pending messages
[01:28] skm and the first connected client calls recv
[01:28] skm does it get one message or are all messages then sent to that
[01:29] skm im using the nodejs javascript binding which that is the case - the first recv gets all messages (and needs to call recv another 99 times to read the other messages)
[01:29] skm im just wondering if that is because of the binding or if that's the way 0mq works
[01:30] Guthur umm does sound right to me
[01:30] Guthur doesn't
[01:30] Guthur well that doesn't sound like it full fills the load-balancing aspect
[01:32] Guthur I can't say for sure to be honest
[01:32] Guthur someone else will have to clarify
[02:43] jugg pieterh, defaults to 'master/2.2.0'. Perhaps it should default to the current stable release?
[02:44] jugg eg, the same as the redirects from the old api pages.
[07:19] Steve-o almost complete for next release of OpenPGM, just need platform tests on Autoconf & Cmake
[07:19] Steve-o performance docs uploaded for Windows, Linux, Sparc, general latency improvement all around
[08:01] CIA-21 zeromq2: 03Martin Sustrik 07master * r5fcef1c 10/ (9 files in 3 dirs):
[08:01] CIA-21 zeromq2: ZMQ_MAXMSGSIZE option added
[08:01] CIA-21 zeromq2: The new option allows user to guard against peers sending
[08:01] CIA-21 zeromq2: oversized messages. Connection to peer sending oversized message
[08:01] CIA-21 zeromq2: is dropped.
[08:01] CIA-21 zeromq2: Signed-off-by: Martin Sustrik <> -
[08:48] eyecue *waves* o/~
[08:48] eyecue watching pieters preso, very cool
[09:30] yrashk ok so apparently erlang binding (say, ezmq) isn't that slow
[09:30] yrashk the perf tests were erroneously scripted as interpreted code
[09:30] yrashk which slowed things down
[09:30] sustrik heh
[09:30] sustrik what are the figures now?
[09:31] yrashk 11-13mb/s
[09:31] sustrik and in msgs/sec?
[09:31] yrashk I don't have that data yet, let me ask the guy who discovered this
[09:32] sustrik sure
[09:35] yrashk 750694msg/s
[09:36] yrashk minor correction, with erlang-to-erlang local-remote it's rather 6mb/s
[09:36] yrashk but still that 750694msg/s
[09:36] yrashk still way better
[09:43] yrashk this is exciting
[09:43] yrashk sustrik: ^^
[09:51] Steve-o that's on 100mb or 1gb?
[09:51] yrashk that's on localhost I believe
[09:53] Steve-o eek
[09:55] Steve-o that's a bit of a hit
[10:02] jugg yrashk, have you confirmed this, as I tried compiling the code previously and it made no difference.
[10:02] yrashk jugg: I can't confirm this myself yet, but the guy clais this is what he gets on compiled modules (not on compiled escripts), and he's promising to send a pull req soon
[10:03] yrashk jugg: have you been trying to compile for erlzmq? using -mode(compile) or by converting them into modules?
[10:03] jugg both, neither changes performance.
[10:03] pieterh re
[10:03] yrashk well maybe ezmq is superior? ;)
[10:03] jugg :)
[10:03] yrashk after all it doesn't have that decoding/encoding overhead
[10:04] jugg I'll be interested if you can confirm the results.
[10:06] yrashk I am waiting for that pull req
[10:06] yrashk I hope I will be able to confirm this soon
[10:06] eyecue reading the guide :]
[10:09] eyecue the install guide mentions uuid-dev and uuid/e2fsprogs libs, but the FreeBSD port for ezmq doesnt seem to depend on them. are these conditional/optional dependencies?
[10:11] yrashk you mean the fbsd port of 0mq?
[10:11] yrashk because ezmq is an erlang library
[10:12] yrashk because ezmq is the erlang library for 0mq*
[10:12] eyecue apologies :)
[10:12] eyecue 0mq.
[10:12] eyecue under for reference
[10:13] eyecue yay, python bindings are in ports too
[10:19] pieterh yrashk: are those figures published anywhere?
[10:23] yrashk pieterh: not yet, it's the other guy who's fixing perf tests
[10:23] pieterh 750K, up from, what was it, 20K or so?
[10:23] pieterh pretty nice
[10:28] yrashk pieterh: ya
[10:28] yrashk from 30Kish
[10:33] yrashk pieterh: it's basically without changing ezmq, just fixing perf tests
[10:34] yrashk although according to jugg the same approach didn't help speeding up erlzmq perf tests
[10:35] pieterh ezmq is the latest rewrite, right?
[10:35] yrashk pieterh: ezmq is my NIF-based complete rewrite
[10:35] yrashk both erlzmqs (yeah, there are two of them) are port driver-based
[10:35] pieterh right, so eventually you want to merge all these together and call the result erlzmq again, I guess
[10:36] yrashk not necessarily
[10:36] yrashk and this is unlikely to happen I think
[10:36] pieterh hmm
[10:36] yrashk they are both quite different
[10:36] pieterh different APIs for apps?
[10:36] yrashk different approaches to interfacing with C
[10:37] pieterh is that relevant to users, except for performance?
[10:37] yrashk it is not
[10:37] yrashk for the most part
[10:37] pieterh so when I say 'merge' I mean, replace
[10:38] yrashk it's really up to saleyn and jugg, really
[10:38] pieterh of course
[10:38] yrashk I am just trying to maintain my own binding as I personally prefer NIF bindings
[10:39] yrashk I don't really care if it will be an official binding or not
[10:39] eyecue yrashk; sorry, newbie here, NIF? :)
[10:39] yrashk eyecue: Native Implemented Function, erlang term
[10:39] eyecue ta :]
[10:39] yrashk pieterh: I am fine if ezmq will remain as secondary binding
[10:40] yrashk as a*
[10:40] eyecue pieterh; btw, loving what i see/read about 0mq so far. trying to figure out how i can apply it to our email architecture
[10:40] pieterh yrashk: I'd personally use names that are more explanatory
[10:41] pieterh e.g. erlzmq-nif
[10:41] pieterh but it's not my call
[10:41] pieterh eyecue: glad you like it
[10:41] eyecue yrashk; just came across the UUID reference in the guide btw: All ?MQ sockets have identities but by default they are generated 'unique universal identifiers' (UUIDs)
[10:42] eyecue perhaps only relevent when using identies with durable sockets?
[10:42] yrashk pieterh: well... I don't have an opinion on that right now
[10:43] yrashk I think erlzmq and erlzmq-nif implies that the latter is a fork of a sort or soemthing
[10:44] yrashk but ezmq is in fact a full rewrite
[10:44] yrashk except for the constants
[10:44] yrashk I admit I stole the header file
[10:44] yrashk either way
[10:45] yrashk this is a minor issue (naming and whatnot)
[10:46] pieterh there are similarly several versions of clrzmq
[10:46] pieterh the latest one simply called itself clrzmq2
[10:46] yrashk .net?
[10:46] pieterh yes
[10:47] yrashk well
[10:47] pieterh naming is always delicate
[10:47] yrashk yes and no
[10:47] pieterh imagine ezmq really takes off (it's lovely and fast)
[10:47] pieterh you'll have confused users asking, for years, why it's 'ezmq' and not 'erlzmq'...
[10:47] pieterh just my 5c
[10:48] eyecue i can vouch for that already :D
[10:48] yrashk I'll take care of this later
[10:48] yrashk I am just too lazy to think about this now
[10:48] pieterh lol
[10:48] yrashk sipping beer feels better
[10:48] yrashk I just hate renaming all files and module names
[10:49] eyecue pieterh; are you guys talking about various iterations/forks of the same essential codebase implementing various new structures/patterns/paradigms, or something entirely different?
[10:49] pieterh yrashk: it doesn't get easier over time, indeed the name will stick
[10:49] eyecue and subsequently what to name the resulting output codebase?
[10:49] pieterh eyecue: it's a rewrite, as yrashk said
[10:49] yrashk ya from scratch
[10:50] pieterh but it exposes the same API to applications, so from users' point of view it's a Version 2
[10:50] eyecue whats the purpose /goal / driving motivator behind it ?
[10:50] yrashk pieterh: not the same
[10:50] yrashk pieterh: it's slightly different
[10:50] eyecue internal extensibility, or abstracting away upgrades behind the scenes?
[10:50] pieterh yrashk: sure
[10:55] yrashk this is ezmq results with fixed perf tests on my mac pro
[10:55] yrashk message size: 1 [B]
[10:55] yrashk message count: 1000000
[10:55] yrashk mean throughput: 1330266.625339717 [msg/s]
[10:55] yrashk mean throughput: 10.642133002717735 [Mb/s]
[10:56] pieterh yrashk: you should do 10M messages, perhaps
[10:56] pieterh I love the 17-digit precision
[10:57] pieterh 1.3M msg/sec is pretty impressive
[10:57] eyecue certainly for sync tasks, id be keen to see the relative changes at incremental message sizes
[10:58] yrashk sure
[10:58] yrashk message size: 1 [B]
[10:58] yrashk message count: 10000000
[10:58] yrashk mean throughput: 1312724.6071802885 [msg/s]
[10:58] yrashk mean throughput: 10.501796857442308 [Mb/s]
[10:58] eyecue yrashk; ipc or?
[10:58] yrashk tcp
[10:58] eyecue local i take it ?
[10:58] yrashk ya
[10:58] eyecue sweet
[10:58] yrashk not bad
[10:59] yrashk jugg: ^^^
[10:59] eyecue whats the tcp overhead above ipc ?
[10:59] eyecue on the same 'box'
[10:59] yrashk I have no #s
[10:59] yrashk ah
[10:59] yrashk that
[10:59] eyecue quite interesting
[10:59] eyecue i can so tell im going to be spending alot more time on 0mq.
[11:00] yrashk never used ipc
[11:00] yrashk what would be an example ipc:// url?
[11:01] eyecue correct me if im wrong, but ipc:///tmp/somesocketname?
[11:02] eyecue first video on shows an example (i may be wrong)
[11:02] yrashk pieterh: on my hw c-to-c perf test is about twice as fast
[11:02] yrashk it makes me sad :-(
[11:03] pieterh yrashk: it's tragic
[11:03] yrashk this is on ipc:
[11:03] yrashk mean throughput: 1343429.6550005474 [msg/s]
[11:03] yrashk mean throughput: 10.747437240004379 [Mb/s]
[11:03] eyecue again, excuse what might be a silly question, c-to-c ?
[11:03] yrashk pretty much the same
[11:03] pieterh eyecue: yes, that would be a good place to put ipc files
[11:03] eyecue yrashk; thats pretty cool
[11:04] yrashk eyecue: perf tests from zeromq itself as opposed those implemented in erlang (which is what I am testing right now)
[11:04] pieterh ipc:// uses a filename, e.g. ipc://somefile.ipc
[11:04] yrashk pieterh: sarcasm? ;)
[11:04] pieterh must be writeable by all processes that use it
[11:04] eyecue yrashk; thats what i thought :)
[11:04] pieterh yrashk: gentle irony
[11:04] eyecue pieterh; i saw the uid note for it :]
[11:04] yrashk pieterh: do you think this is a good result?
[11:04] pieterh yrashk: rather good, yes
[11:05] eyecue i find it interesting that an ipc socket and tcp perf is roughly the same
[11:05] pieterh eyecue: they are identical, on Linux localhost
[11:05] pieterh local domain sockets
[11:05] yrashk it's osx
[11:06] pieterh osx is linux, it just hasn't quite realized it yet
[11:06] pieterh :-)
[11:06] eyecue still the file socket stack and tcp stacks often have quite not-the-same performance characteristics, and tuning parameters
[11:06] pieterh yrashk: you could try inproc to see what the raw API costs are
[11:06] yrashk pieterh: for that I have to tweak tests... not in this beer mode :)
[11:07] pieterh ah, beer mode... :-)
[11:07] pieterh i'm in coffee mode here
[11:08] eyecue i wish i was in coffee mode, food search mode here :)
[11:08] eyecue <-- .au
[11:09] sejo quick question, if I create a FIFO que with 2 servers and multiple clients, is it possible to save the items in the que somewhere to make sure when the servers crash the items are retrievable?
[11:09] sejo or should I write such code myself?
[11:10] pieterh sejo: it's being explained in Chapter 4 of the guide
[11:10] eyecue workers can die too, so resilience there can often be more important than queue resilience
[11:10] pieterh see
[11:10] pieterh I'm working on an implementation of that right now
[11:10] eyecue ooo, a new page to look at
[11:11] eyecue pieterh; why the name majordomo?
[11:11] Guthur pieterh: what stage of completeness is Ch4? out of interest
[11:11] pieterh eyecue: all the patterns in Ch4 get cute names
[11:11] pieterh and Ch4 is about 50% done afaics
[11:11] eyecue not worried about the open source app by the same name ? :]
[11:11] pieterh Majordomo because it provides a reliable service
[11:11] eyecue or are you leveraging it ;)
[11:11] pieterh what other app?
[11:11] eyecue majordomo the mailing list manager :]
[11:11] pieterh no idea what you're talking about
[11:12] eyecue one of the most widely used open source mailing list management softwares :]
[11:12] pieterh does it use 0MQ?
[11:12] eyecue nono, its name is just majordomo
[11:13] pieterh do they have a trademark on their name?
[11:13] Guthur I'd guess no
[11:13] eyecue greatcircle software is the company who makes it, but ive never thought of that :)
[11:13] pieterh they don't even own
[11:13] ianbarber you're probably all right on this one, it's more of a protocol thing than a particular bit of software
[11:13] Guthur wiki has this on the majordomo page: Development status End of life
[11:13] pieterh legally, if there is a trademark and risk of confusion, I'd have to choose another name
[11:14] ianbarber wordpress inspired the ire of many recently by naming their most recent release django, which was clever
[11:14] eyecue yah it was more out of interest than to point out any/possible risks :]
[11:14] Guthur last stable release 11 years ago
[11:14] pieterh morally, this is a 0MQ pattern name, there is no confusion
[11:14] Guthur it's possible it is perfect though
[11:14] pieterh like Libero
[11:14] eyecue there is of course the issue of user confusion, but i digress
[11:15] pieterh Libero is a FSM tool I wrote in the early 90's, which actually had no bugs and no more features to add
[11:15] pieterh so perhaps Majordomo-the-software was perfect, but ... probably it's just dead
[11:15] Guthur it can even run on newer versions of Unix, apparently
[11:15] eyecue its still one of the most widely deployed mailing list managers for open source projects full stop :]
[11:15] Guthur all this from the wiki page mind you
[11:15] Guthur can/can't
[11:15] eyecue im actually quite suprised youd never heard of it
[11:16] pieterh aiksaurus suggests 'chamberlain' as an alternative but somehow I prefer 'Majordomo'
[11:16] eyecue it sounds authoritive :)
[11:37] pieterh ianbarber: so we're on for the 16th?
[11:38] ianbarber yep
[11:40] ianbarber just tweeted about it, i haven't actually been to this place before, but mikko knows it
[11:41] ianbarber it's near oxford circus so it's just straight down the victoria line for kings cross, if you can come, and come in by train
[11:45] ianbarber will be awesome! do you have any clients in London to see or similar, feel bad for your travel if you're just coming to chat to us :)
[11:58] pieterh ianbarber: yes, there are always people to meet in London
[11:59] yrashk night everybody :)
[11:59] ianbarber :)
[11:59] pieterh yrashk: night :-)
[12:12] eyecue night
[12:21] private_meta haven't been in London for 3 years
[12:45] pieterh ianbarber: do you want a link shortener at I've set this up at
[12:46] ianbarber oh, that's an idea
[12:46] ianbarber have to chuck canonical links in heads of pages
[12:47] ianbarber that looks very handy
[12:47] pieterh register at Wikidot if not done, click Join and I'll give you access
[12:49] ianbarber done
[12:52] pieterh done: to create a shortcut you just enter the short URL you want, then click Create, and enter the target URL in the page title and save
[12:52] pieterh ianbarber: I've made you site admin too, so you can give others ccess
[12:52] pieterh *access
[12:53] sejo ping
[12:53] sejo sorry
[12:53] pieterh ianbarber: yay :-) it works
[12:53] ianbarber cool :)
[12:53] ianbarber thanks!
[12:55] pieterh hey, thanks to you for, it's quite a fun toy
[12:57] pieterh have to leave, cyal
[13:44] private_meta pieterh: in the majordomo protocol, why does the client have a nullbyte?
[13:45] private_meta pieterh: is it because the REQ socket normally inserts it?
[14:05] guyvdb Hi, I am just starting to read about zeromq. Is the socket event driven behind the scenes? I.e. if I do zmq_bind() on a server and then 10K low traffic clients do zmq_connect() is the library doing epoll() or similar in the background?
[14:45] djc anyone around? I could use some help with my first zmq thingy
[14:45] drbobbeaty djc: I'm no expert, but you can ask, and I can see if I can help.
[14:46] djc I'm trying to do a pub/sub thing
[14:46] djc but I'm not getting any messages on the client
[14:46] djc (the zmq_send() call on the publisher returns 0, so that seems to work at least)
[14:46] drbobbeaty Have you looked at the Guide? It's got PUB/SUB examples...
[14:46] djc the client just sits there, apparently blocking
[14:46] djc yeah, I'm looking at it
[14:46] djc that's what I derived my code from :)
[14:47] drbobbeaty The client's recv() will block - that's by design.
[14:47] drbobbeaty What transport are you using? The URL used by the PUB and SUB to know where to PUB and SUB.
[14:47] guido_g djc: paste your code (and any error message) to a paste bin, w/o the code we're guessing
[14:49] guido_g <- nice paste bin
[14:50] guido_g oops <- take this
[14:50] djc
[14:50] djc is what I have so far
[14:51] guido_g tz tz tz
[14:51] guido_g you need to subscribe on the sub
[14:51] guido_g see manual and guide
[14:52] djc oops, missed that
[14:56] guido_g case closed, i assume ,)
[14:56] djc it works! \o/
[14:56] djc thanks :)
[14:56] djc (and sorry for not reading better)
[14:57] djc btw, kind of weird that the python bindings don't respect SIGINT
[14:57] djc oh, I guess it respects it only when it awakes or something
[14:57] guido_g depends on the version of the bindings *and* ømq
[14:58] guido_g with pyzma from git master and 2.1.0 ømq ctrl-c works, w/ older ømq not
[14:59] djc yeah, I just picked up because it's on the py bindings page
[15:00] guido_g way to old
[15:00] guido_g if you dare to compile ømq yourself, use the git:// repository I'd say
[15:01] guido_g this will be the "stable" for 2.1.x and is now "release candidate" or so
[15:02] djc well, it doesn't look like 2.1.x is "stable" yet
[15:03] guido_g it is more stable than 2.0.x ever was
[15:04] guido_g and if you start using ømq there is no need to use the old version
[15:04] guido_g i'm using the old 2.1.0 tar and it's fine w/ pyzmq
[15:06] cremes sustrik: what is the use-case for setting ZMQ_MAXMSGSIZE to 0? your changes allow that but i don't see the point
[15:39] CIA-21 zeromq2: 03Martin Sustrik 07master * r18b9ebe 10/ (151 files in 9 dirs):
[15:39] CIA-21 zeromq2: The copyrights in file headers updated.
[15:39] CIA-21 zeromq2: Signed-off-by: Martin Sustrik <> -
[15:47] private_meta pieterh: does zmsg.c actually compile for you?
[15:50] cremes private_meta: i can't get it to compile on osx; it appears to compile ok on linux
[15:50] private_meta I'm trying to compile it on linux, cremes
[15:51] cremes oh
[15:51] private_meta for every function in there I get a "multiple definition" error
[15:51] private_meta cremes: what's your problem on OSX?
[15:52] cremes
[15:54] private_meta actually, "gcc -o file -lzmq ppqueue.c zmsg.c" SHOULD be sufficient, shouldn't it?
[15:57] private_meta cremes: hmm... I get the same result when compiling asyncsrv.c
[15:58] private_meta cremes:
[15:58] private_meta cremes: what's your compile command?
[16:00] guido_g seems that the errors are in the link phase
[16:01] private_meta Seems so
[16:01] guido_g try to compile and link step-by-step
[16:01] private_meta phew... no idea how...
[16:02] private_meta k, -c
[16:02] guido_g compile the source files to .o then link these .o files to an executable using the relevant libs
[16:03] private_meta yeah, I know the steps, but I just never did them manually
[16:03] private_meta I mean, I hardly ever used gcc by hand to start
[16:05] private_meta guido_g: it compiled correctly, but the linker had the same error
[16:07] guido_g seems that all functions are already defined in the asyncsrv file
[16:07] guido_g check the includes i'd say
[16:09] private_meta Well, what makes it break is zmsg.c
[16:09] private_meta but apparently it should work
[16:09] cremes my compile command is "./build all"
[16:09] cremes :)
[16:09] guido_g *sigh*
[16:10] guido_g private_meta: did you really read the error message? it states that the functions are already there, comming from the async file
[16:10] guido_g which is strange, becase they also appear in the zmsg file
[16:12] private_meta I took it as is from the website. asyncsrv.c includes zhelpers and zmsg, zmsg itself includes zhelpers
[16:12] guido_g it includes zmsg.c?
[16:13] private_meta Don't ask me, it's treated as a .h file
[16:13] guido_g link to code
[16:13] private_meta <-- zhelpers.h, zmsg.c, asyncsrv.c
[16:13] private_meta hmm
[16:14] guido_g oh my god
[16:14] private_meta wait,I took the non-branch one
[16:14] private_meta the master one
[16:14] private_meta What?
[16:15] guido_g it really includes a .c file
[16:15] guido_g one should poke pieter for that
[16:15] guido_g private_meta: leave out the zmq.* from the compile/link
[16:16] private_meta guido_g: ?
[16:16] guido_g what?
[16:16] private_meta what do you mean leave it out?
[16:16] guido_g drop it from the command
[16:17] private_meta oh
[16:17] private_meta >_<
[16:17] guido_g line noise?
[16:17] private_meta ok, then I need to set the file as "exclude from build" in eclipse for it to work
[16:18] guido_g in the 21st century?
[16:18] private_meta guido_g: k, thanks, stupid of me not to think of that
[16:19] guido_g also not very clever to *do that* in the first place
[16:19] private_meta ik
[16:19] guido_g you should tell pieterh and demand action
[16:21] cremes alternately, fix it and submit a patch
[16:23] private_meta I'm more interested in a C++ version
[16:24] private_meta But then everything should be a proper class and the problem would be gone
[16:25] guido_g zhelper thingie is for c
[16:26] guido_g you should use the c++ api and build from there, otherwise you put a layer on top of a layer...
[16:26] private_meta IK
[16:26] private_meta Doing that already
[16:26] private_meta In other words, I never intended to use zhelpers.c, but I need it for comparison
[16:27] guido_g compare to what?
[16:27] private_meta zhelpers.h to zhelpers.hpp equivalents
[16:28] private_meta If I use zmsg and translate it, I should do it in a way it's useful for the docs
[16:28] guido_g i don't see any benefit
[16:28] private_meta anyway, what I want to say is I won't use zhelpers.h for C++
[16:29] guido_g a useful c++ helper collection would not look like the c version, i'd guess
[18:29] pieterh re
[18:30] pieterh private_meta: you still around?
[18:49] pieterh guido_g: you around?
[18:50] guido_g jepp, now
[18:50] pieterh so what's the beef with my C coding style?
[18:50] pieterh :-)
[18:51] pieterh the idea is to allow people to compile and link individual examples
[18:51] guido_g you're including a a c file
[18:51] pieterh rather than have to construct libraries
[18:52] guido_g but obviously the normal reflex of a c programmer is to compile c files
[18:52] pieterh there's even a 'build' script provided
[18:52] pieterh yeah
[18:52] pieterh anyhow, I'm renaming these mystery classes to .h
[18:52] guido_g would help
[18:52] pieterh it's either write 2,000 line examples, or include stuff
[18:53] guido_g i think back in the old times one would have used .i or so
[18:53] pieterh would that be clearer?
[18:53] guido_g imho yes
[18:53] guido_g .c is realy ment to be compiled
[18:53] pieterh in 2011 we're allowed as many as 3 chars per extension afaik
[18:53] pieterh .dnc?
[18:53] guido_g .h is for declarations
[18:54] pieterh zmsg.h is a class file in face
[18:54] pieterh in *fact
[18:54] pieterh a mix of declarations and methods
[18:54] pieterh aha
[18:54] guido_g sure, but it's c++ then
[18:54] pieterh #include "zmsg.class"
[18:54] pieterh yes :-)
[18:54] pieterh no, it's class-oriented C
[18:54] pieterh C++ did not take patents on that
[18:55] guido_g the only problem was to use an extension that's been assocciated w/ a specific action
[18:55] pieterh yes, you're right, makefiles and such get very confused
[18:55] pieterh people will ask me wtf the class file business is, and I can then upsell ZFL
[18:55] pieterh very neat, thanks!
[18:56] guido_g np
[18:57] pieterh this makes my evening, in fact
[18:58] pieterh I love it when a semantic comes together
[19:00] guido_g i'm still touched by the idea of including code this way
[19:00] guido_g seems odd after some decades ,)
[19:03] pieterh well, however you do it, you need to say "I want to use class XYZ"
[19:03] guido_g sure
[19:04] pieterh if I break the classes into .h and .c I *force* people to make libraries and maintain them
[19:04] pieterh this way, they can build just the single example and it works
[19:04] private_meta pieterh: what is it?
[19:04] guido_g i finally figured it from the error messages private_meta got while linking
[19:04] pieterh well, people who are smart enough to go make their own compile / link commands instead of using the ones I provided...
[19:04] pieterh 'build all'
[19:04] pieterh 'build asyncsrc'
[19:05] guido_g this use of import has become uncommon in the last decades, that's all I wanted to mention ,)
[19:05] guido_g s/import/include/
[19:05] guido_g curse you, python! :)
[19:06] pieterh Python uses import, right?
[19:06] guido_g yes
[19:06] private_meta pieterh: what did you want?
[19:06] guido_g but it's not like include
[19:06] pieterh even for basic stuff like command line arguments, which I find very quaint, as a modern C programmer
[19:06] guido_g import deals w/ namespaces
[19:07] guido_g so it's more like a combination of include w/ c++ namespaces
[19:07] guido_g w/ some dynmaic linking...
[19:07] guido_g ok
[19:07] guido_g in fact, it's not like include
[19:08] Guthur are messages beginning with 0 reserved for ZMQ, or am I imagining things?
[19:09] Guthur maybe it was identities
[19:09] guido_g the latter, afair
[19:10] guido_g ømq generated identities are starting w/ a 0 byte
[19:10] Guthur ok
[19:11] Guthur at least I wasn't completely imagining things, hehe
[19:11] guido_g yeah
[19:11] guido_g it's amazing how much information is in the so sparse looking documentation
[19:12] cremes pieterh: any particular roadmap or timeframe until 2.1.1 is blessed as a final release?
[19:12] cremes reason i ask is we still have so many folks popping in here asking questions about 2.0.10
[19:12] pieterh ... phone... busy
[19:12] cremes because it is the last "official" release
[19:12] cremes sure, answer whenever
[19:13] guido_g true
[19:14] guido_g and it's somewhat hard to tell people to use a "release candidate" because it's more stable than the "stable"
[19:14] cremes agreed
[19:14] private_meta pieterh: Btw I'm implementing zmsg as a class, I hope that helps with the c++ examples, I just didn't finish yet
[19:15] private_meta c++ class that is
[19:29] pieterh ok, back...
[19:29] pieterh private_meta: zmsg is IMO useful in any language
[19:29] pieterh presumably modified to suit
[19:29] pieterh what it does is let you manipulate whole 0MQ multipart messages as single objects
[19:30] pieterh this is extremely useful unless you like writing repetitive code
[19:30] pieterh cremes: wrt 2.1.x, it's ASAP
[19:30] pieterh I'd like to make an rc2 this week
[19:30] pieterh and then it's probably 7-10 more days for a final stable release
[19:32] private_meta pieterh: I'm just saying I'll write a nice c++ class based version , more or Less
[19:32] pieterh IMO it will be useful in general for C++ developers
[19:33] cremes pieterh: great, i like that timeline
[19:33] cremes only 2 more weeks of questions on 2.0.10 :)
[19:33] pieterh cremes: my goal is to make new stable releases every 3 months or so
[19:33] pieterh perhaps even faster
[19:34] guido_g hmmm
[19:34] pieterh but not every six months like before
[19:34] pieterh s/six/24/
[19:34] cremes good idea; i've been "living" out of master for the past 3 months
[19:34] guido_g any news on the (e)pgm problem i reported?
[19:35] pieterh guido_g: it wasn't this one?
[19:35] guido_g no idea
[19:36] guido_g i was just going to compile from zerom2-1
[19:36] guido_g is this fix in there?
[19:37] private_meta pieterh, how's Majordomo coming along?
[19:37] pieterh private_meta: quickly, I have client and worker APIs, and heartbeating in the broker
[19:37] pieterh it's all on that chapter4-wip branch that you should not be looking at
[19:38] private_meta I've looked through it but thought as this is much WIP, should wait
[19:39] pieterh guido_g: that fix is in the zeromq2-1 git master
[19:40] pieterh we've been backporting all bug fixes from 2.2 to 2.1
[19:40] guido_g ok, just compiled it
[19:41] guido_g ahhh.. great!
[19:41] guido_g multicast is back
[19:41] pieterh excellent!
[19:42] private_meta hmm... "Developing a port was quite difficult as it took us guile long to port all the bugs correctly"
[19:42] private_meta -guilt + quite
[20:30] guido_g pieterh: care to explain wich application would need a 10ms heartbeat?
[20:30] pieterh anything far away
[20:30] guido_g just reading ch4 of the guide
[20:30] pieterh it's strictly dependent on network architecture
[20:30] pieterh sorry!
[20:30] pieterh :) I read "would need more than 10ms"
[20:30] guido_g ok
[20:30] pieterh anything really close and extremely low latency
[20:31] guido_g but...
[20:31] guido_g heratbeat is going over the same socket as the data
[20:31] pieterh if you're getting 1M messages a second and suddenly there's silence for 10msec... that's serious
[20:31] guido_g ack
[20:31] pieterh you want to know after maybe 1msec
[20:31] pieterh and then switch to a backup
[20:31] guido_g but you don't send heartbeats when sending data
[20:31] pieterh precisely
[20:32] guido_g so a received data packet counts as "alive"
[20:32] pieterh to do this optimally, you create a slope
[20:32] guido_g ack
[20:32] pieterh so heartbeat immediately as data stops, then double the time until it's infinite
[20:32] guido_g done this in prototype of a rmc protocol
[20:32] pieterh but that's a lot of work for an example and overkill for 99% of cases
[20:33] guido_g sure
[20:33] pieterh I'd suspect the minimum sane heartbeating is around 1msec for low latency apps
[20:33] pieterh that's a typical SLA
[20:33] guido_g ok
[20:34] pieterh C99 just became C2011
[20:34] guido_g but still the sender should only send a marker (heartbeat) when there is no data to send
[20:35] guido_g should go into spec7, somnehow
[20:35] guido_g *somehow
[20:35] mikko hello
[20:36] guido_g hiho
[20:38] pieterh guido_g: indeed, thanks
[20:39] guido_g its a nice read, btw
[20:39] mikko i've been at a customer lately and no irc during the day
[20:39] guido_g overall i like the idea of sketching these things in this semi-formal form
[20:40] pieterh guido_g: it makes life surprisingly easy
[20:40] pieterh sketch the protocol, implement it
[20:41] guido_g no, not w/ six bytes for the protocol version ,)
[20:41] pieterh what do you mean?
[20:41] guido_g "Frame 0: "MDPW01" (six bytes, representing MDP/Worker v0.1)" <- that
[20:42] guido_g it's even in the heartbeat
[20:42] pieterh yes
[20:42] pieterh there's a good reason for that
[20:42] guido_g i mean, we're talking about the machine where 1byte messages are use for benchmarks, right?
[20:42] pieterh I want to make brokers that can handle any mix of protocols over a single socket
[20:43] pieterh request-reply is *not* performance sensitive
[20:43] pieterh at least not at the 6M msg/second level
[20:43] guido_g depend on the amount of packets send on the segment, right?
[20:43] pieterh use cases are key
[20:44] pieterh data distribution -> millions / second, microsecond latency
[20:44] pieterh task distribution -> thousands / second, millisecond latency
[20:44] guido_g sure
[20:44] guido_g but still, 6 bytes for a protocol version?
[20:45] pieterh didn't hurt HTTP
[20:45] pieterh it should be printable, unique, and include a version number
[20:45] pieterh 'M' is not a protocol name
[20:45] guido_g ah
[20:45] pieterh seriously, this is not worth optimizing, not even marginally
[20:46] guido_g so the command doesn't need to be printable then?
[20:46] pieterh the payload is opaque binary
[20:46] pieterh the command... good suggestion
[20:46] pieterh nope
[20:46] pieterh identities aren't either
[20:46] guido_g i'd say 2 bytes for proto version would be more then enough
[20:47] guido_g how many incarnations could one expect?
[20:47] pieterh it is two bytes for proto version...
[20:47] pieterh MDPSxx
[20:47] guido_g *sigh*
[20:47] pieterh proto name = 3 chars, proto subname = 1 char, proto version = 2 chars
[20:47] Guthur pieterh, do you have a timeframe on when you hope to finished Ch4?
[20:48] pieterh Guthur: depends how long guido_g keeps nagging about these six bytes...
[20:48] Guthur hehe
[20:48] pieterh :-)
[20:48] guido_g pfff...
[20:48] cremes guido_g: i mentioned the same thing when i reviewed an earlier version of the rfc
[20:48] guido_g ,)
[20:48] cremes guido_g: i agree though that it isn't worth arguing about
[20:48] guido_g sure
[20:49] pieterh look...
[20:49] pieterh the mdbroker opens a single socket for both clients and workers
[20:49] guido_g ack
[20:49] pieterh that is much easier for admin than two endpoints
[20:49] guido_g agreed
[20:49] pieterh i want to write a couple more protocols
[20:49] pieterh there'll be one for persistence (to disk)
[20:49] guido_g dozens?
[20:49] guido_g hunderts?
[20:49] pieterh nope, a few
[20:50] pieterh but that's just me
[20:50] pieterh i assume there will be variations on MDP
[20:50] pieterh not versions but variations
[20:50] guido_g we do have the message-parts
[20:50] guido_g so lets start small
[20:50] pieterh say you want to experiment with a new command
[20:50] guido_g like ip
[20:51] pieterh you can fork spec:7 and create a new spec:8
[20:51] pieterh but you will choose a new name
[20:51] guido_g would be more 7.1 :)
[20:51] pieterh so yes, there can be dozens or hundreds of protocols
[20:51] pieterh nope, not 7.1, read COSS if you care
[20:51] guido_g yes, the guttenberg-protocol
[20:51] pieterh each spec is independent, has its own editor, is forkable
[20:51] guido_g because it's mostly copied
[20:52] pieterh why do I get link errors on 0MQ of the style crti.o: No such file or directory
[20:52] pieterh (new Ubuntu install)
[20:53] guido_g compile and link use different versions?
[20:53] pieterh could be
[20:53] guido_g or some packages not properly installed
[20:54] pieterh I just swapped my notebook HDD for an SSD, copied a bunch of stuff across
[20:54] pieterh so probably mix of old/new code
[20:54] pieterh np
[20:54] pieterh look, if you don't like the 6-byte header, you fork MDP and create your own new spec
[20:54] guido_g or you didn't install c++ properly (and/or in the correct version)
[20:54] pieterh (that is, if I don't accept your suggestions)
[20:54] guido_g no, not for this one change
[20:55] pieterh it could be improved by adding a more formal start-up handshake
[20:55] guido_g btw, ssd in laptop rocks :)
[20:55] pieterh yeah, unbreakable...
[20:55] guido_g or smaller headers and the possibility for extensions
[20:56] guido_g like the ip header extensions
[20:56] pieterh bleh... it's a draft protocol, it has to be _simple_ above all
[20:56] guido_g sure
[20:56] pieterh it is counter-productive to optimize a draft, pathological even
[20:56] guido_g but talking (and sometimes even thinking) about it helps
[20:57] pieterh clients and workers would have to 'sign in' and specify a protocol
[20:57] guido_g or more then one
[20:57] pieterh then every message would first require a lookup to determine, "ah, this peer is using protocol XYZ"
[20:57] pieterh and then you could remove the protocol header from every command except the first
[20:57] pieterh you'd save 6 bytes on requests of 1K and responses of 20K
[20:58] pieterh the world would rejoice
[20:58] guido_g if workers stick to one protocol yes
[20:58] pieterh and every single implementor would get that handshake wrong at least once
[20:58] guido_g this is one of those assuptions i generally don't like
[20:59] pieterh so spec:7 makes as few assumptions as it can
[20:59] guido_g they lead to a "cover foreign asses" mentality
[20:59] pieterh i'm not sure we have a good pattern for a connected dialog, yet
[20:59] pieterh when we do, this will become much easier
[21:02] guido_g i think spec:7 is a nice starting point
[21:02] guido_g i'll try to implemnt it w/ pyzmq
[21:03] pieterh guido_g: if you take the chapter4-wip branch, there are already working client & worker APIs in mdcliapi and mdwrkapi
[21:03] pieterh they expose simple classes to applications
[21:04] guido_g uhhh
[21:04] mikko pieterh: so you are coming on 16th?
[21:04] guido_g pyzmq makes this even more simple
[21:04] pieterh mikko: yes
[21:04] pieterh guido_g: how so?
[21:04] pieterh mikko: I'll be there earlier
[21:05] guido_g it has a eventloop built in, including timers etc.
[21:06] mikko pieterh: how much earlier?
[21:06] mikko pieterh: i work very close to oxford circus
[21:06] pieterh mikko: I can be there (in London) at lunchtime
[21:07] pieterh might meet some other people for lunch, or before
[21:07] mikko pieterh: are you having any meetings during the day?
[21:07] pieterh you want to catch up?
[21:07] pieterh happy to come and evangelize 0MQ
[21:07] pieterh guido_g: the complete worker API, in C, is 200 lines of code
[21:07] mikko maybe, not sure about my schedule yet
[21:08] guido_g pieterh: <- less then 50 lines w/ this
[21:08] pieterh guido_g: of which maybe 30 lines is the central poll loop & timers
[21:10] pieterh guido_g: yes, I'm sure it can be even less than 200 lines, maybe 180 or 174, but that's not fair to readers who are learning 0MQ core, rather than language-specific frameworks
[21:10] mikko pieterh: but it would be nice to catch up
[21:10] pieterh mikko: I'll keep some time free in the afternoon, no problem
[21:10] mikko i can ask if devs at the company are interested in hearing about zeromq
[21:10] mikko from horse's mouth
[21:10] pieterh :-)
[21:11] mikko some of them saw ianbarber last friday
[21:11] guido_g pieterh: agreed, but we'll see more bindings with additional functionality i assume
[21:11] pieterh guido_g: any useful pattern should be documented in a language neutral form
[21:11] guido_g anyway, i'm much more a python than a c programmer nowadays
[21:11] pieterh otherwise it's just fragmentation
[21:11] pieterh the language is not relevant, barely
[21:11] guido_g pieterh: ack, i was going for an implementation
[21:11] guido_g the spec:7 is fine
[21:12] pieterh guido_g: yeah, but it's a shame to not first translate the guide examples
[21:12] pieterh if you Pythonistos can hit 100%, you achieve an ianbarbarism
[21:12] pieterh that is, you get a Guide in Python
[21:12] pieterh but I guess it's too difficult... :-/ Python has a reputation as harder to use than PHP or C
[21:13] guido_g i can't avoid the feeling that some wants to talk me into doing something... but i can't put a finger on it...
[21:13] Guthur i really should finish ch3 for clrzmq2
[21:13] pieterh no, in fact I formally forbid you from even thinking about it...!
[21:13] Guthur I got lazy
[21:14] pieterh Guthur: guido_g: race on... C# vs Python! who will be first to get their own Guide???
[21:14] pieterh my money is on Perl, to be honest
[21:14] Guthur pieterh, you like your reverse physiology
[21:14] Guthur hehe
[21:14] mikko PHP won ?
[21:14] guido_g he has to train it, he has kids :)
[21:14] pieterh yup, mikko, Ian Barber won...
[21:15] pieterh
[21:15] Guthur my c# examples wouldn't really qualify, I had using blocks to avoid explicit disposal
[21:15] pieterh to be honest, it's pretty cool to see the examples in random other languages
[21:15] Guthur and other sugar
[21:16] pieterh Guthur: if the sugar makes it easier to write, use it
[21:17] pieterh after all, it's about teaching developers in language X how best to use the 0MQ binding/API
[21:17] Guthur yeah things like returning and accepting strings, instead of byte arrays
[21:18] Guthur even if it can't get its own guide it would be good experience for me with 0MQ
[21:18] Guthur the more the better I think, hehe
[21:18] pieterh One day, when Google human resources conduct their 10-minute screenings they will ask, "how would you rate your 0MQ knowledge, 0 to 10?"
[21:19] mikko haha
[21:19] guido_g who cares ]:->
[21:20] guido_g there are already a lot of python examples
[21:20] pieterh guido_g: yes, only a few left, your chance for eternal fame is slipping away fast
[21:22] Guthur hehe, it's sad but I actually liked seeing my name at the top of the guide
[21:22] Guthur my one bit of programming fame
[21:23] pieterh To be honest, being named as contributor to a successful open source project is better than a good CV
[21:23] guido_g depends on where you want to apply
[21:23] pieterh precisely
[21:24] guido_g my last gig was at a bank...
[21:24] pieterh i mean, any firm that does not google you and check you on github is going to be painful to work for
[21:24] guido_g makes approx. 99.99% of all companies :)
[21:25] guido_g and the others... well there not that much of them
[21:25] Guthur guido_g, Did you find that banks don't dig work on OSS
[21:25] pieterh the thing that eventually caused 0MQ started 6 years ago because someone was once impressed by the C library I wrote
[21:25] pieterh and it was a bank
[21:26] guido_g sure, there might be exceptions
[21:26] pieterh ok, I have a messaging broker to write before the sun rises or the vampires will get me
[21:26] pieterh cyal
[21:26] guido_g Guthur: dig work on? sorry, don't understand that
[21:27] guido_g pieterh: cu
[21:27] pieterh guido_g: ciao
[21:28] Guthur guido_g, rate approvingly
[21:29] guido_g i've been lucky, the bank i worked for used python, this is why i got the job
[21:30] guido_g but they didn't know about oss in general
[21:30] guido_g and now, neither the bank nor the headhunters did check my (non-existant) oss track record
[21:31] Guthur did you mention standout projects in your CV?
[21:31] guido_g i mostly worked w/ oss software in the last 12 years, so lots of names in my cv
[21:32] guido_g but they don't know even postgresql
[21:32] guido_g totaly closed shop (and minds)
[21:33] Guthur yeah that can be quite prevalent in corporate culture
[21:34] guido_g <- true :)
[21:40] dijix guido_g: sounds like my .NET shop.. I can't even carry on a conversation with most of them - they haven't even heard of things like Python.. and most assuredly, not ZeroMQ..
[21:40] guido_g dijix: not even that
[21:41] guido_g they run programs, that's it
[21:41] guido_g they do have 2 or 3 people w/ some technical know-how
[21:42] guido_g but given the fact that of the most important pieces of software there is still written in cobol...
[21:42] eyecue *yawn*
[21:42] eyecue morning
[21:42] guido_g my python gig there was ok
[21:43] guido_g howdy
[21:43] eyecue :)
[21:43] eyecue 0mq kinda blew me away last night, i got home at 6:30 after work, and before i knew it had been reading for a good 5 hours
[21:44] guido_g i'd treat that as a good start
[21:44] eyecue yeh :)
[21:54] cremes anyone know why i would get errno=2 and error_string "No such file or directory" when trying to connect using inproc?
[21:57] sustrik strange
[21:57] sustrik do you have SWAP set?
[21:59] cremes no
[22:00] cremes will it report that error if i bind/connect two sockets within the same context?
[22:00] cremes btw, syntax is: "inproc://some_name" right?
[22:02] sustrik right
[22:02] sustrik the errno looks to be bogus
[22:02] sustrik is it reproducible?
[22:02] cremes it might be a problem with my binding... investigating <sigh>
[22:03] sustrik cremes: win32?
[22:04] cremes no, linux
[22:04] sustrik hm
[22:04] sustrik no idea then
[22:06] cremes now i'm getting: errno [111], error_string [Connection refused]
[22:18] Guthur cremes, which binding is that out of interest
[22:19] cremes ruby
[22:19] cremes looks like it isn't a binding bug... 0mq throws that error if you try to zmq_connect() an inproc before yu zmq_bind() it
[22:20] guido_g ahhh
[22:26] mikko cremes: thats a known limitation
[22:27] cremes mikko: yeah, but i thought it failed silently
[22:27] cremes it now returns -1.... which is new to 2.1.1 i think
[22:29] guido_g effect of an assert that got removed?
[22:37] guido_g oh late already, night all!
[22:42] pieterh cremes: afaik it always returned a -1
[23:20] JoderCoder Hi everyone, I am trying to write some IPC code that can pass data between 2 processes on a single machine. Am I at right place? :)
[23:24] Guthur JoderCoder, You can certainly use ZeroMQ to do IPC
[23:24] Guthur what platform?
[23:28] JoderCoder Guthur, Linux initially, then Windows..
[23:29] JoderCoder
[23:29] Guthur you can use IPC transport on Linux but on Windows you would need to use TCP
[23:29] JoderCoder this is the mail I sent to the forum
[23:29] Guthur IPC via named pipes on windows is not yet available
[23:29] JoderCoder yes, I came across with that note on the docs somewhere
[23:30] JoderCoder Linux is my priority anyways..
[23:30] JoderCoder is there any sample code available somewhere that I could re-use?
[23:31] Guthur the guide is a code place to start
[23:31] pieterh JoderCoder: have you read the Guide?
[23:31] Guthur code/good
[23:31] Guthur hehe, too much coding in the brain
[23:32] JoderCoder pieterh, I did read it, to some extend. Firstly, I am not sure what pattern to use.
[23:32] pieterh JoderCoder: read it again, until it makes sense
[23:32] JoderCoder I was recommended to use Router-to-Router
[23:33] pieterh this would be your first 0MQ program?
[23:33] JoderCoder yes
[23:33] pieterh so please start at Ch1 not Ch3
[23:33] pieterh do take the time to learn this properly
[23:33] pieterh otherwise you're going to get lost
[23:33] pieterh walk, then run
[23:33] JoderCoder :-)
[23:36] JoderCoder well, I guess I am not the first one, but under some pressure due to the tight schedule.. but you're right.. I will read it tomorrow.
[23:36] pieterh if you want to save time, hire someone who already knows the technology well
[23:36] JoderCoder :-)
[23:36] pieterh but jumping into deep water before you can swim will really not save you time
[23:37] pieterh it's about 2-3 days to learn the basics and feel them 'click' in your mind
[23:37] pieterh you will save that easily on the first app you make properly
[23:37] Guthur JoderCoder, what language are you going to use, out of interest
[23:37] JoderCoder I will replace my current Sys V MQ with ZMQ since we plan to use it as a MQ across network
[23:37] JoderCoder c++
[23:40] JoderCoder OK guys, thanks for your time.. I'll get to it tomorrow morning. g'night
[23:40] pieterh JoderCoder: cyat
[23:40] Guthur night JoderCoder
[23:40] Guthur it's worth reading the guide
[23:41] Guthur lots of idea
[23:41] Guthur ideas*
[23:41] JoderCoder ok danke schon
[23:42] Guthur your welcome, i must sleep now too
[23:56] jugg yrashk, pieterh: The only reason the csrl/erlzmq fork exists is because the zeromq/erlzmq bindings are incomplete/broken. I'd love to use yrashk/ezmq, but my application runs on R13B04 which can't run ezmq (requires R14). When I have time to move to R14, I'll most likely move to ezmq.
[23:56] jugg yrashk, nice perf results.
[23:56] pieterh jugg: makes sense, I'd advise you to agree on one name and two version numbers, for your users' sanity