Saturday April 16, 2011

[Time] NameMessage
[00:32] Toba danolj: that's an interesting problem. are you sure someone else hasn't already built exactly what you're looking for?
[00:44] danolj I am not sure - been googling for patterns / solutions
[00:49] danolj I suspect that an intermediate forwarder is likely with some kind of mechanism for discovery of all the publishing endpoints
[01:12] jsimmons why do they need to know all the end points danolj?
[01:12] jsimmons you can bind on either side of the connection. ie your subscriber can bind, and publishers connect
[03:21] danolj it is more about wanting "magic" to happen so that I don't have to keep configs updated across nodes
[03:21] danolj as to what all the end points are today
[03:22] danolj there are some things that spread offers in terms of service discovery over udp that I'd like to replicate
[03:22] danolj btw, thanks for the discussion!
[03:56] Toba danolj: well, there are tools that manage configs. lots of htem. you may be solving the wrong problem.
[05:02] danolj Toba: you may be correct. Thanks.
[05:02] Toba good luck, I'm going to bed
[06:41] guido_g danolj: you can either use a forwarder device or the (e)pgm transport
[06:41] guido_g with the forwarder device there is only one endpoint to be known (configured) for publishers and subscribers
[06:42] guido_g otoh, it's a spof
[06:42] guido_g or you use the (e)pgm, which is real multicast and you get want you want: a bus structure
[06:43] guido_g but multicast does have it's own set of probl^W implications :)
[09:30] sustrik mikko: are you there?
[09:30] sustrik any idea whether openpgm should work on osx?
[09:33] locklace it claims to support os x / amd64
[09:35] sustrik thanks
[09:35] sustrik there's an bug report about it
[09:37] mikko sustrik: i think it should
[09:37] mikko i can test it
[09:37] mikko i saw the report yesterday but was at the airport
[09:38] sustrik you have osx?
[09:38] mikko yes
[09:38] sustrik give it a try then
[09:39] mikko building
[09:41] mikko same result
[09:42] sustrik hm, let me check on linux
[09:42] mikko that struct doesn't seem to be defined on ox s
[09:42] mikko os x
[09:42] mikko based on grep -r /usr/include
[09:44] sustrik looks like an openpgm issue then
[09:45] mikko one for steve maybe
[09:45] mikko same thing happens on mingw32
[09:45] mikko as the struct is missing
[09:46] sustrik would you ping him?
[09:46] sustrik or should i?
[09:47] mikko i can ping him
[09:48] mikko he should be back today / tomorrow i think
[09:48] sustrik ok
[09:48] mikko he was travelling europe this week
[09:48] sustrik yes, i recall
[09:51] pieterh g'morning
[09:53] sustrik morning
[09:53] pieterh I made a rough proposal for libzutil, no response from the list
[09:54] sustrik what's the difference from zfl?
[09:54] pieterh target audience
[09:54] pieterh libzutil is aimed specifically at language bindings
[09:54] pieterh sits underneath libzapi, for example
[09:54] pieterh libzfl sits on top of libzapi and is specifically for C apps
[09:55] pieterh example use case for libzutil is how to make swap device accessible to Java apps
[09:56] sustrik i like this line:
[09:56] sustrik * Provide a set of clock and timer functions (which aren't otherwise portable).
[09:57] sustrik if the idea gets traction
[09:57] pieterh yes, you remember the old discussion about providing timers in the core library
[09:57] sustrik i can remove zmq_utils from libzmq
[09:57] pieterh yes
[09:57] pieterh I have these timer/clock functions in zapi already, would move them down a layer
[09:58] sustrik ok
[09:58] sustrik cyl
[09:58] pieterh cyl
[10:14] mikko hmm, i integrated zeromq to this project already
[10:14] pieterh mikko: which project?
[10:14] mikko needed to test coordinates of camera and noticed that zeromq is a lot easier than writing to file and copying it over
[10:14] mikko pieterh:
[10:15] pieterh lol
[10:15] mikko now i got coordinate data coming from kinect sensor and camera controlling daemon subscribes to that data
[10:15] mikko and moves the hardware camera based on it
[10:15] pieterh 0MQ is like a magic connectivity wand
[12:16] Tapan Hello!! Is anyone out there?
[12:17] mikko ye
[12:18] Tapan Thanks... I am seeking some help in installing zeromq on redhat ES 5.5 64 bit.
[12:18] mikko sure, do you want RPMS?
[12:18] Tapan i have used ./configure, make and make install successfully but i don't know how to start zeromq.
[12:18] mikko
[12:18] mikko theres rpms built on centos5
[12:18] Tapan Can you please suggest how should i start zeromq?
[12:18] mikko should work in EL5 as well
[12:19] mikko ahmm, what do you mean by starting zeromq?
[12:19] Tapan thanks for the rpms links.
[12:19] Tapan i means like zeromq -d -p 8000 something like this.
[12:20] mikko zeromq is not really a daemon
[12:20] Tapan Can you please suggest which version i should try as once sucussfully tested we want to install under production.
[12:20] mikko it's more of a library to build your own daemons
[12:20] Tapan Ohhhh
[12:20] Tapan ahmm
[12:21] Tapan I have seen one video on the zeromq site which shows how to connect with php.
[12:21] pieterh Tapan: how much of the Guide did you read yet?
[12:22] mikko Tapan: you mean ian barber's presentation?
[12:22] Tapan i am sorry, i have only few hours of understanding of zeromq yet.
[12:22] pieterh Tapan: it's best if you read first
[12:22] Tapan I think yes....
[12:24] Tapan Ok i will read the guide first and will come back with question if i have.....Thanks for the support....
[12:24] Tapan Must admit good community...good support....
[13:55] Tapan Hello! Is any out there?
[13:57] mikko yea
[13:59] Tapan I rad the guide and i am able to bind on perticuler port.
[13:59] Tapan my example code is
[13:59] Tapan <?php /* The server waits for messages from the client and echoes back the received message */ $server = new ZMQSocket(new ZMQContext(), ZMQ::SOCKET_REP); $server->bind("tcp://"); /* Loop receiving and echoing back */ while ($message = $server->recv()) { echo date("Y-m-d H:i:s")."-" ."$message\n"; /* echo back the message */ $server->send($message); } ?>
[14:00] Tapan Is anyone out there now?
[14:00] mikko yes
[14:00] Tapan Thx...
[14:00] Tapan I am also able to send message to queue like this
[14:01] Tapan <?php /* The client sends two messages using two different sockets and then exits */ $is_persistent = true; /* Create new queue object */ $queue = new ZMQSocket(new ZMQContext($is_persistent), ZMQ::SOCKET_REQ, "MySock1"); $queue->connect("tcp://"); echo "Start Time:-" . date("Y-m-d H:i:s") . "\n"; /* Assign socket 1 to the queue, send and receive */ $queue->send("hello there!"); echo "End Time:-" . date("Y-m-d H:
[14:01] Tapan Now i want to test how much throughput i am getting this code so that i can think of using in my production application.
[14:02] Tapan and my example code to test zeromq throughput is
[14:02] Tapan but after 1st request sent to socket it is throwing error.
[14:02] Tapan and the error is PHP Fatal error: Uncaught exception 'ZMQSocketException' with message 'Failed to send message: Operation cannot be accomplished in current state' in /var/www/html/zeromq/test.php:18 Stack trace: #0 /var/www/html/zeromq/test.php(18): ZMQSocket->send('hello there!-1') #1 {main} thrown in /var/www/html/zeromq/test.php on line 18
[14:03] Tapan is this persistance connection issue?
[14:03] mikko nope
[14:03] mikko with request - reply pattern you need to do things in sequence
[14:03] mikko request,reply,request,reply
[14:04] mikko Operation cannot be accomplished in current state means that you are not in the correct state
[14:05] Tapan can you please share very basic code with looping? so that i can check zeromq throughput.
[14:05] Tapan If you have...
[14:07] mikko just make sure that your server always does recv,send,recv,send and client does send,recv,send.recv
[14:08] Tapan Thanks Mikko
[14:08] Tapan I understood.
[14:08] Tapan now my script is like this and it is working...
[14:09] Tapan This community is really rocks and working for fresher...:-
[14:09] Tapan )
[14:10] Tapan Can you please suggest what is the maximum throughput which we can expect from zeromq?
[14:14] neopallium Tapan: See the through benchmark results on this page:
[14:18] neopallium Tapan: See the through benchmark results on this page:
[14:18] neopallium *throughput
[14:18] Tapan how much throughput we can expect from zeromq?
[14:18] Tapan any suggestion?
[14:18] Tapan thanks checking
[14:18] neopallium that page shows throughtput for Lua and C++ code.
[14:19] neopallium Tapan: also for throughput testing it is using PUB/SUB sockets not REQ/REP sockets like the code you posted.
[14:19] guido_g <- more on performance
[14:21] guido_g pieterh: there're no history links on the specs, makes checking the changes a bit hard
[14:22] pieterh guido_g: let me check...
[14:23] guido_g thx
[14:23] Tapan Yes indeed the sites shows huge (very good--No i should say great throughput) throughput.
[14:23] pieterh guido_g: if you reload the page you should see the links at the bottom. now
[14:24] guido_g pieterh: sorry, no change yet
[14:25] pieterh guido_g: may be cached for a while
[14:25] guido_g pieterh: right, will check again later, thanks!
[14:25] neopallium Tapan: If you are going to use zeromq from PHP you might want to port the latency & throughput benchmarks to PHP
[14:30] Tapan Sure...
[14:59] Tapan neopallium: How to run local_thr.cpp to check throughput of my installation?
[15:00] guido_g Tapan:
[15:01] guido_g pieterh: still no history link
[15:01] pieterh guido_g: I'll logout of wikidot and try it... hang on
[15:01] Tapan neopallium: And how should i check the throughput in php?
[15:01] pieterh guido_g: I see the link, bottom right
[15:01] guido_g pieterh: i'm logged in too, does this matter?
[15:02] pieterh guido_g: shouldn't matter
[15:02] guido_g pieterh: on which page do you check?
[15:03] guido_g mine is
[15:03] pieterh guido_g: well, it's helpful if we agree what page we're looking at, I'm at
[15:03] Tapan guido_g: I have already opened that link and when i firing the scritp i am getting error.
[15:03] Tapan [root@node03 perf]# ./local_thr.cpp tcp://eth0:5555 1 100000 ./local_thr.cpp: line 1: /bin: is a directory ./local_thr.cpp: line 2: syntax error near unexpected token `c' ./local_thr.cpp: line 2: ` Copyright (c) 2007-2011 iMatix Corporation' [root@node03 perf]#
[15:03] pieterh sorry, meant
[15:04] pieterh rfc is another wiki, I'll fix the permissions on that now
[15:05] pieterh guido_g: ok, try that now?
[15:05] guido_g ahhh....
[15:05] pieterh I missed where you said 'the specs'
[15:05] guido_g np
[15:07] guido_g there it is, but only if i'm logged in
[15:07] pieterh permissions are definitely correct now, must be a refresh issue
[15:08] guido_g as said, when i'm logged in the links are there
[15:08] pieterh I'm testing via a proxy browser (totally anonymous), links are there
[15:09] guido_g then my firefox is playing games on me :/
[15:10] guido_g opera is showing them when i'm not signed in anymore
[15:10] guido_g so everything ok, i assume
[15:10] guido_g thanks
[15:12] guido_g and all this because 4 words changed :)
[15:13] pieterh :)
[15:13] guido_g oh, this number 12 looks very interesting...
[15:13] pieterh it's the Clone pattern
[15:14] pieterh damn thing took weeks to get right
[15:14] pieterh actually the client-server part is easy, it's the reliable server failover that was nasty
[15:14] guido_g as usual
[15:14] pieterh as usual
[15:16] guido_g i mean, even ibm didn't get it right in the first (2nd, 3rd...) go
[15:16] guido_g i remember the fuzz when we tested that wllm thing
[15:17] pieterh ironically even when failover works, it's pretty much pointless
[15:17] pieterh better to have simpler boxes that break less often
[15:17] guido_g yes
[15:18] guido_g afair, they (dt. börse) dropped the intra-day service restart requirement after the tests
[15:19] pieterh lol
[15:19] guido_g agreed (now, back then it wasn't funny at all)
[15:20] pieterh infrastructure projects always seem fairly painful
[15:20] guido_g i wish i could show you the communication diagram we all lived of
[15:20] guido_g the diagram was actually an accident
[15:21] guido_g it resulted out of a meeting where all programming groups were invited
[15:21] pieterh how long ago was this?
[15:21] jond guido_g: I assume yr referrring to dt borse optimise platform ... isnt it based on wllm?
[15:21] guido_g ack
[15:21] guido_g pieterh: 2009 and early 2010
[15:22] guido_g jond: it is
[15:22] guido_g pieterh: what? why?
[15:23] pieterh at the time 0MQ was a stalking horse to get IBM to lower their license prices
[15:23] jond guido_g: it's just gone live hasnt it?
[15:23] pieterh s/was/was used as/
[15:23] guido_g ic
[15:23] jond pieterh: did you know that at the time?
[15:24] pieterh jond: it became more or less obvious
[15:24] jond pieterh: depressing....
[15:25] guido_g the whole project was
[15:25] guido_g or is, i think they're still on it
[15:26] pieterh jond: it'd have been more depressing to spend another X years of ones' life trying to make upper management happy
[15:26] jond guido_g: but the press releases said 'best in class etc etc' ;-)
[15:27] guido_g jond: did you see the cialis ad next to it?
[15:27] guido_g more substance, really
[15:27] guido_g was my first project afer going freelance
[15:27] jond pieterh: too true, and many decisions appear to be political rather than technical
[15:28] guido_g except for optimise
[15:28] guido_g there was *no* technical decision ever
[15:28] jond guido_g: nope, what's cialis ?
[15:29] guido_g jond: go check your spam folder :)
[15:29] jond guido_g: lol
[15:30] guido_g jond: there once was a quote from a top-manager at dt. börse that the new trading paltorm was doing >1 million trades per second
[15:31] pieterh guido_g: sounds like they got their money's worth!
[15:31] guido_g jond: this was at a time where the perf tests needed to talk to tha backend services directy, because the gateway service either a) seegfaulted or b) processed at max. 5k msgs/sec
[15:32] guido_g but this time didn't last for long
[15:32] guido_g after even the top-managers realized the real performance, the perf tests were cancelled
[15:33] pieterh hopefully they fired the idiot who wrote such slow perf tests
[15:33] guido_g *sigh*
[15:33] jond guido_g: I can see I could have an interesting conversation around this area, but not on irc.
[15:34] pieterh what's the point for paying for perf tests that don't add a few zeros here and there... :-)
[15:34] pieterh guido_g: can't you come to the 0MQ Unconf?
[15:34] jond pieterh: I'm looking into it now
[15:34] guido_g jond: *sigh* yes, unfortunately
[15:34] pieterh jond: great!
[15:35] guido_g pieterh: what i've seen so far it'll be more about ømq development and process and such, nothing i'm interested in
[15:36] pieterh guido_g: what would you be interested in?
[15:36] pieterh I mean, it's an unconf, the participants define the agenda
[15:36] guido_g pieterh: mostly how to *use* ømq
[15:36] mikko guido_g: i can demo my current project there
[15:36] mikko guido_g: possibly
[15:37] jond what i don't understand is why there's isnt more interest in sub forwarding....
[15:37] guido_g i'm still having troubles to udnerstand some things in there (or not)
[15:37] pieterh jond: too difficult to make it work IMO
[15:37] pieterh no plugin framework for sockets in 0MQ
[15:37] guido_g pieterh: sure, i read about the unconf thingy
[15:37] pieterh guido_g: it'd be great to see you there, and you'd have time to get into any area you want
[15:38] jond pieterh: i thought 29 west managed it?
[15:38] pieterh jond: managed it? you mean sub forwarding?
[15:39] pieterh in theory it's pretty simple
[15:39] guido_g pieterh: ok, i'll get to a travel agency next week
[15:39] pieterh the difficulty is just writing code that fits into the core lib afaics
[15:39] mikko guido_g: where are you based?
[15:39] guido_g mikko: hamburg, germany
[15:39] pieterh guido_g: yay! :)
[15:39] mikko cool
[15:40] jond mikko: are you and ian barber going on the eurostar
[15:40] mikko jond: my plan is still a bit open
[15:40] mikko ian is going on morning of 10th
[15:40] guido_g pieterh: this will cost me over half a month of living w/o the need to work, better the beer is tasty and the chocolate plenty (or vice versa) :)
[15:40] mikko and leaving in the evening
[15:40] mikko i am either going on morning of 10th or evening of 9th
[15:40] pieterh guido_g: I promise the beer will be the best in Belgium and the company even better
[15:41] jond mikko: thanks for info
[15:41] guido_g pieterh: ok, then
[15:42] pieterh guido_g: :)
[15:42] jond gotta go do some family stuff, cyl
[15:42] pieterh cyal
[15:43] guido_g cu
[15:43] guido_g hmmm... train is ~7h
[15:59] pieterh guido_g: you don't fly?
[16:00] guido_g pieterh: given that it *needs* a cell phone to get a flight, i was looking for alternatives
[16:01] pieterh hmm, 7h is a long train journey
[16:01] guido_g that's why i decided to dump the whole thing at a travel agency
[16:01] guido_g full ack
[16:01] guido_g oh... need to fix the laptop
[16:03] pieterh guido_g: all the flights I can see are insanely expensive :(
[16:04] guido_g pieterh: ~250€ lufthansa
[16:04] pieterh return? that's a good price...
[16:04] guido_g it's a privilege to live in germany :(
[16:04] pieterh he
[16:05] guido_g have fun
[19:44] thatch I am trying to get pyzmq running on rhel5 zeromq 2.1.4 compiles without problems, but the pyzmq bindings fail in the decause of a python 2.5 construct. What version of pyzmq will run on python 2.4? If any? I am having a hard time finding the python versioning info for pyzmq
[19:50] guido_g i don't think that 2.4 is supported at all
[19:58] thatch curses
[20:01] guido_g it's been quite a time since 2004-11-30
[20:02] thatch heh, I built an application running on zeromq that I was hoping to get working on RHEL5, but it seems that is not going to be possible
[20:02] guido_g there is now newer python? how old is this thing?
[20:02] thatch oh well, it runs great on Fedora and Arch, so, nuts to RHEL 5 I guess :)
[20:03] thatch Red Hat enterprise 5? It is crazy old
[20:03] thatch like 2007
[20:03] guido_g OUCH
[20:03] thatch right
[20:03] thatch but most of the Linux servers in the world are running on in
[20:03] thatch on it
[20:03] guido_g at least they had electricity back then
[20:04] guido_g thatch_: say who?
[20:04] guido_g *says
[20:04] thatch ok, maybe not "most" but a lot of em'
[20:04] thatch no worries, I will just not support it
[20:05] thatch it will be dead soon anyway since RHEL 6 is out
[20:05] guido_g ahh
[20:05] thatch what python version is suported btw? My stuff seems to work just fine on 2.6
[20:06] guido_g no idea
[20:06] thatch ok
[20:06] guido_g i'm all on debian
[20:06] guido_g except my phone :)
[20:06] thatch well, with RHEL5 out of the way that bumps me up to 2.6
[20:06] thatch debian eh?
[20:06] thatch Debian is awesome
[20:06] thatch but I am an Arch Linux dev
[20:07] guido_g it works for me since 2003, never did an upgrade :)
[20:07] thatch Are there zeromq/pyzmq packages out there for Debian 6?
[20:08] thatch I did not see them in the main repos
[20:09] headzone only testing/unstable (2.0) and experimental (2.1)
[20:09] guido_g libzmq 2.0.10 is in testing nut no pyzmq
[20:09] thatch thanks!
[20:09] thatch thats enough for me :)
[20:10] guido_g just get the tar-ball
[20:10] thatch I should ask you guys to tell me what I can do better in my zeromq app
[20:10] thatch no, I can install it just fine
[20:10] thatch I just want to have packages so that it is easier to others to deploy my software
[20:10] thatch
[20:11] guido_g unfortunately the update-frequency of libzmq and pyzmq is quite high
[20:11] guido_g so you will need the freshest tar-balls for bug-fixes anyway
[20:11] thatch yes, I maintain pyzmq for Arch, I am kept on my toes
[20:12] thatch well, good info, thanks guido_g!
[20:12] guido_g np
[20:14] thatch BTW, zeromq is AMAZING, and has let me do some very powerfull stuff, thanks for being brilliant zeromq team!
[20:15] guido_g what is this salt thingy ment to do?
[20:15] thatch salt is a remote execution system, basically I made it to replace func and puppet labs' marionate collective
[20:16] thatch so you can execute commands on groups of servers in paralell
[20:16] guido_g ic
[20:16] thatch I also have some guys using it for general distributed computing and som hpc work
[20:17] thatch I also use it to gather system stats, and as the backend communication layer for my cloud controller
[20:18] thatch I have a bunch of explanations here:
[20:18] thatch
[20:18] guido_g nice
[20:18] thatch thanks
[20:19] guido_g i thnink pieterh should know about your projects
[20:20] thatch pieterh?
[20:20] guido_g he's the uber-marketing guy for ømq
[20:20] thatch Should I post to the mailing list?
[20:20] guido_g yes, i think he'll love it
[20:20] thatch I would love to be on the list "stuf made with zeromq"
[20:21] thatch ok, I will do just that
[20:21] guido_g and i'm sure he'll ask you to add the projects to the wiki :)
[20:21] thatch sweet, because I want more attention for salt, it is amazing what zeromq has enabled me to do with it
[20:22] thatch I will have a full replacement backend for nagios and munin/cactai soo based on salt and on zeromq
[20:22] guido_g sounds great
[20:22] guido_g all python based?
[20:22] thatch I think it will be
[20:22] thatch yes, all python
[20:23] thatch well, and some cythin for the bits that need speed of course
[20:23] thatch cython
[20:23] guido_g of course
[20:23] guido_g <- page with the projects
[20:24] thatch Sweet, a posting I wil go
[20:27] guido_g have fun!
[22:08] cmarth I installed the 2.1.4 released on OS X 10.6.7 with : ./configure; make; make install. All seemed to work. Now, trying to link the hello world server example from "The Guide" I get a "Undefined symbols for architecture x86_64" for anything 0MQ related. Any thoughts?
[23:08] cmarth Never mind - forgot to add -lzmq to my gcc ...
[23:15] Toba never forget!