IRC Log

Wednesday April 20, 2011

[Time] NameMessage
[03:33] crodas does pub/sub works with 'inproc' ?
[04:51] sustrik crodas: yes
[04:51] crodas sustrik: I supposed that, I can't make it work with node's driver
[04:51] crodas so it's a driver bug.
[04:54] sustrik possibly
[05:10] guido_g good morning all
[05:11] guido_g while browsing i found http://jeffkreeftmeijer.com/2010/why-arent-you-using-git-flow/
[05:11] guido_g just as a point for the next process discussion ,)
[05:19] sustrik thx
[05:19] guido_g i'm not sure it'll fit the mindset of you and the other main devs
[05:26] sustrik it was already discussed in the past
[05:27] sustrik the main problem, imo, is to agree on who's going to do what etc.
[05:27] guido_g oh sorry, i must have missed that
[05:27] sustrik the physical proces is just a consequence of that
[05:28] guido_g given that you don't know beforehand who will participate, this is a big problem
[05:28] sustrik there's me, mikko and pieter at least
[05:28] sustrik we are doing most of the work
[05:28] guido_g but the link wasn't meant to re-start that discussion
[05:28] guido_g ack
[05:29] guido_g we should talk about that at brussels
[05:29] sustrik definitely
[05:31] guido_g hmmm.... seems i need to write down the points i want to talk about
[05:31] sustrik sure, do so
[05:31] sustrik i should do the same
[05:31] sustrik maybe we can post that on the wiki page
[05:32] guido_g good idea
[06:03] guido_g added a few lines to the wiki page
[07:39] djc where is this talk video from ianbarber?
[07:40] ianbarber PHP UK 2011
[08:01] djc http://www.phpconference.co.uk/talk/zeromq-answer to be exact :)
[08:01] djc thanks
[10:14] pieterh hi folks
[10:17] mikko pieterh: hi
[10:18] mikko pieterh: two new attendees from london
[10:18] mikko possibly more coming later
[10:18] pieterh sweet!
[10:18] pieterh how are you doing mikko?
[10:18] pieterh still doing the weird stuff with the kinnect?
[10:18] mikko awesomely busy
[10:18] mikko yes
[10:18] pieterh :)
[10:19] mikko http://code.google.com/p/lecturerecorder/wiki/ProgramSummary
[10:19] pieterh i have a serious question for you
[10:19] mikko sure
[10:19] pieterh oso I'
[10:19] pieterh fingers aren't awake yet...
[10:19] pieterh so I'd like to start on libzutil
[10:20] pieterh i'd like people who get the zeromq package to get a single library (libzmq) but containing libzutil as well
[10:20] pieterh is that possible in your opinion?
[10:21] pieterh assuming libzutil is a separate git, like libzapi, using autotools
[10:21] mikko how do you want them to get it?
[10:21] mikko single ./configure?
[10:21] pieterh yes, I think so
[10:21] pieterh but probably via a packaging project of some kind
[10:21] pieterh e.g. zeromq2-2
[10:22] mikko zeromq-bundle or something?
[10:22] pieterh actually, also including libzapi
[10:22] pieterh zeromq-bundle is right
[10:22] pieterh anyone going to libzmq git will get just the core library
[10:22] mikko you could create packaging project that contains simple build that invokes the sub builds
[10:22] pieterh yes
[10:23] pieterh hmm, excellent
[10:23] pieterh so zeromq2-2 can be this, simple build that invokes subbuilds for libzmq, libzutil, libzapi
[10:24] pieterh would you help me make this?
[10:24] pieterh all I need are some outlines, I can do all the grunt work
[10:26] mikko sure, this is very much similar to how openpgm is built inside zeromq build
[10:26] pieterh that's what I thought but I've not looked more closely
[10:27] pieterh ok, so I'll start with a minimal libzutil rsn, with one class, and then we can experiment with this
[10:27] pieterh zeromq2-2 will basically be the same stable 2.1 plus this extra stuff (zutil, zapi)
[10:27] pieterh the point is to make life easier for people when 3.0 arrives
[10:27] pieterh premigration
[10:28] pieterh sustrik: you around?
[11:58] NikolaVeber I'm trying to open one socket per thread
[11:58] NikolaVeber for a benchmark
[11:58] NikolaVeber and then stress the server
[11:58] NikolaVeber should I open one context
[11:58] NikolaVeber or one per thread?
[11:59] NikolaVeber does it make any sense at all, sending from multiple threads VS sequential loop
[12:00] NikolaVeber it's a PUSH socket
[12:08] pieterh hi NikolaVeber
[12:09] pieterh it makes no sense to start multiple threads
[12:09] pieterh unless you know you're full with 1 thread
[12:09] djc pieterh: thanks for the release
[12:09] djc mikko: around?
[12:10] pieterh djc: happy to oblige
[12:11] djc pieterh: --with-system-pgm doesn't seem to entirely do the trick here :(
[12:11] pieterh djc: interesting... mikko: you around?
[12:11] pieterh djc: you want to discuss this with mikko
[12:11] djc yeah, I figured :)
[12:19] NikolaVeber pieterh, ok, tx :)
[12:20] pieterh NikolaVeber: a single client thread, looping, can overwhelm a server that does *anything* at all with the messages
[12:20] pieterh the only reason to have multiple client threads is to test server parallelism
[12:24] NikolaVeber then it should work for me, I just need to bring the server to the max
[12:24] NikolaVeber and measure latencies along the way
[12:36] djc hmm, maybe I could use some advice
[12:36] djc I have a set of n sources (like ZMQ publishers)
[12:36] djc and a set of m sinks
[12:37] djc and in the middle are some rules about which sinks want to receive what data (think ZMQ_SUBSCRIBE filters) from what sources
[12:37] djc what do I use?
[12:37] guido_g more coffee :)
[12:38] djc I don't do coffee, maybe that's my problem :P
[12:38] drbobbeaty pieterh: Question about the newly released 2.1.5... and maybe this is for mikko, but I'm getting errors in the build - all with the 'make check-TESTS' target. Throwing a lot of zmq::error_t exceptions. What's up?
[12:38] guido_g djc: oh sorry
[12:39] pieterh drbobbeaty: we'll check it out
[12:39] pieterh seems I was a little hasty with the 2.1.5 release
[12:39] drbobbeaty pieterh: thanks for checking on this. I really appreciate it.
[12:40] guido_g djc: did you see chapter 5 of guide? http://zguide.zeromq.org/page:all#Chapter-Five-Advanced-Publish-Subscribe
[12:40] guido_g maybe there is something you can use
[12:40] pieterh djc: "rules" meaning what?
[12:41] pieterh data that you can move around, or code?
[12:41] djc pieterh: things that will change during execution
[12:41] pieterh I'd suggest a couple of flows
[12:41] djc e.g. the client says I now want to subscribe to source b with filters c
[12:41] pieterh a. pub to sub of all data
[12:41] pieterh b. rules to subs of all rules, as they change
[12:41] pieterh subs can then unsubcribe/resubscribe dynamically
[12:44] pieter_hintjens sigh. my son turned the power off for the whole building
[12:45] pieter_hintjens djc: did you get that?
[12:45] guido_g pieter_hintjens: last was: <pieterh> subs can then...
[12:45] pieter_hintjens ok, so one pubsub flow for data, one pubsub flow for rules
[12:46] pieter_hintjens subscribers can then get the rules they want, apply them by subscribing/unsubscribing dynamically
[12:46] djc pieter_hintjens: ok, so I basically code my own FORWARDER with the subscriber connecting to multiple publishers, so I get a single pub-stream?
[12:47] djc and then use a smart header for those messages to subs can do useful filtering?
[12:47] pieter_hintjens you have N subscribers?
[12:47] djc yes
[12:47] pieter_hintjens so cross connect them all, it's simplest
[12:48] pieter_hintjens every subscriber to every publisher
[12:48] djc hm
[12:48] guido_g or use (e)pgm
[12:48] pieter_hintjens then, separately, connect every subscriber to the rule broker
[12:48] pieter_hintjens using the same SUB socket, for simplicity
[12:48] pieter_hintjens now process incoming messages
[12:48] pieter_hintjens make sure your rules have an identifiable key
[12:48] djc I don't need the rule broker because each sub sets its own sub rules
[12:49] pieter_hintjens you said "in the middle are some rules"
[12:49] pieter_hintjens who produces rules at runtime?
[12:49] djc that was just my mental model failing
[12:49] djc the subs produce the rules
[12:49] pieter_hintjens ah, so even simpler
[12:49] djc yeah
[12:49] pieter_hintjens just sub/unsub dynamically
[12:50] pieter_hintjens are you trying to get a single consistent stream to all subscribers?
[12:51] pieter_hintjens or is it OK for each subscriber to get messages in different orders
[12:51] djc I don't think exact order matters here
[12:51] djc certainly not order accross streams
[12:52] djc why would the order be different on different subscribers?
[12:52] pieter_hintjens publishers will unicast to each subscriber
[12:52] pieter_hintjens arrival order at subscribers will vary
[12:52] pieter_hintjens if you have >1 publisher, messages from each will arrive in varied order
[12:53] pieter_hintjens sub 1 will get A1, B1, B2, A2
[12:53] djc because of the interleaving, right?
[12:53] pieter_hintjens sub 2 will get A1, A2, B1, B2
[12:53] pieter_hintjens etc.
[12:53] djc but A1 will always come before A2?
[12:53] pieter_hintjens yes
[12:53] djc yeah, I don't think that's a problem
[12:54] pieter_hintjens so don't use a device, it'll add complexity for nothing
[12:54] pieter_hintjens unless you want to make publishers invisible to subscribers, then it can help
[12:54] djc well, everything is flowing through a single process
[12:55] pieter_hintjens what does that mean?
[12:55] pieter_hintjens you just said you have N publishers and N subscribers :)
[12:55] djc yeah, but there's a single process handling the connection
[12:56] djc it's a WebSockets server that clients can connect to, subscribe to ZeroMQ streams and get JSON messages
[12:56] pieter_hintjens djc: your explanation isn't helping, you need to draw a picture
[12:56] djc sorry :)
[12:57] pieter_hintjens 'flowing through' means, to me, 0MQ sockets talking to each other, but you're not explaining that
[12:57] pieter_hintjens so, confusion
[12:57] djc is there like a pastebin for drawing graphs? :P
[12:58] pieter_hintjens hmm, that'd be a neat service... I use ditaa for graphs but there's no pastebin for that
[12:58] headzone http://www.lucidchart.com/pages/
[12:59] djc using gdocs for now...
[13:02] djc https://docs.google.com/drawings/pub?id=1n2IFhfm-mO2mCAzwHLk4-tESDG9D9mCxbIaKAlKgckc&w=960&h=720
[13:02] djc is what I'm thinking of
[13:03] pieter_hintjens headzone: sweet god that's amazingly well done...
[13:04] pieter_hintjens djc: where is the "single process"?
[13:04] djc it's the big fat thing in the middle
[13:04] pieter_hintjens multithreaded?
[13:04] djc if needed, yes
[13:04] pieter_hintjens well, if multithreaded, it's effectively N processes
[13:05] pieter_hintjens not relevant to discussion, really
[13:05] djc so that's why I said it was N processes :P
[13:05] pieter_hintjens anyhow, yes, this'll work, you don't need any forwarder device afaics
[13:06] djc I just said it was flowing through a single process to explain that I might use a forwarder to forward stuff from local to inproc, so I don't need three or four streams of the same over my LAN
[13:06] djc (the left side)
[13:06] pieter_hintjens what's the message size/rate you expect?
[13:08] djc the sizes are not very impressive, but I expect a fairly rate
[13:08] djc +high
[13:08] djc not sure how high, exactly
[13:09] djc zmq doesn't handle that for me, right? like, it doesn't multiplex for multiple subscribers subscribing to the same publisher within the same context?
[13:28] djc would actually be cool if it could do that
[13:29] th what does a "what(): Bad address" mean when doing recv()? i could understand that for bind/connect
[13:29] th this happens for me in c++ with 2.1.5 and it did not happen in 2.1.4
[13:34] th hmmm #define EFAULT 14 /* Bad address */
[13:36] th http://api.zeromq.org/2-1:zmq-recv tells me EFAULT can mean invalid socket or message; and the changelog for 2.1.5 mentions "Checks zmq_msg_t validity at each operation"
[13:42] mikko i'm back
[13:42] mikko making moroccon lamb stew
[13:44] mikko djc: which os?
[13:51] djc mikko: gentoo linux
[13:52] mikko ok, which version of OpenPGM?
[13:53] mikko and can you run with ./configure --disable-silent-rules
[13:53] mikko so that i can see the output of linking
[13:54] djc 5.1.115
[13:54] djc I'll try that
[13:55] mikko it sounds like pkg-config for openpgm might fail
[13:55] mikko there is currently no check for that
[13:56] pieter_hintjens djc: I'm back, sorry
[13:57] pieter_hintjens currently zmq sub sockets will multiplex to all sub sockets
[13:57] pieter_hintjens there's a work in progress to be smarter about this but it's not got a lot of traction
[13:58] pieter_hintjens using an inprocess frontend device is an obvious optimization
[13:58] pieter_hintjens http://zguide.zeromq.org/page:all#High-speed-Subscribers-Black-Box-Pattern
[13:59] pieter_hintjens th: hi
[13:59] pieter_hintjens I think we have a problem in the 2.1.5 release
[13:59] pieter_hintjens but it's not clear yet
[14:00] mikko what problem?
[14:00] pieter_hintjens tests and other code are failing due to last commit which checks msg validity
[14:00] pieter_hintjens make check-TESTS
[14:01] pieter_hintjens make[2]: Entering directory `/vol1/home/ph/work/zeromq2-1/tests'
[14:01] pieter_hintjens terminate called after throwing an instance of 'zmq::error_t'
[14:01] pieter_hintjens what(): Bad address
[14:01] pieter_hintjens /bin/bash: line 5: 883 Aborted (core dumped) ${dir}$tst
[14:01] pieter_hintjens FAIL: test_pair_inproc
[14:01] mikko http://build.zero.mq/view/libzmq%202.1/ shows all ok
[14:01] mikko strange
[14:01] mikko is the commit after this morning build?
[14:02] djc mikko: http://paste.pocoo.org/raw/375143/
[14:03] pieter_hintjens mikko: I applied a patch from sustrik after this mornings' build
[14:03] mikko djc: the whole output if you can
[14:04] mikko djc: as that linking failure is for perf programs
[14:04] mikko would be interesting if -lpgm is skipped for libzmq.la for some reason
[14:05] djc http://dirkjan.ochtman.nl/files/build.log
[14:06] pieter_hintjens sustrik: ping, are you around?
[14:09] mikko djc: strange
[14:09] mikko can you pastebin the output of 'nm' for openpgm.so
[14:14] th a simple chain of new zmq::context_t(1); new zmq::socket_t(*context, ZMQ_XREP); bind("tcp://127.0.0.1:5555"); zsock->recv(&message); results in "Bad address"
[14:14] th this makes 2.1.5 somewhat unusable at least for me here
[14:14] mikko Please do *not* use the 2.1.5 package released today, the zmq_msg_t
[14:14] mikko validity checking appears to be reporting errors on valid code.
[14:15] mikko peter's mail 8 minutes ago
[14:19] th oh - i did not even notice that this release was from today
[14:20] th i just checked the page, because someone told me a bug was fixed
[14:21] djc mikko: nm finds no symbols in libpgm.so
[14:21] djc ..
[14:21] pieter_hintjens th: so sorry, I've reverted the download page to point to 2.1.4
[14:21] th pieter_hintjens: but not the changelog;)
[14:21] pieter_hintjens th: I'll push a clean git history (aaagh) in a second
[14:22] th pieter_hintjens: is there any recommended git revision > 2.1.4?
[14:22] pieter_hintjens th: nope
[14:22] pieter_hintjens take https://github.com/zeromq/zeromq2-1/commits/v2.1.4
[14:23] pieter_hintjens or take the package
[14:23] th pieter_hintjens: which is the same as ... ok
[14:23] th pieter_hintjens: that still has the multipart intermixing issues
[14:23] th under higher load
[14:23] pieter_hintjens th: I'm not familiar with that bug, and do not think it's been downstreamed to 2.1.5 at all
[14:24] djc pieter_hintjens: you shouldn't unpublish changes from a public repo
[14:25] pieter_hintjens djc: well, I shouldn't allow buggy code to get into a stable distro
[14:25] th pieter_hintjens: i'll see if i can make the test-case smaller and more readable. it's 100% reproducable already for me. but it needs a './server & ; while ./client ; do;done'
[14:25] djc pieter_hintjens: so publish a new revision
[14:26] djc or even a new tag
[14:26] pieter_hintjens djc: ok, I'll respect the rules for once
[14:26] pieter_hintjens question then
[14:26] pieter_hintjens 2.1.6 or 2.1.5 or 2.1.5.1?
[14:26] djc if you're taking 2.1.5 and just removing on buggy cset, call it 2.1.5.1
[14:26] djc s/on/one/
[14:27] pieter_hintjens ok, that's what I'd done before sustrik asked me to pause
[14:40] pieter_hintjens sustrik: ping, are you around?
[15:20] mikko th: no symbols at all?
[16:30] sustrik re
[16:31] sustrik pieter_hintjens: sorry, the reply abnout the issue was accidentally sent only to artem
[16:31] sustrik let me forward it to the list
[16:39] pieterh sustrik: thx
[16:40] pieterh sustrik: seems we have a problem with cherry-picking
[16:40] pieterh it does not work reliably
[16:42] sustrik np
[16:42] pieterh what happened was this...
[16:43] pieterh well, yes, we do have a problem somewhere... :-/
[16:43] pieterh when I apply the commit, it gives a conflict
[16:43] pieterh there should be no conflict
[16:43] pieterh clearly I'm not modifying these files anywhere else
[16:43] pieterh incorrect conflict resolution -> garbage
[16:44] pieterh anyone with sufficient git fu to help figure this out?
[16:45] pieterh the only way I can see now to fix this is to reset history to before the cherry-pick and do it again
[16:45] pieterh this is really unpleasant
[16:46] headzone what's the situation (in generic git terms)
[16:47] pieterh headzone: ok, here's where I find myself...
[16:47] pieterh working in github.com/zeromq/zeromq2-1, which fetches commits from github.com/zeromq/libzmq
[16:47] pieterh 'git fetch --no-tags libzmq' followed by cherry-picking of specific commits
[16:48] pieterh one of these manages to fail, partially and silently, on zmq.cpp
[16:48] sustrik the conflict is presumably between changes that were not backported and those that were backported
[16:49] sustrik in one of the previous commits in libzmq i've moved message-related functions into a separate compilation unit
[16:49] sustrik msg.hpp/cpp
[16:49] sustrik that's what causing the conflict imo
[16:49] pieterh sustrik: there wasn't even a conflict on this, in fact
[16:49] pieterh http://pastebin.com/ddC7HfDH
[16:49] pieterh yes, I got an error on msg.cpp
[16:49] pieterh but the missing chunks were on zmq.cpp...
[16:50] pieterh anyhow, I now have a bit of a mess in my zeromq2-1 repository
[16:50] sustrik which contains functions that are in zmq.cpp in 2.x
[16:50] sustrik backporting i shard
[16:50] pieterh ah... so the fix is in a file I don't actually have...
[16:50] sustrik you may try to apply the parch that moved the functions to msg.cpp
[16:51] sustrik first
[16:51] sustrik ack
[16:52] pieterh sustrik: I really don't want to be creative with the libzmq code...
[16:52] pieterh the changes that are missing in zmq.cpp are presumably few
[16:52] sustrik let me have a look
[16:53] pieterh sustrik: can you clone git@github.com:zeromq/zeromq2-1.git?
[16:53] sustrik i did
[16:53] pieterh nice
[16:53] pieterh headzone: with luck sustrik will be able to help me patch this up and we can go forwards...
[16:54] headzone it sounds like you're trying to cherry-pick something from a branch that's diverged from yours, which is usually not a great idea
[16:54] pieterh yup
[16:54] pieterh i should have caught this in testing, which will happen from now on
[16:54] pieterh properly
[16:54] sustrik ok, afaics the only thing you need to do is to apply the changes reported as been done to msg.cpp
[16:55] sustrik to zmq.cpp instead
[16:55] pieterh ok... let me review those
[16:56] pieterh sustrik: I'll give that a shot, it should be doable
[16:56] sustrik ok
[17:02] headzone in future, you may also find it useful to use the -n option to cherry-pick or format-patch instead, to see and inspect how things will look first without messing up your tree
[17:04] pieterh headzone: yes, will use -n option on cherry-pick
[17:05] pieterh I assume after reviewing, one commits with 'git commit -c nnnnnn'?
[17:06] headzone sure
[17:08] pieterh thx, I'm keeping the idiot proof instructions here: http://www.zeromq.org/docs:distributions
[17:10] headzone in the second part, "git pull source master; git fetch source" is better written "git fetch source; git merge source/master"
[17:13] pieterh headzone: thanks, I'll fix it (you can also edit, it's a wiki)
[17:14] pieterh sustrik: what code is producing this error...?
[17:15] pieterh terminate called after throwing an instance of 'zmq::error_t'
[17:15] pieterh what(): Bad address
[17:15] pieterh Aborted (core dumped)
[17:15] pieterh in the test cases?
[17:15] pieterh after patching zmq.cpp, it fails exactly the same way, I can't trace where
[17:17] sustrik try to run the failing test case under dgb
[17:17] sustrik gdb
[17:17] sustrik there's should be a way to say 'stop at exception'
[17:17] pieterh not executable format? wtf?
[17:18] pieterh agh
[17:18] sustrik aha, Bad address is EFAULT
[17:19] sustrik ie. message validation fails
[17:19] pieterh I can't even see where the test executables are...
[17:19] sustrik tests subdir
[17:19] pieterh all I see are scripts that magically run the test executables
[17:20] th mikko: symbols? you meant me?
[17:20] mikko yes
[17:20] mikko no
[17:20] mikko i meant djc
[17:20] sustrik i see, that's because it uses shared libs
[17:20] sustrik so there's a script that changes paths
[17:20] sustrik to load the right library
[17:20] sustrik you can configure with --disable-shared
[17:20] sustrik which will force the static linking
[17:21] sustrik and thus remove the need for scripts
[17:21] pieterh well, when I tried artem's fix, I didn't do any of that, and the fix seemed to work
[17:21] pieterh now I can't get even a 'printf' to display
[17:21] sustrik it will fail if the message is shared
[17:21] pieterh sure, that's not my point
[17:21] pieterh it's that I find myself stuck in a twisty maze of "am I running this version or not..."
[17:22] pieterh I'll disable shared libraries
[17:22] sustrik i do it that
[17:22] sustrik way
[17:22] sustrik you'll get executables with statically linked libzmq
[17:22] sustrik directly in tests subdir
[17:23] pieterh sure, makes sense but I never had to do that before
[17:25] pieterh nah, this is just messed up
[17:25] pieterh it aborts on zmq.hpp:278
[17:25] pieterh the file only has 266 lines
[17:25] pieterh somehow I'm getting versions totally mixed up :-(
[17:26] pieterh mikko: if I run 'make' in zeromq2-1 root, it shouldn't be pulling in so's from elsewhere, should it...?
[17:27] pieterh isn't this the day skynet takes control?
[17:28] pieterh sustrik: for some godforsaken reason the tests won't build correctly for me
[17:36] mikko pieterh: sorry?
[17:37] pieterh mikko: sorry, skynet temporarily got past my cortical firewall, it's gone now...
[17:38] pieterh i found out what the problem was, and it seems much better now
[17:41] pieterh mikko: djc reported earlier today that --with-system-pgm didn't work in 2.1.5
[17:41] mikko yep
[17:41] pieterh do you want me to hold off with 2.1.5.1?
[17:41] mikko please
[17:42] mikko i'll look into it tonight
[17:42] pieterh ok, np
[17:42] pieterh btw we can also get that redhat spec file into it if you have it
[17:42] pieterh depends how long your nights are... :)
[17:55] mikko that change is pretty trivial
[18:00] pieterh everything is trivial to the right brain :-)
[18:05] mikko i need to take a small break
[18:05] mikko juggling million things at the moment
[18:05] mikko bbl
[18:06] pieterh cyl
[18:10] guido_g pieterh: do you have an idea what a taxi ride from the airport to the meeting place might cost?
[18:11] pieterh guido_g: I'll add a travel guide to the wiki page, taxis are extortionate
[18:11] pieterh it'
[18:11] pieterh it's easy to get to by public transport
[18:11] guido_g ok
[18:12] guido_g just wanted to know if i could do it the lazy way :)
[18:18] jond pieterh: are master and 2.1 repo still bust with the message check bug?
[18:19] pieterh jond: libzmq master was always working, 2.1 master is now in theory working again
[18:20] jond pieterh : so how did the patch for libzmq -> 2.1 go wrong then? does it have to be applied and fixed manually?
[18:21] pieterh libzmq had diverged and the patch was not usable as-such, and my resolution of that error was wrong
[18:22] jond pieterh: ok understood, just wanted to be clear, so essentially patches to libzmq master now may have to be manually reworked for the backport.
[18:23] pieterh jond: not until now, no
[18:23] pieterh but this seems to be inevitable as 3.0 diverges
[18:23] pieterh I guess we were kind of expecting it but missed it when it actually happened :-(
[18:25] jond pieterh: yeah, looked bad on the list....but stuff happens
[18:25] pieterh well, stuff happens... I don't mind looking bad, so long as we don't make the same mistakes twice
[18:26] pieterh Its far worse to not make releases out of fear of getting it wrong
[18:40] guido_g ohh... an irish pub, how pleasant
[18:47] pieterh guido_g: an irish pub wandered into your office?
[18:48] guido_g ah no
[18:48] guido_g a) i'm at home
[18:48] guido_g and b) was looking at brussels w/ google-earth
[18:48] traviscline just pushed some further cython optimizations to gevent-zeromq's HEAD please report any issues if they occur
[19:07] pieterh guido_g: :) irish pubs in brussels are crap (my word of the day)
[19:08] pieterh the best pubs are the authentically ethnic ones
[19:08] guido_g ok
[19:08] pieterh hehehe
[19:09] pieterh I meant, the Irish Pubs are fantastic!
[19:09] pieterh brussels imports all its beer from dublin
[19:09] pieterh srsly
[19:10] guido_g you should be happy that i'm not the average german
[19:10] guido_g you know how picky they are wrt beer
[19:18] pieterh guido_g: I once had a GF from Bavaria... sigh...
[19:18] pieterh beautiful but beer crazy
[19:18] pieterh this is very relevant to #zeromq somehow
[19:19] pieterh btw I put up some transport tips on the unconf page
[19:19] guido_g would be great!
[19:19] pieterh see if they make even random sense
[19:19] guido_g will do
[19:20] pieterh guido_g: I was asserting that beer *is* relevant to zeromq
[19:20] guido_g i can confirm that
[19:21] jond pieterh: only if it passes certain purity and strength tests!
[19:21] pieterh jond: purity and strength are the defining characteristics of 0MQ!
[19:22] pieterh maybe I can change the 0MQ website logo to "Purity and Strength!" ?
[19:22] pieterh nah, people would think we're a cult of some kind
[19:22] pieterh which is totally not the case
[19:24] guido_g does sacrificial blood count as liquid when flying?
[19:25] pieterh guido_g: only if it's not yours afaik
[19:25] guido_g no problem, can collect some at the airport
[19:26] pieterh they sell it in Brussels, np
[19:27] guido_g what a town
[19:27] pieterh not sure if it's entirely within the Reinheitsgebot
[19:27] pieterh probably got added chocolate or somesuch
[19:27] guido_g hehe
[19:53] guido_g n8 all!
[19:54] pieterh bye guido_g
[19:54] phpcodemonkey hi all, am here looking for help with 0mq / php issue - I'm running 2.1.4 / PHP pecl zmq-beta 0.7.0 on PHP 5.3.6, x86_64 Fedora 14: I'm getting a segfault when running the basic hwclient.php from https://github.com/imatix/zguide/blob/master/examples/PHP/hwclient.php the related hwserver.php runs fine sits there waiting for client connections. I've posted a backtrace at: http://pastebin.com/ruEzjAdT anyone got any ide
[19:55] pieterh phpcodemonkey: there seems to be an error on a uid generating function
[19:55] pieterh you might need to install libuuid or similar
[19:56] pieterh gen_uuid.c: No such file or directory.
[19:56] phpcodemonkey k
[19:57] phpcodemonkey already got: libuuid-devel-2.18-4.8.fc14.x86_64 rpm installed
[19:58] pieterh that should be sufficient IMO
[19:59] pieterh mikko should be able to help, he's the guy who wrote the PHP binding
[19:59] pieterh he'll be around later on this evening normally
[19:59] phpcodemonkey provides uuid.h in /usr/include so all seems to be correct
[19:59] phpcodemonkey pieterh: he's already asked for a backtrace via twitter :)
[19:59] pieterh phpcodemonkey: some of the uuid packages don't provide the right functions
[20:00] phpcodemonkey nice
[20:01] pieterh googling that error gives quite a lot of results
[20:01] pieterh check if there are other uuid rpms
[20:01] mikko here
[20:01] pieterh :)
[20:01] mikko enjoyed a bit of C++ dev on windows
[20:01] mikko and when i say enjoy i mean sticking bamboo sticks under nails
[20:02] mikko phpcodemonkey: have you got imagick / uuid pecl extension loaded?
[20:04] phpcodemonkey imagick 3.0.1
[20:04] mikko can you try loading zmq extension before imagick?
[20:04] mikko this is a known unknown issue with imagemagick / uuid
[20:04] phpcodemonkey e.g. change zeromq.ini to 0mq.ini?
[20:04] mikko yes
[20:04] mikko that should do the trick
[20:05] mikko http://usrportage.de/archives/922-PHP-segfaulting-with-pecluuid-and-peclimagick.html
[20:05] mikko it looks like this issue
[20:05] mikko i have no idea what is going on in there
[20:05] mikko seems to be some global state inside uuid lib or so
[20:05] phpcodemonkey mikko: you rock, that was it!
[20:05] phpcodemonkey perhaps that out to be in the docs somewhere?
[20:06] mikko sure, if you can add a git pull request ill merge it
[20:07] phpcodemonkey git != svn ;)
[20:12] mikko pieterh: --with-system-pgm works for me (tm)
[20:13] pieterh mikko: did djc provide details of what "doesn't work" means for him?
[20:13] mikko yes
[20:13] mikko fails to link
[20:13] mikko and nm on openpgm.so doesnt show _any_ symbols
[20:13] pieterh ah
[20:13] mikko djc: are you still here?
[20:13] pieterh empty library or somesuch
[20:14] pieterh mikko: nice catch on phpcodemonkey's problem there
[20:14] mikko thanks
[20:15] phpcodemonkey definitely, was frustrating the heck out of me!
[20:16] phpcodemonkey though having read the blog post you referenced, it seems that 0mq and imagick on their own do the damage, don't have pecl/uuid installed
[20:37] mikko phpcodemonkey: it's any extension using uuid functions + imagick
[20:38] phpcodemonkey mikko: figured that, was just confirming :)
[20:47] mikko pieterh: want me to push the spec file change directly?
[20:47] pieterh mikko: sure, that'd be simplest
[20:49] mikko ok, will push soon(ish)
[20:49] mikko i'll make sure that the change makes it midnight's rpm builds
[20:58] mikko pieterh: can you test 2.1?
[20:58] pieterh you mean a normal make check?
[20:59] pieterh ok, running a full build / test process
[21:00] mikko if make dist works it should be fine
[21:00] mikko i'll fix small bug in ztools and run rpm builds now
[21:02] pieterh mikko: it all seems to work, with your changes
[21:06] mikko build running
[21:06] mikko bbr
[21:06] mikko brb
[21:21] mikko pieterh: seems to be ok
[22:53] coopernurse hi zeromq folks. anyone have a few minutes to chat with a zeromq newb? I have some basic architectural questions. I have read the whitepaper. I'm coming from a traditional broker based background, so I'm trying to unlearn some concepts.
[23:04] traviscline coopernurse: irc etiquette is "to ask, not to ask to ask"
[23:04] traviscline *cough cough* it's mostly so people don't accidentally get roped into questions they can't answer and look wrong :P
[23:05] coopernurse ok, great
[23:06] coopernurse I just read this: http://www.zeromq.org/whitepapers:brokerless
[23:07] coopernurse and I want to make sure I understand the suggested approach
[23:07] coopernurse is it correct to consider zeromq a toolkit for building your own message routers?
[23:07] coopernurse for example, in the distributed broker diagram
[23:08] coopernurse Q1, Q2, and Q3 are daemons that I would write correct? that would encapsulate whatever routing topology was relevant
[23:21] jond coopernurse: I think reading the guide will be better, some of the whitepapers are quite old ; http://zguide.zeromq.org/page:all
[23:22] coopernurse jond: I've read through that as well (about a month ago). Nothing I've read talks about suggested deployment architectures.
[23:23] coopernurse For example, I want to have a conversation with some operations folks about this
[23:23] coopernurse And they think in terms of redundancy and points of failure
[23:24] jond coopernurse: well in that case , it would be better to be on here on CET timezone. Though yes with zeromq you could build custom routers
[23:25] coopernurse The whitepaper was the closest thing I've seen so far to discussing deployment topologies
[23:25] coopernurse I understood the suggestion to potentially use a directory service for components to locate queue endpoints
[23:26] coopernurse But I was unclear whether the Q1, Q2, etc were provided out of the box by zeromq, or were daemons you write yourself
[23:26] coopernurse I'm assuming it's something I would have to write
[23:26] jond there also is some stuff in the later chapters of the guide which may have been added recently: It would be best to hook up with pieterh on here
[23:26] coopernurse ok
[23:27] coopernurse but in general is it correct to view zeromq as a toolkit for creating your own messaging topologies?
[23:27] coopernurse or are there higher level turnkey solutions that I have missed that come with the distribution?
[23:29] jond well I've just looked atb the whitepaper, if you look at the stuff below the distributed broker which clicks to a chat example which is now marked as deprecated, so the best place is gonna be the guide and then irc or the mailing list
[23:29] headzone coopernurse: funnily enough, we have just been debating that question
[23:29] coopernurse jond: ok thanks
[23:30] coopernurse headzone: oh yeah?
[23:31] headzone coopernurse: the current thinking appears to be that the goal of 0mq is to provide standard/pre-packaged known-good messaging patterns (topologies), rather than be a toolkit that users use to build their own custom ones
[23:32] coopernurse ah, interesting. and if that's the case, would those pre-packaged patterns take the form of daemons you can run out of the box?
[23:32] headzone no, they take the form of the patterns supported by the sockets, e.g. req/rep, pub/sub
[23:33] coopernurse ok, so this is where I really just don't get it
[23:33] coopernurse perhaps I need to rtfm more carefully
[23:33] headzone with maybe some "devices" on top at a higher layer (the things currently called queue/forwarder/streamer)
[23:33] jond headzone: I think you can use it for both, but in some areas that is quite fluid. There was a concept of devices -> queue, forwarder, streamer but they are being moved out of the core
[23:34] coopernurse ok, so "devices" is the term I probably need to be examining in the guide more closely
[23:34] headzone coopernurse: the reason this is confusing at the moment is that the current state of 0mq is sort of stuck in the middle of "standard patterns" and "toolkit for building custom application-layer routing topologies", for historical reasons
[23:35] coopernurse presumably the Q1, Q2, etc in the (perhaps deprecated?) whitepaper are potentially devices zeromq provides
[23:35] coopernurse ok
[23:36] jond coopernurse: the guide doesnt mention them much and it also uses this libzapi which sits atop 0mq; pieterh is the one behind all this
[23:36] headzone coopernurse: i found this out by assuming the goal was to become a generic application-layer routing topology construction kit and suggesting a protocol stack that would accomplish this in a sane manner
[23:38] coopernurse As someone who is coming from ActiveMQ/RabbitMQ/etc it's difficult to find analogs in zeromq
[23:38] headzone coopernurse: but it is far from clear that that is a desirable goal to aim for, because it amounts to recapitulating what the IP network and transport layers already provide
[23:38] coopernurse headzone: yes, I'm just trying to wrap my head around how an application developer could use zeromq to transition a monolithic web app into a set of more loosely coupled components
[23:39] coopernurse I agree with the "broker is a single point of failure" stuff in the zeromq literature. That got me interested.
[23:39] coopernurse But I'm not sure how I avoid writing lots of little devices that are each single points of failure
[23:40] headzone right, and basically you do that by combining the standard messaging patterns, the "device" functions (which are really just commonly used pieces of application logic), and application logic to handle anything else that's required
[23:41] coopernurse headzone: ok, thanks. I appreciate your time.
[23:41] headzone regarding redundancy, i happen to think that solving that problem in a general way inexorably leads you to reinventing routing
[23:42] coopernurse is there a recommended way to do it in zeromq?
[23:42] coopernurse (redundancy that is)
[23:42] headzone well, there are lots of examples in the guide
[23:43] coopernurse ok
[23:43] headzone a fair portion of it is about the hoops you have to jump through to get reliability
[23:44] coopernurse all of it seems very point to point though
[23:44] headzone it's also all very ad-hoc
[23:44] coopernurse all the examples I've seen seem to assume the components know who to talk to
[23:45] coopernurse I'm looking at the "Designing Reliability" section again now
[23:48] coopernurse ok, well thanks headzone and jond. I'll keep reading. I'm off.

Web site design and content is copyright © 2010-2011 Peter Alexander, ProDataLab. Site content licensed under the Creative Commons Attribution-Share Alike 3.0 License. ØMQ is copyright © Copyright © 2007-2011 iMatix Corporation and Contributors. Contact us for professional support. ØMQ is free software licensed under the LGPL. ØMQ, ZeroMQ, and 0MQ are trademarks of iMatix Corporation. Terms of Use ? Privacy Policy