[Time] Name | Message |
[03:33] crodas
|
does pub/sub works with 'inproc' ?
|
[04:51] sustrik
|
crodas: yes
|
[04:51] crodas
|
sustrik: I supposed that, I can't make it work with node's driver
|
[04:51] crodas
|
so it's a driver bug.
|
[04:54] sustrik
|
possibly
|
[05:10] guido_g
|
good morning all
|
[05:11] guido_g
|
while browsing i found http://jeffkreeftmeijer.com/2010/why-arent-you-using-git-flow/
|
[05:11] guido_g
|
just as a point for the next process discussion ,)
|
[05:19] sustrik
|
thx
|
[05:19] guido_g
|
i'm not sure it'll fit the mindset of you and the other main devs
|
[05:26] sustrik
|
it was already discussed in the past
|
[05:27] sustrik
|
the main problem, imo, is to agree on who's going to do what etc.
|
[05:27] guido_g
|
oh sorry, i must have missed that
|
[05:27] sustrik
|
the physical proces is just a consequence of that
|
[05:28] guido_g
|
given that you don't know beforehand who will participate, this is a big problem
|
[05:28] sustrik
|
there's me, mikko and pieter at least
|
[05:28] sustrik
|
we are doing most of the work
|
[05:28] guido_g
|
but the link wasn't meant to re-start that discussion
|
[05:28] guido_g
|
ack
|
[05:29] guido_g
|
we should talk about that at brussels
|
[05:29] sustrik
|
definitely
|
[05:31] guido_g
|
hmmm.... seems i need to write down the points i want to talk about
|
[05:31] sustrik
|
sure, do so
|
[05:31] sustrik
|
i should do the same
|
[05:31] sustrik
|
maybe we can post that on the wiki page
|
[05:32] guido_g
|
good idea
|
[06:03] guido_g
|
added a few lines to the wiki page
|
[07:39] djc
|
where is this talk video from ianbarber?
|
[07:40] ianbarber
|
PHP UK 2011
|
[08:01] djc
|
http://www.phpconference.co.uk/talk/zeromq-answer to be exact :)
|
[08:01] djc
|
thanks
|
[10:14] pieterh
|
hi folks
|
[10:17] mikko
|
pieterh: hi
|
[10:18] mikko
|
pieterh: two new attendees from london
|
[10:18] mikko
|
possibly more coming later
|
[10:18] pieterh
|
sweet!
|
[10:18] pieterh
|
how are you doing mikko?
|
[10:18] pieterh
|
still doing the weird stuff with the kinnect?
|
[10:18] mikko
|
awesomely busy
|
[10:18] mikko
|
yes
|
[10:18] pieterh
|
:)
|
[10:19] mikko
|
http://code.google.com/p/lecturerecorder/wiki/ProgramSummary
|
[10:19] pieterh
|
i have a serious question for you
|
[10:19] mikko
|
sure
|
[10:19] pieterh
|
oso I'
|
[10:19] pieterh
|
fingers aren't awake yet...
|
[10:19] pieterh
|
so I'd like to start on libzutil
|
[10:20] pieterh
|
i'd like people who get the zeromq package to get a single library (libzmq) but containing libzutil as well
|
[10:20] pieterh
|
is that possible in your opinion?
|
[10:21] pieterh
|
assuming libzutil is a separate git, like libzapi, using autotools
|
[10:21] mikko
|
how do you want them to get it?
|
[10:21] mikko
|
single ./configure?
|
[10:21] pieterh
|
yes, I think so
|
[10:21] pieterh
|
but probably via a packaging project of some kind
|
[10:21] pieterh
|
e.g. zeromq2-2
|
[10:22] mikko
|
zeromq-bundle or something?
|
[10:22] pieterh
|
actually, also including libzapi
|
[10:22] pieterh
|
zeromq-bundle is right
|
[10:22] pieterh
|
anyone going to libzmq git will get just the core library
|
[10:22] mikko
|
you could create packaging project that contains simple build that invokes the sub builds
|
[10:22] pieterh
|
yes
|
[10:23] pieterh
|
hmm, excellent
|
[10:23] pieterh
|
so zeromq2-2 can be this, simple build that invokes subbuilds for libzmq, libzutil, libzapi
|
[10:24] pieterh
|
would you help me make this?
|
[10:24] pieterh
|
all I need are some outlines, I can do all the grunt work
|
[10:26] mikko
|
sure, this is very much similar to how openpgm is built inside zeromq build
|
[10:26] pieterh
|
that's what I thought but I've not looked more closely
|
[10:27] pieterh
|
ok, so I'll start with a minimal libzutil rsn, with one class, and then we can experiment with this
|
[10:27] pieterh
|
zeromq2-2 will basically be the same stable 2.1 plus this extra stuff (zutil, zapi)
|
[10:27] pieterh
|
the point is to make life easier for people when 3.0 arrives
|
[10:27] pieterh
|
premigration
|
[10:28] pieterh
|
sustrik: you around?
|
[11:58] NikolaVeber
|
I'm trying to open one socket per thread
|
[11:58] NikolaVeber
|
for a benchmark
|
[11:58] NikolaVeber
|
and then stress the server
|
[11:58] NikolaVeber
|
should I open one context
|
[11:58] NikolaVeber
|
or one per thread?
|
[11:59] NikolaVeber
|
does it make any sense at all, sending from multiple threads VS sequential loop
|
[12:00] NikolaVeber
|
it's a PUSH socket
|
[12:08] pieterh
|
hi NikolaVeber
|
[12:09] pieterh
|
it makes no sense to start multiple threads
|
[12:09] pieterh
|
unless you know you're full with 1 thread
|
[12:09] djc
|
pieterh: thanks for the release
|
[12:09] djc
|
mikko: around?
|
[12:10] pieterh
|
djc: happy to oblige
|
[12:11] djc
|
pieterh: --with-system-pgm doesn't seem to entirely do the trick here :(
|
[12:11] pieterh
|
djc: interesting... mikko: you around?
|
[12:11] pieterh
|
djc: you want to discuss this with mikko
|
[12:11] djc
|
yeah, I figured :)
|
[12:19] NikolaVeber
|
pieterh, ok, tx :)
|
[12:20] pieterh
|
NikolaVeber: a single client thread, looping, can overwhelm a server that does *anything* at all with the messages
|
[12:20] pieterh
|
the only reason to have multiple client threads is to test server parallelism
|
[12:24] NikolaVeber
|
then it should work for me, I just need to bring the server to the max
|
[12:24] NikolaVeber
|
and measure latencies along the way
|
[12:36] djc
|
hmm, maybe I could use some advice
|
[12:36] djc
|
I have a set of n sources (like ZMQ publishers)
|
[12:36] djc
|
and a set of m sinks
|
[12:37] djc
|
and in the middle are some rules about which sinks want to receive what data (think ZMQ_SUBSCRIBE filters) from what sources
|
[12:37] djc
|
what do I use?
|
[12:37] guido_g
|
more coffee :)
|
[12:38] djc
|
I don't do coffee, maybe that's my problem :P
|
[12:38] drbobbeaty
|
pieterh: Question about the newly released 2.1.5... and maybe this is for mikko, but I'm getting errors in the build - all with the 'make check-TESTS' target. Throwing a lot of zmq::error_t exceptions. What's up?
|
[12:38] guido_g
|
djc: oh sorry
|
[12:39] pieterh
|
drbobbeaty: we'll check it out
|
[12:39] pieterh
|
seems I was a little hasty with the 2.1.5 release
|
[12:39] drbobbeaty
|
pieterh: thanks for checking on this. I really appreciate it.
|
[12:40] guido_g
|
djc: did you see chapter 5 of guide? http://zguide.zeromq.org/page:all#Chapter-Five-Advanced-Publish-Subscribe
|
[12:40] guido_g
|
maybe there is something you can use
|
[12:40] pieterh
|
djc: "rules" meaning what?
|
[12:41] pieterh
|
data that you can move around, or code?
|
[12:41] djc
|
pieterh: things that will change during execution
|
[12:41] pieterh
|
I'd suggest a couple of flows
|
[12:41] djc
|
e.g. the client says I now want to subscribe to source b with filters c
|
[12:41] pieterh
|
a. pub to sub of all data
|
[12:41] pieterh
|
b. rules to subs of all rules, as they change
|
[12:41] pieterh
|
subs can then unsubcribe/resubscribe dynamically
|
[12:44] pieter_hintjens
|
sigh. my son turned the power off for the whole building
|
[12:45] pieter_hintjens
|
djc: did you get that?
|
[12:45] guido_g
|
pieter_hintjens: last was: <pieterh> subs can then...
|
[12:45] pieter_hintjens
|
ok, so one pubsub flow for data, one pubsub flow for rules
|
[12:46] pieter_hintjens
|
subscribers can then get the rules they want, apply them by subscribing/unsubscribing dynamically
|
[12:46] djc
|
pieter_hintjens: ok, so I basically code my own FORWARDER with the subscriber connecting to multiple publishers, so I get a single pub-stream?
|
[12:47] djc
|
and then use a smart header for those messages to subs can do useful filtering?
|
[12:47] pieter_hintjens
|
you have N subscribers?
|
[12:47] djc
|
yes
|
[12:47] pieter_hintjens
|
so cross connect them all, it's simplest
|
[12:48] pieter_hintjens
|
every subscriber to every publisher
|
[12:48] djc
|
hm
|
[12:48] guido_g
|
or use (e)pgm
|
[12:48] pieter_hintjens
|
then, separately, connect every subscriber to the rule broker
|
[12:48] pieter_hintjens
|
using the same SUB socket, for simplicity
|
[12:48] pieter_hintjens
|
now process incoming messages
|
[12:48] pieter_hintjens
|
make sure your rules have an identifiable key
|
[12:48] djc
|
I don't need the rule broker because each sub sets its own sub rules
|
[12:49] pieter_hintjens
|
you said "in the middle are some rules"
|
[12:49] pieter_hintjens
|
who produces rules at runtime?
|
[12:49] djc
|
that was just my mental model failing
|
[12:49] djc
|
the subs produce the rules
|
[12:49] pieter_hintjens
|
ah, so even simpler
|
[12:49] djc
|
yeah
|
[12:49] pieter_hintjens
|
just sub/unsub dynamically
|
[12:50] pieter_hintjens
|
are you trying to get a single consistent stream to all subscribers?
|
[12:51] pieter_hintjens
|
or is it OK for each subscriber to get messages in different orders
|
[12:51] djc
|
I don't think exact order matters here
|
[12:51] djc
|
certainly not order accross streams
|
[12:52] djc
|
why would the order be different on different subscribers?
|
[12:52] pieter_hintjens
|
publishers will unicast to each subscriber
|
[12:52] pieter_hintjens
|
arrival order at subscribers will vary
|
[12:52] pieter_hintjens
|
if you have >1 publisher, messages from each will arrive in varied order
|
[12:53] pieter_hintjens
|
sub 1 will get A1, B1, B2, A2
|
[12:53] djc
|
because of the interleaving, right?
|
[12:53] pieter_hintjens
|
sub 2 will get A1, A2, B1, B2
|
[12:53] pieter_hintjens
|
etc.
|
[12:53] djc
|
but A1 will always come before A2?
|
[12:53] pieter_hintjens
|
yes
|
[12:53] djc
|
yeah, I don't think that's a problem
|
[12:54] pieter_hintjens
|
so don't use a device, it'll add complexity for nothing
|
[12:54] pieter_hintjens
|
unless you want to make publishers invisible to subscribers, then it can help
|
[12:54] djc
|
well, everything is flowing through a single process
|
[12:55] pieter_hintjens
|
what does that mean?
|
[12:55] pieter_hintjens
|
you just said you have N publishers and N subscribers :)
|
[12:55] djc
|
yeah, but there's a single process handling the connection
|
[12:56] djc
|
it's a WebSockets server that clients can connect to, subscribe to ZeroMQ streams and get JSON messages
|
[12:56] pieter_hintjens
|
djc: your explanation isn't helping, you need to draw a picture
|
[12:56] djc
|
sorry :)
|
[12:57] pieter_hintjens
|
'flowing through' means, to me, 0MQ sockets talking to each other, but you're not explaining that
|
[12:57] pieter_hintjens
|
so, confusion
|
[12:57] djc
|
is there like a pastebin for drawing graphs? :P
|
[12:58] pieter_hintjens
|
hmm, that'd be a neat service... I use ditaa for graphs but there's no pastebin for that
|
[12:58] headzone
|
http://www.lucidchart.com/pages/
|
[12:59] djc
|
using gdocs for now...
|
[13:02] djc
|
https://docs.google.com/drawings/pub?id=1n2IFhfm-mO2mCAzwHLk4-tESDG9D9mCxbIaKAlKgckc&w=960&h=720
|
[13:02] djc
|
is what I'm thinking of
|
[13:03] pieter_hintjens
|
headzone: sweet god that's amazingly well done...
|
[13:04] pieter_hintjens
|
djc: where is the "single process"?
|
[13:04] djc
|
it's the big fat thing in the middle
|
[13:04] pieter_hintjens
|
multithreaded?
|
[13:04] djc
|
if needed, yes
|
[13:04] pieter_hintjens
|
well, if multithreaded, it's effectively N processes
|
[13:05] pieter_hintjens
|
not relevant to discussion, really
|
[13:05] djc
|
so that's why I said it was N processes :P
|
[13:05] pieter_hintjens
|
anyhow, yes, this'll work, you don't need any forwarder device afaics
|
[13:06] djc
|
I just said it was flowing through a single process to explain that I might use a forwarder to forward stuff from local to inproc, so I don't need three or four streams of the same over my LAN
|
[13:06] djc
|
(the left side)
|
[13:06] pieter_hintjens
|
what's the message size/rate you expect?
|
[13:08] djc
|
the sizes are not very impressive, but I expect a fairly rate
|
[13:08] djc
|
+high
|
[13:08] djc
|
not sure how high, exactly
|
[13:09] djc
|
zmq doesn't handle that for me, right? like, it doesn't multiplex for multiple subscribers subscribing to the same publisher within the same context?
|
[13:28] djc
|
would actually be cool if it could do that
|
[13:29] th
|
what does a "what(): Bad address" mean when doing recv()? i could understand that for bind/connect
|
[13:29] th
|
this happens for me in c++ with 2.1.5 and it did not happen in 2.1.4
|
[13:34] th
|
hmmm #define EFAULT 14 /* Bad address */
|
[13:36] th
|
http://api.zeromq.org/2-1:zmq-recv tells me EFAULT can mean invalid socket or message; and the changelog for 2.1.5 mentions "Checks zmq_msg_t validity at each operation"
|
[13:42] mikko
|
i'm back
|
[13:42] mikko
|
making moroccon lamb stew
|
[13:44] mikko
|
djc: which os?
|
[13:51] djc
|
mikko: gentoo linux
|
[13:52] mikko
|
ok, which version of OpenPGM?
|
[13:53] mikko
|
and can you run with ./configure --disable-silent-rules
|
[13:53] mikko
|
so that i can see the output of linking
|
[13:54] djc
|
5.1.115
|
[13:54] djc
|
I'll try that
|
[13:55] mikko
|
it sounds like pkg-config for openpgm might fail
|
[13:55] mikko
|
there is currently no check for that
|
[13:56] pieter_hintjens
|
djc: I'm back, sorry
|
[13:57] pieter_hintjens
|
currently zmq sub sockets will multiplex to all sub sockets
|
[13:57] pieter_hintjens
|
there's a work in progress to be smarter about this but it's not got a lot of traction
|
[13:58] pieter_hintjens
|
using an inprocess frontend device is an obvious optimization
|
[13:58] pieter_hintjens
|
http://zguide.zeromq.org/page:all#High-speed-Subscribers-Black-Box-Pattern
|
[13:59] pieter_hintjens
|
th: hi
|
[13:59] pieter_hintjens
|
I think we have a problem in the 2.1.5 release
|
[13:59] pieter_hintjens
|
but it's not clear yet
|
[14:00] mikko
|
what problem?
|
[14:00] pieter_hintjens
|
tests and other code are failing due to last commit which checks msg validity
|
[14:00] pieter_hintjens
|
make check-TESTS
|
[14:01] pieter_hintjens
|
make[2]: Entering directory `/vol1/home/ph/work/zeromq2-1/tests'
|
[14:01] pieter_hintjens
|
terminate called after throwing an instance of 'zmq::error_t'
|
[14:01] pieter_hintjens
|
what(): Bad address
|
[14:01] pieter_hintjens
|
/bin/bash: line 5: 883 Aborted (core dumped) ${dir}$tst
|
[14:01] pieter_hintjens
|
FAIL: test_pair_inproc
|
[14:01] mikko
|
http://build.zero.mq/view/libzmq%202.1/ shows all ok
|
[14:01] mikko
|
strange
|
[14:01] mikko
|
is the commit after this morning build?
|
[14:02] djc
|
mikko: http://paste.pocoo.org/raw/375143/
|
[14:03] pieter_hintjens
|
mikko: I applied a patch from sustrik after this mornings' build
|
[14:03] mikko
|
djc: the whole output if you can
|
[14:04] mikko
|
djc: as that linking failure is for perf programs
|
[14:04] mikko
|
would be interesting if -lpgm is skipped for libzmq.la for some reason
|
[14:05] djc
|
http://dirkjan.ochtman.nl/files/build.log
|
[14:06] pieter_hintjens
|
sustrik: ping, are you around?
|
[14:09] mikko
|
djc: strange
|
[14:09] mikko
|
can you pastebin the output of 'nm' for openpgm.so
|
[14:14] th
|
a simple chain of new zmq::context_t(1); new zmq::socket_t(*context, ZMQ_XREP); bind("tcp://127.0.0.1:5555"); zsock->recv(&message); results in "Bad address"
|
[14:14] th
|
this makes 2.1.5 somewhat unusable at least for me here
|
[14:14] mikko
|
Please do *not* use the 2.1.5 package released today, the zmq_msg_t
|
[14:14] mikko
|
validity checking appears to be reporting errors on valid code.
|
[14:15] mikko
|
peter's mail 8 minutes ago
|
[14:19] th
|
oh - i did not even notice that this release was from today
|
[14:20] th
|
i just checked the page, because someone told me a bug was fixed
|
[14:21] djc
|
mikko: nm finds no symbols in libpgm.so
|
[14:21] djc
|
..
|
[14:21] pieter_hintjens
|
th: so sorry, I've reverted the download page to point to 2.1.4
|
[14:21] th
|
pieter_hintjens: but not the changelog;)
|
[14:21] pieter_hintjens
|
th: I'll push a clean git history (aaagh) in a second
|
[14:22] th
|
pieter_hintjens: is there any recommended git revision > 2.1.4?
|
[14:22] pieter_hintjens
|
th: nope
|
[14:22] pieter_hintjens
|
take https://github.com/zeromq/zeromq2-1/commits/v2.1.4
|
[14:23] pieter_hintjens
|
or take the package
|
[14:23] th
|
pieter_hintjens: which is the same as ... ok
|
[14:23] th
|
pieter_hintjens: that still has the multipart intermixing issues
|
[14:23] th
|
under higher load
|
[14:23] pieter_hintjens
|
th: I'm not familiar with that bug, and do not think it's been downstreamed to 2.1.5 at all
|
[14:24] djc
|
pieter_hintjens: you shouldn't unpublish changes from a public repo
|
[14:25] pieter_hintjens
|
djc: well, I shouldn't allow buggy code to get into a stable distro
|
[14:25] th
|
pieter_hintjens: i'll see if i can make the test-case smaller and more readable. it's 100% reproducable already for me. but it needs a './server & ; while ./client ; do;done'
|
[14:25] djc
|
pieter_hintjens: so publish a new revision
|
[14:26] djc
|
or even a new tag
|
[14:26] pieter_hintjens
|
djc: ok, I'll respect the rules for once
|
[14:26] pieter_hintjens
|
question then
|
[14:26] pieter_hintjens
|
2.1.6 or 2.1.5 or 2.1.5.1?
|
[14:26] djc
|
if you're taking 2.1.5 and just removing on buggy cset, call it 2.1.5.1
|
[14:26] djc
|
s/on/one/
|
[14:27] pieter_hintjens
|
ok, that's what I'd done before sustrik asked me to pause
|
[14:40] pieter_hintjens
|
sustrik: ping, are you around?
|
[15:20] mikko
|
th: no symbols at all?
|
[16:30] sustrik
|
re
|
[16:31] sustrik
|
pieter_hintjens: sorry, the reply abnout the issue was accidentally sent only to artem
|
[16:31] sustrik
|
let me forward it to the list
|
[16:39] pieterh
|
sustrik: thx
|
[16:40] pieterh
|
sustrik: seems we have a problem with cherry-picking
|
[16:40] pieterh
|
it does not work reliably
|
[16:42] sustrik
|
np
|
[16:42] pieterh
|
what happened was this...
|
[16:43] pieterh
|
well, yes, we do have a problem somewhere... :-/
|
[16:43] pieterh
|
when I apply the commit, it gives a conflict
|
[16:43] pieterh
|
there should be no conflict
|
[16:43] pieterh
|
clearly I'm not modifying these files anywhere else
|
[16:43] pieterh
|
incorrect conflict resolution -> garbage
|
[16:44] pieterh
|
anyone with sufficient git fu to help figure this out?
|
[16:45] pieterh
|
the only way I can see now to fix this is to reset history to before the cherry-pick and do it again
|
[16:45] pieterh
|
this is really unpleasant
|
[16:46] headzone
|
what's the situation (in generic git terms)
|
[16:47] pieterh
|
headzone: ok, here's where I find myself...
|
[16:47] pieterh
|
working in github.com/zeromq/zeromq2-1, which fetches commits from github.com/zeromq/libzmq
|
[16:47] pieterh
|
'git fetch --no-tags libzmq' followed by cherry-picking of specific commits
|
[16:48] pieterh
|
one of these manages to fail, partially and silently, on zmq.cpp
|
[16:48] sustrik
|
the conflict is presumably between changes that were not backported and those that were backported
|
[16:49] sustrik
|
in one of the previous commits in libzmq i've moved message-related functions into a separate compilation unit
|
[16:49] sustrik
|
msg.hpp/cpp
|
[16:49] sustrik
|
that's what causing the conflict imo
|
[16:49] pieterh
|
sustrik: there wasn't even a conflict on this, in fact
|
[16:49] pieterh
|
http://pastebin.com/ddC7HfDH
|
[16:49] pieterh
|
yes, I got an error on msg.cpp
|
[16:49] pieterh
|
but the missing chunks were on zmq.cpp...
|
[16:50] pieterh
|
anyhow, I now have a bit of a mess in my zeromq2-1 repository
|
[16:50] sustrik
|
which contains functions that are in zmq.cpp in 2.x
|
[16:50] sustrik
|
backporting i shard
|
[16:50] pieterh
|
ah... so the fix is in a file I don't actually have...
|
[16:50] sustrik
|
you may try to apply the parch that moved the functions to msg.cpp
|
[16:51] sustrik
|
first
|
[16:51] sustrik
|
ack
|
[16:52] pieterh
|
sustrik: I really don't want to be creative with the libzmq code...
|
[16:52] pieterh
|
the changes that are missing in zmq.cpp are presumably few
|
[16:52] sustrik
|
let me have a look
|
[16:53] pieterh
|
sustrik: can you clone git@github.com:zeromq/zeromq2-1.git?
|
[16:53] sustrik
|
i did
|
[16:53] pieterh
|
nice
|
[16:53] pieterh
|
headzone: with luck sustrik will be able to help me patch this up and we can go forwards...
|
[16:54] headzone
|
it sounds like you're trying to cherry-pick something from a branch that's diverged from yours, which is usually not a great idea
|
[16:54] pieterh
|
yup
|
[16:54] pieterh
|
i should have caught this in testing, which will happen from now on
|
[16:54] pieterh
|
properly
|
[16:54] sustrik
|
ok, afaics the only thing you need to do is to apply the changes reported as been done to msg.cpp
|
[16:55] sustrik
|
to zmq.cpp instead
|
[16:55] pieterh
|
ok... let me review those
|
[16:56] pieterh
|
sustrik: I'll give that a shot, it should be doable
|
[16:56] sustrik
|
ok
|
[17:02] headzone
|
in future, you may also find it useful to use the -n option to cherry-pick or format-patch instead, to see and inspect how things will look first without messing up your tree
|
[17:04] pieterh
|
headzone: yes, will use -n option on cherry-pick
|
[17:05] pieterh
|
I assume after reviewing, one commits with 'git commit -c nnnnnn'?
|
[17:06] headzone
|
sure
|
[17:08] pieterh
|
thx, I'm keeping the idiot proof instructions here: http://www.zeromq.org/docs:distributions
|
[17:10] headzone
|
in the second part, "git pull source master; git fetch source" is better written "git fetch source; git merge source/master"
|
[17:13] pieterh
|
headzone: thanks, I'll fix it (you can also edit, it's a wiki)
|
[17:14] pieterh
|
sustrik: what code is producing this error...?
|
[17:15] pieterh
|
terminate called after throwing an instance of 'zmq::error_t'
|
[17:15] pieterh
|
what(): Bad address
|
[17:15] pieterh
|
Aborted (core dumped)
|
[17:15] pieterh
|
in the test cases?
|
[17:15] pieterh
|
after patching zmq.cpp, it fails exactly the same way, I can't trace where
|
[17:17] sustrik
|
try to run the failing test case under dgb
|
[17:17] sustrik
|
gdb
|
[17:17] sustrik
|
there's should be a way to say 'stop at exception'
|
[17:17] pieterh
|
not executable format? wtf?
|
[17:18] pieterh
|
agh
|
[17:18] sustrik
|
aha, Bad address is EFAULT
|
[17:19] sustrik
|
ie. message validation fails
|
[17:19] pieterh
|
I can't even see where the test executables are...
|
[17:19] sustrik
|
tests subdir
|
[17:19] pieterh
|
all I see are scripts that magically run the test executables
|
[17:20] th
|
mikko: symbols? you meant me?
|
[17:20] mikko
|
yes
|
[17:20] mikko
|
no
|
[17:20] mikko
|
i meant djc
|
[17:20] sustrik
|
i see, that's because it uses shared libs
|
[17:20] sustrik
|
so there's a script that changes paths
|
[17:20] sustrik
|
to load the right library
|
[17:20] sustrik
|
you can configure with --disable-shared
|
[17:20] sustrik
|
which will force the static linking
|
[17:21] sustrik
|
and thus remove the need for scripts
|
[17:21] pieterh
|
well, when I tried artem's fix, I didn't do any of that, and the fix seemed to work
|
[17:21] pieterh
|
now I can't get even a 'printf' to display
|
[17:21] sustrik
|
it will fail if the message is shared
|
[17:21] pieterh
|
sure, that's not my point
|
[17:21] pieterh
|
it's that I find myself stuck in a twisty maze of "am I running this version or not..."
|
[17:22] pieterh
|
I'll disable shared libraries
|
[17:22] sustrik
|
i do it that
|
[17:22] sustrik
|
way
|
[17:22] sustrik
|
you'll get executables with statically linked libzmq
|
[17:22] sustrik
|
directly in tests subdir
|
[17:23] pieterh
|
sure, makes sense but I never had to do that before
|
[17:25] pieterh
|
nah, this is just messed up
|
[17:25] pieterh
|
it aborts on zmq.hpp:278
|
[17:25] pieterh
|
the file only has 266 lines
|
[17:25] pieterh
|
somehow I'm getting versions totally mixed up :-(
|
[17:26] pieterh
|
mikko: if I run 'make' in zeromq2-1 root, it shouldn't be pulling in so's from elsewhere, should it...?
|
[17:27] pieterh
|
isn't this the day skynet takes control?
|
[17:28] pieterh
|
sustrik: for some godforsaken reason the tests won't build correctly for me
|
[17:36] mikko
|
pieterh: sorry?
|
[17:37] pieterh
|
mikko: sorry, skynet temporarily got past my cortical firewall, it's gone now...
|
[17:38] pieterh
|
i found out what the problem was, and it seems much better now
|
[17:41] pieterh
|
mikko: djc reported earlier today that --with-system-pgm didn't work in 2.1.5
|
[17:41] mikko
|
yep
|
[17:41] pieterh
|
do you want me to hold off with 2.1.5.1?
|
[17:41] mikko
|
please
|
[17:42] mikko
|
i'll look into it tonight
|
[17:42] pieterh
|
ok, np
|
[17:42] pieterh
|
btw we can also get that redhat spec file into it if you have it
|
[17:42] pieterh
|
depends how long your nights are... :)
|
[17:55] mikko
|
that change is pretty trivial
|
[18:00] pieterh
|
everything is trivial to the right brain :-)
|
[18:05] mikko
|
i need to take a small break
|
[18:05] mikko
|
juggling million things at the moment
|
[18:05] mikko
|
bbl
|
[18:06] pieterh
|
cyl
|
[18:10] guido_g
|
pieterh: do you have an idea what a taxi ride from the airport to the meeting place might cost?
|
[18:11] pieterh
|
guido_g: I'll add a travel guide to the wiki page, taxis are extortionate
|
[18:11] pieterh
|
it'
|
[18:11] pieterh
|
it's easy to get to by public transport
|
[18:11] guido_g
|
ok
|
[18:12] guido_g
|
just wanted to know if i could do it the lazy way :)
|
[18:18] jond
|
pieterh: are master and 2.1 repo still bust with the message check bug?
|
[18:19] pieterh
|
jond: libzmq master was always working, 2.1 master is now in theory working again
|
[18:20] jond
|
pieterh : so how did the patch for libzmq -> 2.1 go wrong then? does it have to be applied and fixed manually?
|
[18:21] pieterh
|
libzmq had diverged and the patch was not usable as-such, and my resolution of that error was wrong
|
[18:22] jond
|
pieterh: ok understood, just wanted to be clear, so essentially patches to libzmq master now may have to be manually reworked for the backport.
|
[18:23] pieterh
|
jond: not until now, no
|
[18:23] pieterh
|
but this seems to be inevitable as 3.0 diverges
|
[18:23] pieterh
|
I guess we were kind of expecting it but missed it when it actually happened :-(
|
[18:25] jond
|
pieterh: yeah, looked bad on the list....but stuff happens
|
[18:25] pieterh
|
well, stuff happens... I don't mind looking bad, so long as we don't make the same mistakes twice
|
[18:26] pieterh
|
Its far worse to not make releases out of fear of getting it wrong
|
[18:40] guido_g
|
ohh... an irish pub, how pleasant
|
[18:47] pieterh
|
guido_g: an irish pub wandered into your office?
|
[18:48] guido_g
|
ah no
|
[18:48] guido_g
|
a) i'm at home
|
[18:48] guido_g
|
and b) was looking at brussels w/ google-earth
|
[18:48] traviscline
|
just pushed some further cython optimizations to gevent-zeromq's HEAD please report any issues if they occur
|
[19:07] pieterh
|
guido_g: :) irish pubs in brussels are crap (my word of the day)
|
[19:08] pieterh
|
the best pubs are the authentically ethnic ones
|
[19:08] guido_g
|
ok
|
[19:08] pieterh
|
hehehe
|
[19:09] pieterh
|
I meant, the Irish Pubs are fantastic!
|
[19:09] pieterh
|
brussels imports all its beer from dublin
|
[19:09] pieterh
|
srsly
|
[19:10] guido_g
|
you should be happy that i'm not the average german
|
[19:10] guido_g
|
you know how picky they are wrt beer
|
[19:18] pieterh
|
guido_g: I once had a GF from Bavaria... sigh...
|
[19:18] pieterh
|
beautiful but beer crazy
|
[19:18] pieterh
|
this is very relevant to #zeromq somehow
|
[19:19] pieterh
|
btw I put up some transport tips on the unconf page
|
[19:19] guido_g
|
would be great!
|
[19:19] pieterh
|
see if they make even random sense
|
[19:19] guido_g
|
will do
|
[19:20] pieterh
|
guido_g: I was asserting that beer *is* relevant to zeromq
|
[19:20] guido_g
|
i can confirm that
|
[19:21] jond
|
pieterh: only if it passes certain purity and strength tests!
|
[19:21] pieterh
|
jond: purity and strength are the defining characteristics of 0MQ!
|
[19:22] pieterh
|
maybe I can change the 0MQ website logo to "Purity and Strength!" ?
|
[19:22] pieterh
|
nah, people would think we're a cult of some kind
|
[19:22] pieterh
|
which is totally not the case
|
[19:24] guido_g
|
does sacrificial blood count as liquid when flying?
|
[19:25] pieterh
|
guido_g: only if it's not yours afaik
|
[19:25] guido_g
|
no problem, can collect some at the airport
|
[19:26] pieterh
|
they sell it in Brussels, np
|
[19:27] guido_g
|
what a town
|
[19:27] pieterh
|
not sure if it's entirely within the Reinheitsgebot
|
[19:27] pieterh
|
probably got added chocolate or somesuch
|
[19:27] guido_g
|
hehe
|
[19:53] guido_g
|
n8 all!
|
[19:54] pieterh
|
bye guido_g
|
[19:54] phpcodemonkey
|
hi all, am here looking for help with 0mq / php issue - I'm running 2.1.4 / PHP pecl zmq-beta 0.7.0 on PHP 5.3.6, x86_64 Fedora 14: I'm getting a segfault when running the basic hwclient.php from https://github.com/imatix/zguide/blob/master/examples/PHP/hwclient.php the related hwserver.php runs fine sits there waiting for client connections. I've posted a backtrace at: http://pastebin.com/ruEzjAdT anyone got any ide
|
[19:55] pieterh
|
phpcodemonkey: there seems to be an error on a uid generating function
|
[19:55] pieterh
|
you might need to install libuuid or similar
|
[19:56] pieterh
|
gen_uuid.c: No such file or directory.
|
[19:56] phpcodemonkey
|
k
|
[19:57] phpcodemonkey
|
already got: libuuid-devel-2.18-4.8.fc14.x86_64 rpm installed
|
[19:58] pieterh
|
that should be sufficient IMO
|
[19:59] pieterh
|
mikko should be able to help, he's the guy who wrote the PHP binding
|
[19:59] pieterh
|
he'll be around later on this evening normally
|
[19:59] phpcodemonkey
|
provides uuid.h in /usr/include so all seems to be correct
|
[19:59] phpcodemonkey
|
pieterh: he's already asked for a backtrace via twitter :)
|
[19:59] pieterh
|
phpcodemonkey: some of the uuid packages don't provide the right functions
|
[20:00] phpcodemonkey
|
nice
|
[20:01] pieterh
|
googling that error gives quite a lot of results
|
[20:01] pieterh
|
check if there are other uuid rpms
|
[20:01] mikko
|
here
|
[20:01] pieterh
|
:)
|
[20:01] mikko
|
enjoyed a bit of C++ dev on windows
|
[20:01] mikko
|
and when i say enjoy i mean sticking bamboo sticks under nails
|
[20:02] mikko
|
phpcodemonkey: have you got imagick / uuid pecl extension loaded?
|
[20:04] phpcodemonkey
|
imagick 3.0.1
|
[20:04] mikko
|
can you try loading zmq extension before imagick?
|
[20:04] mikko
|
this is a known unknown issue with imagemagick / uuid
|
[20:04] phpcodemonkey
|
e.g. change zeromq.ini to 0mq.ini?
|
[20:04] mikko
|
yes
|
[20:04] mikko
|
that should do the trick
|
[20:05] mikko
|
http://usrportage.de/archives/922-PHP-segfaulting-with-pecluuid-and-peclimagick.html
|
[20:05] mikko
|
it looks like this issue
|
[20:05] mikko
|
i have no idea what is going on in there
|
[20:05] mikko
|
seems to be some global state inside uuid lib or so
|
[20:05] phpcodemonkey
|
mikko: you rock, that was it!
|
[20:05] phpcodemonkey
|
perhaps that out to be in the docs somewhere?
|
[20:06] mikko
|
sure, if you can add a git pull request ill merge it
|
[20:07] phpcodemonkey
|
git != svn ;)
|
[20:12] mikko
|
pieterh: --with-system-pgm works for me (tm)
|
[20:13] pieterh
|
mikko: did djc provide details of what "doesn't work" means for him?
|
[20:13] mikko
|
yes
|
[20:13] mikko
|
fails to link
|
[20:13] mikko
|
and nm on openpgm.so doesnt show _any_ symbols
|
[20:13] pieterh
|
ah
|
[20:13] mikko
|
djc: are you still here?
|
[20:13] pieterh
|
empty library or somesuch
|
[20:14] pieterh
|
mikko: nice catch on phpcodemonkey's problem there
|
[20:14] mikko
|
thanks
|
[20:15] phpcodemonkey
|
definitely, was frustrating the heck out of me!
|
[20:16] phpcodemonkey
|
though having read the blog post you referenced, it seems that 0mq and imagick on their own do the damage, don't have pecl/uuid installed
|
[20:37] mikko
|
phpcodemonkey: it's any extension using uuid functions + imagick
|
[20:38] phpcodemonkey
|
mikko: figured that, was just confirming :)
|
[20:47] mikko
|
pieterh: want me to push the spec file change directly?
|
[20:47] pieterh
|
mikko: sure, that'd be simplest
|
[20:49] mikko
|
ok, will push soon(ish)
|
[20:49] mikko
|
i'll make sure that the change makes it midnight's rpm builds
|
[20:58] mikko
|
pieterh: can you test 2.1?
|
[20:58] pieterh
|
you mean a normal make check?
|
[20:59] pieterh
|
ok, running a full build / test process
|
[21:00] mikko
|
if make dist works it should be fine
|
[21:00] mikko
|
i'll fix small bug in ztools and run rpm builds now
|
[21:02] pieterh
|
mikko: it all seems to work, with your changes
|
[21:06] mikko
|
build running
|
[21:06] mikko
|
bbr
|
[21:06] mikko
|
brb
|
[21:21] mikko
|
pieterh: seems to be ok
|
[22:53] coopernurse
|
hi zeromq folks. anyone have a few minutes to chat with a zeromq newb? I have some basic architectural questions. I have read the whitepaper. I'm coming from a traditional broker based background, so I'm trying to unlearn some concepts.
|
[23:04] traviscline
|
coopernurse: irc etiquette is "to ask, not to ask to ask"
|
[23:04] traviscline
|
*cough cough* it's mostly so people don't accidentally get roped into questions they can't answer and look wrong :P
|
[23:05] coopernurse
|
ok, great
|
[23:06] coopernurse
|
I just read this: http://www.zeromq.org/whitepapers:brokerless
|
[23:07] coopernurse
|
and I want to make sure I understand the suggested approach
|
[23:07] coopernurse
|
is it correct to consider zeromq a toolkit for building your own message routers?
|
[23:07] coopernurse
|
for example, in the distributed broker diagram
|
[23:08] coopernurse
|
Q1, Q2, and Q3 are daemons that I would write correct? that would encapsulate whatever routing topology was relevant
|
[23:21] jond
|
coopernurse: I think reading the guide will be better, some of the whitepapers are quite old ; http://zguide.zeromq.org/page:all
|
[23:22] coopernurse
|
jond: I've read through that as well (about a month ago). Nothing I've read talks about suggested deployment architectures.
|
[23:23] coopernurse
|
For example, I want to have a conversation with some operations folks about this
|
[23:23] coopernurse
|
And they think in terms of redundancy and points of failure
|
[23:24] jond
|
coopernurse: well in that case , it would be better to be on here on CET timezone. Though yes with zeromq you could build custom routers
|
[23:25] coopernurse
|
The whitepaper was the closest thing I've seen so far to discussing deployment topologies
|
[23:25] coopernurse
|
I understood the suggestion to potentially use a directory service for components to locate queue endpoints
|
[23:26] coopernurse
|
But I was unclear whether the Q1, Q2, etc were provided out of the box by zeromq, or were daemons you write yourself
|
[23:26] coopernurse
|
I'm assuming it's something I would have to write
|
[23:26] jond
|
there also is some stuff in the later chapters of the guide which may have been added recently: It would be best to hook up with pieterh on here
|
[23:26] coopernurse
|
ok
|
[23:27] coopernurse
|
but in general is it correct to view zeromq as a toolkit for creating your own messaging topologies?
|
[23:27] coopernurse
|
or are there higher level turnkey solutions that I have missed that come with the distribution?
|
[23:29] jond
|
well I've just looked atb the whitepaper, if you look at the stuff below the distributed broker which clicks to a chat example which is now marked as deprecated, so the best place is gonna be the guide and then irc or the mailing list
|
[23:29] headzone
|
coopernurse: funnily enough, we have just been debating that question
|
[23:29] coopernurse
|
jond: ok thanks
|
[23:30] coopernurse
|
headzone: oh yeah?
|
[23:31] headzone
|
coopernurse: the current thinking appears to be that the goal of 0mq is to provide standard/pre-packaged known-good messaging patterns (topologies), rather than be a toolkit that users use to build their own custom ones
|
[23:32] coopernurse
|
ah, interesting. and if that's the case, would those pre-packaged patterns take the form of daemons you can run out of the box?
|
[23:32] headzone
|
no, they take the form of the patterns supported by the sockets, e.g. req/rep, pub/sub
|
[23:33] coopernurse
|
ok, so this is where I really just don't get it
|
[23:33] coopernurse
|
perhaps I need to rtfm more carefully
|
[23:33] headzone
|
with maybe some "devices" on top at a higher layer (the things currently called queue/forwarder/streamer)
|
[23:33] jond
|
headzone: I think you can use it for both, but in some areas that is quite fluid. There was a concept of devices -> queue, forwarder, streamer but they are being moved out of the core
|
[23:34] coopernurse
|
ok, so "devices" is the term I probably need to be examining in the guide more closely
|
[23:34] headzone
|
coopernurse: the reason this is confusing at the moment is that the current state of 0mq is sort of stuck in the middle of "standard patterns" and "toolkit for building custom application-layer routing topologies", for historical reasons
|
[23:35] coopernurse
|
presumably the Q1, Q2, etc in the (perhaps deprecated?) whitepaper are potentially devices zeromq provides
|
[23:35] coopernurse
|
ok
|
[23:36] jond
|
coopernurse: the guide doesnt mention them much and it also uses this libzapi which sits atop 0mq; pieterh is the one behind all this
|
[23:36] headzone
|
coopernurse: i found this out by assuming the goal was to become a generic application-layer routing topology construction kit and suggesting a protocol stack that would accomplish this in a sane manner
|
[23:38] coopernurse
|
As someone who is coming from ActiveMQ/RabbitMQ/etc it's difficult to find analogs in zeromq
|
[23:38] headzone
|
coopernurse: but it is far from clear that that is a desirable goal to aim for, because it amounts to recapitulating what the IP network and transport layers already provide
|
[23:38] coopernurse
|
headzone: yes, I'm just trying to wrap my head around how an application developer could use zeromq to transition a monolithic web app into a set of more loosely coupled components
|
[23:39] coopernurse
|
I agree with the "broker is a single point of failure" stuff in the zeromq literature. That got me interested.
|
[23:39] coopernurse
|
But I'm not sure how I avoid writing lots of little devices that are each single points of failure
|
[23:40] headzone
|
right, and basically you do that by combining the standard messaging patterns, the "device" functions (which are really just commonly used pieces of application logic), and application logic to handle anything else that's required
|
[23:41] coopernurse
|
headzone: ok, thanks. I appreciate your time.
|
[23:41] headzone
|
regarding redundancy, i happen to think that solving that problem in a general way inexorably leads you to reinventing routing
|
[23:42] coopernurse
|
is there a recommended way to do it in zeromq?
|
[23:42] coopernurse
|
(redundancy that is)
|
[23:42] headzone
|
well, there are lots of examples in the guide
|
[23:43] coopernurse
|
ok
|
[23:43] headzone
|
a fair portion of it is about the hoops you have to jump through to get reliability
|
[23:44] coopernurse
|
all of it seems very point to point though
|
[23:44] headzone
|
it's also all very ad-hoc
|
[23:44] coopernurse
|
all the examples I've seen seem to assume the components know who to talk to
|
[23:45] coopernurse
|
I'm looking at the "Designing Reliability" section again now
|
[23:48] coopernurse
|
ok, well thanks headzone and jond. I'll keep reading. I'm off.
|