[Time] Name | Message |
[02:16] Jay7
|
Hey I'm new to ZMQ and tried installing in on a Mac OSX 10.6.3; but it appears --with-pgm is not supported currently on darwin. Is this expected in the near future by chance?
|
[02:28] cremes
|
Jay7_: i'm not a pgm expert, but apparently the openpgm package (a separate project from 0mq) doesn't have good/any osx support
|
[02:29] cremes
|
Jay7_: check the mailing list archives for more discussion on this
|
[02:29] Jay7
|
@cremes_: ah, okay. Thank you. I'll look through those.
|
[07:51] pieterh
|
sustrik: hi
|
[07:53] rem7
|
Hi, I was going over the zmq guide, I noticed the examples use strings as messages. If I want to send something like floats is it ok to just memcpy the float, or should I use something like protobuffers?
|
[07:53] pieterh
|
rem7, depends on your needs
|
[07:54] pieterh
|
sending data as-such is easy and cheap but doesn't give you any kind of leverage
|
[07:54] rem7
|
I just need to pass float arrays... using a PUSH/PULL system.
|
[07:55] pieterh
|
if it's a throw-away app, go for it
|
[07:55] pieterh
|
if it's something you are more serious about, make formal 'contracts'
|
[07:55] pieterh
|
i.e. serialize your data properly (portably), document the message formats, etc.
|
[07:55] pieterh
|
it's a lot more work up front but lets you work more cheaply long term
|
[07:56] pieterh
|
e.g. plug in a worker in a different language
|
[07:56] rem7
|
would using google proto buffers take care of that?
|
[07:57] rem7
|
I don't have much experience with this kinda of stuff...
|
[08:15] pieterh
|
rem7, yes, it would
|
[08:18] rem7
|
pieterh_: I guess I'll start looking into that. Thanks.
|
[08:18] pieterh
|
let us know how it goes
|
[08:20] guido_g
|
smaller alternative might be http://msgpack.org/
|
[08:22] rem7
|
guido_g: thanks, I'll look into it
|
[08:42] sustrik
|
pieterh_: morning
|
[08:56] pieterh
|
sustrik, hi
|
[08:58] pieterh
|
do you think the red theme is readable enough for the man pages?
|
[10:28] pieter_hintjens
|
anyone looked at distributed hash tables on top of 0MQ, e.g. Chord?
|
[10:52] travlr
|
hi all. i've got a question about interfacing zmq in Qt having read up a lot on what's going on I'm still a little confused about zmq's edge-trigger....
|
[10:53] travlr
|
Qt has a QSocketNotifier class that provides a signal when there is activity on a profided FD
|
[10:53] travlr
|
like from zmq_getsockopt(... ZMQ_FD...)
|
[10:54] travlr
|
I'm assuming that the Qt sig is level triggered.
|
[10:56] travlr
|
so when I get a signal from QSocketNotifier, I check the ZMQ_EVENTS for ZMQ_POLLIN (in this case) and ....
|
[10:56] travlr
|
what do I do exactly when I get EAGAIN?
|
[11:02] travlr
|
I'm guessing that I need to keep calling zmq_recv() until I do get the EAGAIN condition.
|
[11:06] pieter_hintjens
|
travlr: sorry, can't really help, not sure what the edge trigger vs. level trigger is about
|
[11:06] pieter_hintjens
|
try invoking sustrik... :-)
|
[11:07] travlr
|
hi pieter.. ok, thanks just the same
|
[11:42] private_meta
|
Is it possible to download a certain language bound example-set (or all of them if necessary) without having to install the entire git suite?
|
[11:42] travlr
|
private_meta: each binding has its own git repo.. i think
|
[11:43] private_meta
|
I meant that one https://github.com/imatix/zguide/tree/master/examples
|
[11:43] private_meta
|
and/or the dedicated subdirectories/trees
|
[11:43] private_meta
|
brb
|
[11:44] travlr
|
no. but the that download isn't that big anyway
|
[12:05] sustrik
|
travlr: yes
|
[12:05] sustrik
|
you have to call recv() till you get EAGAIN
|
[12:06] private_meta
|
travlr: I don't want to download 40 files per language, it's a hassle
|
[12:15] mikko
|
good morning all
|
[12:16] sustrik
|
morning
|
[12:24] mikko
|
build failure on s390x
|
[12:24] mikko
|
but that is due to shutdown_stress
|
[12:35] pieter_hintjens
|
mikko: quick question
|
[12:39] pieter_hintjens
|
private_meta: it takes about 30 seconds to clone the git
|
[12:40] pieter_hintjens
|
and it's nice that people get the git because they can then translate the examples into other languages
|
[12:42] pieter_hintjens
|
private_meta: ah, you can click on 'Downloads' on the github page and get a .zip or .tar.gz. file
|
[12:45] mikko
|
pieter_hintjens: sure
|
[12:45] mikko
|
what is the question?
|
[12:45] pieter_hintjens
|
mikko: I have scripts that will rebuild api.zero.mq from the git repo
|
[12:46] pieter_hintjens
|
bash/perl/python
|
[12:46] mikko
|
do you want to run it automatically on daily builds?
|
[12:46] pieter_hintjens
|
at least for master, I think, yes
|
[12:46] mikko
|
of course
|
[12:46] pieter_hintjens
|
also for any branch that is being maintained...
|
[12:46] mikko
|
is the script available somewhere?
|
[12:47] mikko
|
thats perfectly fine, the box is idle apart from the 5am/pm builds
|
[12:47] pieter_hintjens
|
I guess the question is whether the scripts should be in their own git, the zeromq2 git, or just floating around somewhere
|
[12:47] pieter_hintjens
|
eventually we could do the same for the zfl man pages
|
[12:47] mikko
|
well, i think we could have git for random infra stuff
|
[12:47] pieter_hintjens
|
ztools?
|
[12:47] mikko
|
i got some scripts on the build server that could be there as well
|
[12:48] mikko
|
which are not available anywhere now etc
|
[12:48] pieter_hintjens
|
indeed
|
[12:49] pieter_hintjens
|
so a new git that collects all the random tools we make?
|
[12:49] mikko
|
it wouldn't probably something usable for people out of the box
|
[12:49] mikko
|
but just to make sure that everything is available
|
[12:49] pieter_hintjens
|
right
|
[12:49] pieter_hintjens
|
in the zeromq organization would be best
|
[12:50] mikko
|
yeah, probably
|
[12:50] pieter_hintjens
|
git@github.com:zeromq/ztools.git
|
[12:51] mikko
|
cool, i'll add things there later
|
[12:51] pieter_hintjens
|
I'll whip up a README, I suggest one directory per tool
|
[12:51] mikko
|
yeah
|
[12:51] mikko
|
makes sense
|
[12:51] mikko
|
lunch, brb
|
[12:53] pieter_hintjens
|
ok, you should have rw access
|
[12:53] pieter_hintjens
|
cyl
|
[12:53] private_meta
|
pieter_hintjens: the download page was empty for me
|
[12:54] pieter_hintjens
|
private_meta: ? it should pop up a window asking if you want a .zip or .tar.gz...
|
[12:54] pieter_hintjens
|
what URL did you get?
|
[12:54] pieter_hintjens
|
You want to be here: https://github.com/zeromq/zfl
|
[12:54] pieter_hintjens
|
then click on Downloads
|
[12:55] pieter_hintjens
|
sorry!
|
[12:55] pieter_hintjens
|
https://github.com/imatix/zguide
|
[12:56] private_meta
|
pieter_hintjens: "should" is such a nice word, covers all kinds of possibilities, mostly the "not" possibility for me
|
[12:56] Guthur
|
it works fine here
|
[12:57] pieter_hintjens
|
private_meta: please explain in more detail what you do, and what happens
|
[12:57] pieter_hintjens
|
"it doesn't work" is not useful...
|
[13:01] pieter_hintjens
|
ok, lunch, bbl
|
[13:04] private_meta
|
yay... demonstrative effect... had "no downloads" or something like that for half a week, now I ASK about it nd it shows downloads
|
[13:05] private_meta
|
I hate that
|
[13:07] Guthur
|
well, at least it's working, hehe
|
[13:13] private_meta
|
I'm not sure anymore, someone might have told me already, but where in the guide can I find some info about how to implement heartbeating. according to the guide it has to do something with zmq_poll, but I don't find anything helpful on zmq_poll. I want to do something like heartbeat pings, but I'm not sure how to do it exactly
|
[13:19] mikko
|
pieter_hintjens: http://snapshot.zero.mq/zfl/
|
[13:23] mikko
|
private_meta: normally you would send a heartbeat request to client
|
[13:23] mikko
|
zmq_poll for incoming messages
|
[13:24] mikko
|
no message in given timeout -> consider dead
|
[13:25] private_meta
|
I don't quite understand zmq_poll and how to use it yet. I'm reading the man page, but i need to see how to be able to use the zmq_poll system.
|
[13:26] private_meta
|
So the heartbeat request would be a normal message that's recognizable as a heartbeat for the other side I assume?
|
[13:26] mikko
|
see "Handling Multiple Sockets
|
[13:26] mikko
|
in the zguide
|
[13:27] mikko
|
yes, or even a separate heartbeat socket
|
[13:27] mikko
|
depends on the application
|
[13:28] private_meta
|
A separate socket might be an interesting idea, thanks. I'll read up the multiple sockets first
|
[13:29] mikko
|
for example if you got PUSH/PULL as main socket you can't really heartbeat over it
|
[13:31] private_meta
|
So do push/pull stand for dedicated one-way-communication endpoints?
|
[13:32] mikko
|
http://api.zero.mq/2-1-0:zmq-socket
|
[13:32] mikko
|
you can see the characteristics of each socket type from here
|
[13:32] private_meta
|
i hate surfing on gprs
|
[13:32] private_meta
|
thanks
|
[13:32] mikko
|
whether they force a pattern, uni-directional/bi-directional
|
[13:32] mikko
|
if you got zeromq installed the man pages should work as well
|
[13:33] private_meta
|
thought as much
|
[13:35] mikko
|
XREP/XREQ is probably the most suitable socket for heartbeating
|
[13:37] private_meta
|
Well, I have 1 server, multiple clients, and I need to address these clients separately and each of these clients need to check heartbeats, or at least know if a connection terminates
|
[13:37] private_meta
|
while these connections are bidirectional
|
[13:39] mikko
|
yep, definitely XREQ/XREP
|
[13:39] mikko
|
i think zguide has an example of custom routing
|
[13:39] mikko
|
which is needed here
|
[13:39] private_meta
|
so the normal req/rep is balanced automatically all the time?
|
[13:40] private_meta
|
or is xreq and xrep as well, normally?
|
[13:41] private_meta
|
I have a slight difficulty understanding the terminology for the custom routing I find in the guide. dealer, mama, papa, router. Assuming I need one of those, I don't know what these terms actually mean
|
[13:42] private_meta
|
ah...think i found the explanation
|
[13:44] private_meta
|
so i'd need router-to-router,
|
[13:45] mikko
|
private_meta: xrep/xrep doesnt force request-reply pattern as strictly as req/rep
|
[13:45] mikko
|
in your case you might want "req client 1, req client 2, req client 3", "wait responses from all"
|
[13:45] mikko
|
i guess
|
[13:45] private_meta
|
uhm... train station coming up, brb
|
[13:45] mikko
|
not necessarily req client 1, wait for rep client 1, req client 2
|
[13:45] mikko
|
etc
|
[13:46] mikko
|
pieterh_: snapshots for zfl are now generated during builds
|
[13:57] private_meta
|
mikko: According to some of the terminology: Server-to-N-Clients connection, no forced request-to-reply pattern and standard-async-but-sync-by-choice (by higher level wrapper)
|
[13:57] private_meta
|
Is what I need
|
[13:58] private_meta
|
and non-balanced directly addressed messaging
|
[14:04] private_meta
|
mikko: concerning multiple sockets, Actually I want the connections incoming at the SAME socket, like all connection incoming at tcp port (e.g.) 5555. Still I want to address all connected clients separately
|
[14:10] pieterh
|
private_meta, see LRU routing in Ch3 of the Guide
|
[14:10] pieterh
|
mikko: nice, snapshots
|
[14:12] private_meta
|
pieterh_: you mean as in "lru uses several clients/workers connecting to the same socket"?
|
[14:12] pieterh
|
private_meta, yes, it gives you the basic framework
|
[14:12] pieterh
|
you will want to do your own routing, not LRU
|
[14:13] private_meta
|
Yes
|
[14:13] private_meta
|
pieterh_: do you know why the c++ version uses void* instead of char* for messages??
|
[14:14] pieterh
|
private_meta, no idea...
|
[14:14] private_meta
|
kk
|
[14:18] private_meta
|
hmm... 3g networks are way too spotty here outside of towns... it's a hassle working in trains :/
|
[14:21] private_meta
|
pieterh_: not quite sure I understand correctly, for demonstrative purposes, in the lru example the client and server code is in one file and only executed in threads so you don't have separate server/client files?
|
[14:22] pieterh
|
private_meta, yes, it's just to simplify the exercise
|
[14:22] private_meta
|
ok
|
[14:22] pieterh
|
the code in each thread is the same as it would be in a separate process
|
[14:22] pieterh
|
with a little less wrapping
|
[14:22] private_meta
|
Thanks!
|
[14:30] private_meta
|
hmm... the lru example on git seems pretty different from the guide example
|
[14:32] pieterh
|
private_meta, what URL?
|
[14:32] pieterh
|
are you looking at the same two languages?
|
[14:33] private_meta
|
Ah, let me look a second
|
[14:34] private_meta
|
Ah, i thought i was looking at the lruqueue example in the guide, it was rtmama
|
[14:34] private_meta
|
Sorry
|
[14:35] private_meta
|
mama is not ideal for me i guess, everyone should be able to initiate a message
|
[16:12] mikko
|
private_meta: it's void* because it's not really an array of characters
|
[16:12] mikko
|
rather a blob of data of size X
|
[16:42] private_meta
|
mikko: "just because you are sending strings over a line most of the time doesn't mean everyone does"
|
[16:42] private_meta
|
mikko: something like that?
|
[16:43] private_meta
|
I guess representing something other than a string AS a string might be a severe waste of space...
|
[16:48] mikko
|
private_meta: it's also conceptual thing
|
[16:48] mikko
|
private_meta: using the inproc transport you might send a pointer to a struct
|
[16:48] mikko
|
or just about anything
|
[16:48] mikko
|
rather than char*
|
[16:50] mikko
|
private_meta: but yes, you are right
|
[16:51] private_meta
|
I didn't think of using a message queue from an external library for in-process messaging
|
[16:51] private_meta
|
ah, between threads it might be handy i guess
|
[16:53] mikko
|
private_meta: http://www.zeromq.org/blog:multithreading-magic
|
[16:53] mikko
|
related post
|
[17:24] Guthur
|
imho inproc messaging is a big win
|
[17:24] Guthur
|
the multithreading magic article explains it best though
|
[17:47] andrewvc
|
anyone know if zmq_queue supports xreq/xrep or just rep/req
|
[17:56] sustrik
|
andrewvc: just xreq/xrep :)
|
[17:56] andrewvc
|
oh, awesome :)
|
[18:00] travlr
|
sustrik: thanks for the help earlier
|
[18:02] sustrik
|
you are welcome
|
[18:02] sustrik
|
travlr: if you feel the reference is unclear, feel free to suggest a patch
|
[18:05] travlr
|
i will. i'm just coming back to my work with zmq after having to totally focus on other stuff for a while.
|
[18:05] travlr
|
I also have to get the internal api bot working again at http://travlr.github.com/zeromq2
|
[18:06] travlr
|
as its broken atm
|
[18:07] Guthur
|
is the new API doc only beta at the moment
|
[18:07] Guthur
|
I'm just wondering why it's not linked from the main site
|
[18:08] sustrik
|
well, pieter have written the generator yesterday
|
[18:08] sustrik
|
that's why
|
[18:08] Guthur
|
ah, ok
|
[18:09] Guthur
|
it does seem very good though
|
[18:10] travlr
|
nice! l like the the new reference links from a simple index.
|
[18:10] sustrik
|
what's particularly nice is that all the released versions are covered
|
[18:10] travlr
|
yup
|
[18:20] andrewvc
|
Are there any examples of the XML for zmq_queue?
|
[18:21] andrewvc
|
the manpage is empty of course, and my google-fu is failing me
|
[18:26] andrewvc
|
nm, figured it out
|
[18:53] pieterh
|
sustrik, mikko and I will arrange to rebuild the docs once a day or so
|
[18:55] sustrik
|
pieterh_: great
|
[18:56] sustrik
|
if you do it twice a day, the docs will be in sync with latest snapshots
|
[18:56] pieterh
|
we've made a ztools git for random tools, incl the rebuild shells scripts
|
[18:56] sustrik
|
ack
|
[18:56] pieterh
|
let's see if people have more comments on the look & feel
|
[18:57] pieterh
|
i already toned it down, darker red, no logo on man pages, etc.
|
[18:57] sustrik
|
let me see
|
[18:57] sustrik
|
nice
|
[18:57] sustrik
|
btw, what about the stable branch, do you have any schedule for doing the fork?
|
[18:58] pieterh
|
next week sometime, I was thinking
|
[18:58] sustrik
|
great
|
[18:59] pieterh
|
ok, good!
|
[18:59] pieterh
|
people will be happy to see a new stable 2.1.x version
|
[18:59] pieterh
|
so many new goodies :-)
|
[18:59] sustrik
|
i think so
|
[18:59] sustrik
|
so will I
|
[18:59] pieterh
|
I'll keep the 2.0.x version of the Guide online as well
|
[19:00] sustrik
|
i like that all the versions are available
|
[19:00] sustrik
|
ah, the guide
|
[19:00] pieterh
|
yes, people have asked for that...
|
[19:00] pieterh
|
so I will update that for 2.1, and keep the 2.0 version for reference
|
[19:00] sustrik
|
i've checked it lately and it seemed to cover both 2.0 and 2.1
|
[19:01] pieterh
|
in several places it covers things that were changed
|
[19:01] pieterh
|
e.g. socket migration, socket shutdown
|
[19:01] pieterh
|
needs fixing for 2.1
|
[19:01] sustrik
|
right
|
[19:01] pieterh
|
hmm, perhaps better is a section with explicit differences
|
[19:01] sustrik
|
it's up to you
|
[19:02] pieterh
|
i think as a programmer, what I want is "what changed in 2.1 and what does this mean for my code?"
|
[19:02] pieterh
|
expanded release notes with some jokes
|
[19:02] sustrik
|
sounds good
|
[19:02] pieterh
|
ok!
|
[19:03] mikko
|
good stuff
|
[19:03] mikko
|
do you have the build script online already?
|
[19:03] mikko
|
the documentation one
|
[19:04] mikko
|
my todo-list is almost empty now
|
[19:04] mikko
|
apart from testing openpgm on solaris
|
[19:06] pieterh
|
mikko: I'm documenting / cleaning it up, will publish it asap
|
[19:07] mikko
|
pieterh_: cool
|
[19:07] mikko
|
any other periodic things that should be automated?
|
[19:08] pieterh
|
mikko the machine
|
[19:08] pieterh
|
no todo list can survive
|
[19:08] Guthur
|
pieterh_, my first impression of the new api doc is that it is very good
|
[19:08] Guthur
|
well done
|
[19:08] pieterh
|
Guthur, I'm glad you like it...
|
[19:09] Guthur
|
I'd imagine most will have similar impressions
|
[19:09] pieterh
|
not too painful to read? it's the usual cortex red scheme but toned down a little
|
[19:09] Guthur
|
nope the styling is very good, for my tastes
|
[19:10] mikko
|
lestrrat: are you there?
|
[19:10] Guthur
|
must dash though
|
[19:13] pieterh
|
mikko: you'll need to get an API key from Wikidot
|
[19:13] pieterh
|
it's tied to your account
|
[19:13] pieterh
|
if you don't already have a Wikidot.com account, register and tell me your account name
|
[19:14] pieterh
|
then I'll get them to give you API access and a key
|
[19:14] mikko
|
mkoppanen
|
[19:17] pieterh
|
ok
|
[19:49] pieterh
|
mikko: ok, the tool is documented and packaged for reuse
|
[20:42] jobytaffey
|
Hi, I was wondering if anyone can help out. I'm new to 0MQ. I'm trying to write a TCP server which accepts many incoming connections, reads fixed sized packets from TCP and generates 0MQ messages for handling somewhere else. I'm using libev and the receive side seems fine. But, for sending (receive 0MQ message then put it onto a TCP socket), it looks like I need to zmq_poll() when libev gives me EV_WRITE, is there a better way of doing this? A zmq ca
|
[20:43] mikko
|
jobytaffey: yes
|
[20:43] mikko
|
jobytaffey: you should schedule libev EV_WRITE only if you got message coming from zeromq
|
[20:44] mikko
|
so roughly the following (assuming zeromq 2.1.x)
|
[20:44] mikko
|
you schedule constant read event for incoming tcp messages and incoming zeromq messages
|
[20:44] mikko
|
if you get tcp message in you schedule a EV_WRITE for zeromq out socket
|
[20:45] mikko
|
and if you get zeromq message in you schedule EV_WRITE for tcp socket
|
[20:45] jobytaffey
|
Ok, that makes sense. I didn't see anything in the API for getting at zeromq's underlying sockets though?
|
[20:46] mikko
|
jobytaffey: it's a feature in 2.1.x
|
[20:46] mikko
|
give me a sec
|
[20:46] mikko
|
jobytaffey: http://api.zero.mq/master:zmq-getsockopt see ZMQ_FD and ZMQ_EVENTS there
|
[20:48] jobytaffey
|
That looks like what I need. Thanks.
|
[20:49] mikko
|
it doesn't make sense to schedule EV_WRITE event unless you know that you have something to write
|
[22:52] lestrrat
|
wassup?
|
[23:05] lestrrat
|
oh I get it, there's a build failure. http://build.zero.mq/job/ZeroMQPerl-master_ZeroMQ2-master_GCC/313/
|
[23:05] lestrrat
|
mikko: I didn't change anything in the last few days
|
[23:48] mikko
|
lestrrat: ok, will rerun to see
|
[23:49] mikko
|
succeeds
|
[23:49] mikko
|
probably an intermittent failure
|