Thursday April 1, 2010

[Time] NameMessage
[00:03] wutang I am very familiar with 29west...are there any documents out there describing the differences in architecture and possible API differences between the two?
[00:03] wutang Also, I haven't found many examples beyond the most basic...are there any good repositories of examples or real applications that are open source?
[08:00] mikko hmm.. having an odd issue when linking with a C program
[08:02] sustrik mikko: hi
[08:03] mikko hi
[08:03] sustrik what's the problem?
[08:03] mikko /usr/local/lib/ undefined symbol: _ZN3zmq7epoll_t4loopEv
[08:04] sustrik only a single one?
[08:04] sustrik single undefined symbol?
[08:04] mikko there are other symbols as well but it all seems very random
[08:04] mikko i can run the program let's say 8 times successfully and on ninth i get lookup error
[08:04] mikko i wonder if something happened to my environment
[08:05] sustrik strange
[08:05] sustrik looks like your system is broken
[08:08] mikko trying with LD_DEBUG now
[08:11] mikko
[08:11] mikko not sure what this tells me
[08:12] mikko trying with clean system
[08:21] mikko sustrik: also getting this occasionally symbol lookup error: /usr/local/lib/ undefined symbol: uuid_unparse, version UUID_1.0
[08:21] mikko but not on all runs, it seems very random
[08:21] sustrik there must be something wrong with your system; library loading should be a deterministic process
[08:22] sustrik can you try on a different box?
[08:22] mikko yeah, setting up another VM on the background
[09:24] sjampoo Cool, got PGM working in Python code. Seems that i can only use a single context per process (because of refcounting?).
[09:25] sjampoo How do i set an optimal multicast rate, by experimentation?
[09:47] sustrik sjampoo: yes, one context per process suffices
[09:48] sustrik as for the rate, set it up to match your requirements
[09:48] mikko oh, btw i forgot to mention: i am working on php bindings for zeromq
[09:48] mikko should have something working by the end of the day
[09:48] sustrik mikko: nice
[09:49] sustrik btw, there were several people messing with php
[09:49] sustrik so you may ask about it on the mailing list
[09:49] sustrik the main problem afaiu is to have a persistent connection from php
[09:50] sustrik rather than establishing and tearing the connection all the time
[09:51] mikko sustrik: that should not really be hard
[09:53] mikko sustrik: i'll take a peek at the mailing-lists
[10:02] ud hi
[10:03] ud Why PGM may loss messages ?
[10:10] sjampoo ud, it might be that you are sending the messages from one context and that all your sends have finished but ZMQ is still pushing them to the network (because of the multicast rate). However, if you close the context before that is finished the rest of your message will never arrive.
[10:24] sustrik ud_: also there's a problem of the finite retransmit window
[10:25] sustrik sender may store at most that much data
[10:25] sustrik thus is receiver requests retransmission of data that were already dropped by the sender
[10:25] sustrik sender has no way to retransmit
[10:36] ud Also, somtimes I handle this: Assertion failed: it != peers.end () (pgm_receiver.cpp:161)
[10:39] sustrik ud_: let me see
[10:40] sustrik ud_: there have been a bug in OpenPGM lately
[10:41] sustrik this looks like it may be the consequence
[10:41] sustrik however, if you don't overload your network, you shouldn't hit the problem
[10:42] sustrik anyway, the original bug will be solved in next release (2.0.7)
[10:46] ud ok, when will the next release ? :)
[11:08] sustrik ud_: couple of weeks from now
[11:09] sjampoo ud, i got the same message when closing the context prematurely and then firing up a different context.
[11:12] sustrik sjampoo: that's interesting
[11:13] sustrik i can imagine a way how that could happen
[11:13] sustrik can you fill a bug report, please?
[11:14] sjampoo hmm, i did not expect it to be a bug really, just misunderstanding on my part
[11:16] sustrik sjampoo: i think it is a bug
[11:16] sjampoo ok, ill file a report
[11:17] sustrik what's happening imo is that the two subsequent PGM session interact in a strange way
[11:17] sustrik yes, please
[11:17] sustrik it definitely shouldn't assert, in worse case it should return error
[11:18] sjampoo I am running it from Python code, also when loading two context in a single process i get an assertion error as well
[11:18] sjampoo (when i compiled zmq with pgm support)
[11:18] sjampoo not sure if this is somewhat related
[11:34] sustrik what's the assertion in that case?
[11:34] sjampoo Assertion failed: !pgm_supported () (zmq.cpp:234)
[11:35] sjampoo There is a comment about refcounting above it
[11:40] sustrik that's 0MQ/2.0.6?
[11:40] sjampoo yes
[11:41] sustrik ok, i see; to solve the problem OpenPGM has to be patched
[11:43] sjampoo Hmmm, was just able to succesfully reproduce the peer.end() problem with the performance tools local_thr, and remote_thr
[11:44] sjampoo Where should i file the bug?
[11:44] sustrik go to github repo
[11:44] sjampoo zeromq right?
[11:44] sustrik there's "issues" tab ther
[11:44] sjampoo Nog OpenPGM?
[11:44] sjampoo not even
[11:45] sustrik the peer.end() is 0mq problem
[11:45] sjampoo ok.
[11:45] sustrik if you want the pgm_supported problem solved you should ask on OpenPGM mailing list
[11:46] sjampoo ok, thanks for the info
[11:47] sustrik np
[11:53] keenerd Dumb question; where did the C bindings go in 2.0.6? The docs just loop back on themselves.
[11:59] mikko keenerd: you mean ?
[12:02] keenerd Yeah. Is there an undocumented configure flag?
[12:02] mikko sustrik: am i waiting on a message?
[12:02] sustrik mikko: looks like
[12:03] sustrik keenerd, what configure flag>
[12:03] sustrik ?
[12:03] sustrik ./configure --help
[12:03] mikko hmm. the same thing works on a single process but as soon as i got multiple forks things go downhill
[12:03] sustrik gives you list of all available flags
[12:03] keenerd By loop back on themselves, I mean the homepage. It presumes C bindings are just there, and does not give a source to make them.
[12:04] sustrik 0MQ native interface is C
[12:04] sustrik so when you build it, you have it
[12:04] sustrik no separate project is needed
[12:04] keenerd No, nothing helpful in configure --help
[12:04] keenerd sustrik: You sure? It really looks like C++ is the naitive interface now.
[12:05] sustrik no, C is native (zmq.h)
[12:05] sustrik C++ is a thin wrapper over C (zmq.hpp)
[12:05] sustrik the documentation is confusing though
[12:05] sustrik when clicking on C in the left pane is should say that it's native interface etc.
[12:06] keenerd Okay, I see it. "make install" ignores it, though.
[12:06] sustrik nope, it should copy it to use/local/include
[12:06] keenerd Does not work for me.
[12:06] sustrik it's possible that you dan't have that on your path
[12:07] keenerd No, I'm packaging the library. It does not get moved from the source directoey to the package directory.
[12:07] sustrik by packaging you mean make dist?
[12:07] keenerd No.
[12:08] keenerd I maintain it for Arch.
[12:08] sustrik ok, can you be more specific on what you are doing?
[12:08] sustrik ./
[12:08] sustrik ./configure
[12:08] sustrik make
[12:08] sustrik make install
[12:08] sustrik ?
[12:09] keenerd ./configure prefix=...
[12:09] keenerd make
[12:09] keenerd make install
[12:09] sustrik ok
[12:09] sustrik what's missing afterwards?
[12:09] keenerd zmq.h, at the very least :-)
[12:09] sustrik ok, let me try...
[12:10] sustrik sjampoo: thanks
[12:10] keenerd Is autogen really needed now? Trying that...
[12:10] sustrik depends on whether you build from the tar.gz package
[12:10] sustrik if so, you don't need it
[12:10] sustrik if you are building directly from github repo, you need it
[12:11] keenerd Ah, no. Someone else maintains the git.
[12:12] sustrik then just ommit the autogen step
[12:13] keenerd Otherwise, building it is a lot simpler now.
[12:14] keenerd The old --with would crash easily.
[12:14] mikko sustrik: the interesting thing is that it looks like im waiting for a message but i dont see the "request" in tcpdump
[12:14] mikko so nothing has been sent at that point
[12:15] sustrik keenerd: yes, we've trew out all the stuf that wasn't absolutely needed
[12:16] sustrik mikko: different things may got wrong, send your test prog to the mailing list and i'll have a look
[12:18] mikko sustrik: my test prog is the full php extension
[12:18] mikko it might be a bit heavy
[12:19] sustrik then just describe what you are doing
[12:19] mikko i think it might have something to do with this
[12:19] mikko but not sure yet
[12:19] sustrik grr, that's the bug i cannot get my hands on
[12:20] sustrik i am not able to reproduce it here
[12:20] sustrik would it be able to get access to your box?
[12:20] mikko Assertion failed: !incomplete_in (session.cpp:123)
[12:20] mikko Aborted
[12:20] mikko on the server side
[12:20] mikko sustrik: it's a virtual machine
[12:20] mikko i can zip it up and send it to you
[12:21] mikko i can try to make it available online first though (not sure about local firewall)
[12:21] sustrik that would be great
[12:24] sustrik mikko: you are using bleeding edge version, this is a problem with the code committed last week
[12:24] sustrik you may want to use 2.0.6 instead
[12:25] sustrik keenerd: i've tested the build:
[12:25] sustrik ./configure --prefix=xyz
[12:25] sustrik make
[12:25] sustrik make install
[12:26] sustrik ls xyz/include
[12:26] sustrik zmq.h
[12:26] sustrik zmq.hpp
[12:26] keenerd Huh.
[12:27] keenerd Very odd.
[12:28] sustrik maybe a typo in the path?
[12:28] keenerd Which path? I see zmq.hpp.
[12:29] sustrik --prefix=path
[12:29] mikko sustrik: can you give me your pubkey?
[12:29] sustrik what's your email?
[12:30] mikko
[12:30] sustrik wait a sec
[12:30] sustrik mikko: ok, sent
[12:31] keenerd sustrik: Okay, I reinstalled make and related packages. Problem seems to be gone. I blame cosmic rays, and will replace my aging drive soon.
[12:31] keenerd Sorry for the hassle.
[12:31] sustrik :)
[12:32] keenerd Thanks.
[12:34] sustrik np, autotools is a mysterious piece of software with it's own will
[15:20] mikko sustrik: still here?
[15:36] mikko anyone had this:
[15:36] mikko Assertion failed: !engine (session.cpp:263)
[16:20] Skaag ping?
[16:38] mikko pong
[17:28] Skaag an active person!
[17:36] mikko well, active and active
[17:36] mikko my first day here
[17:41] mikko Skaag: looks like you found my github repo
[17:41] Skaag lol
[17:42] Skaag yes :-)
[17:42] Skaag I wondered if you'd notice while here
[17:43] Skaag While it was a half joke, I will be happy to test it and add/edit documentation
[17:43] Skaag I think I may have made a mistake wt
[17:43] mikko sure, that would probably be useful
[17:43] mikko especially missing features
[17:43] Skaag with choosing rabbitmq prematurely
[17:43] mikko brb
[18:05] sustrik mikko: Assertion failed: !engine (session.cpp:263) looks like a bug
[18:05] sustrik any idea how to reproduct it?
[18:05] mikko sustrik: yes
[18:05] sustrik reproduce*
[18:05] mikko i downgraded to 2.0.6
[18:05] mikko when i set IDENTITY
[18:05] sustrik aha
[18:05] mikko it happens on second request
[18:06] mikko sustrik: btw i got the php bindings running
[18:06] mikko also persistent connections should be working
[18:06] sustrik the IDENTITY is set on req or rep?
[18:06] sustrik mikko: great
[18:06] mikko REQ
[18:06] sustrik ok, i'll give it a try - although after easter holiday
[18:06] sustrik as for the binding, would you like me to link to it from the website?
[18:07] mikko that's would nice, although it's not stable yet
[18:07] mikko i guess people understand as its on github
[18:07] sustrik call it "alpha", no?
[18:07] mikko
[18:07] mikko well, no tags / releases yet
[18:08] mikko let me add EXPERIMENTAL file
[18:08] sustrik ok
[18:08] mikko sustrik: i think the earlier epoll issue is related to forks
[18:09] mikko context created in parent and accessed in forked child
[18:09] mikko i might be mistaken on that
[18:09] sustrik mikko: i am not sure whether that works
[18:09] sustrik never tried it myself
[18:10] mikko i think that's what i ended up with php module running in apache
[18:10] mikko i moved the context creation 'per object' and all my problems went away
[18:10] sustrik in any case it's more clean solution
[18:10] mikko btw, zmq_msg_t seems to be empty after it's been sent. is that due to possible async sending where everything is not sent on one go?
[18:10] Skaag very nice
[18:10] sustrik context basically represents "library instance"
[18:11] sustrik mikko: it's a performance issue
[18:11] sustrik if the message was retained
[18:12] sustrik there would have to be a reference count attached to it
[18:12] sustrik (one reference from sending I/O thread, other from client's thread)
[18:12] sustrik the refcount would have to be manipulated using atomic operations
[18:12] sustrik those are pretty expensive
[18:12] mikko ok, now i get it
[18:12] sustrik however, if you need to retain the message, there's zmq_msg_copy
[18:13] sustrik just copy it before sending
[18:14] mikko i would think copy is as expensive as initing a new one?
[18:14] mikko ah
[18:14] mikko reading man pages
[18:14] sustrik no, there's ref count turned on in such case
[18:14] sustrik so it doens't really copy the data
[18:15] mikko one thing i noticed here:
[18:15] sustrik mikko: ok, go here:
[18:15] sustrik
[18:15] mikko Setting I/O thread affinity
[18:16] sustrik i've copied the text from python binding
[18:16] mikko it passes 2 as the parameter
[18:16] sustrik create an account on the website and you may modify it as you see fit
[18:16] mikko which causes zmq to crash
[18:16] mikko thanks!
[18:16] sustrik np
[18:16] sustrik let me know your login so I can grant you write access btq
[18:16] sustrik btw
[18:17] sustrik as for the crash, does it crash in zmq_setsockopt itself?
[18:17] mikko yep
[18:17] sustrik segfault?
[18:18] mikko invalid read size of 8 (on my arch)
[18:18] mikko probably due to it expecting a pointer
[18:18] sustrik yes, it expects a pointer
[18:18] sustrik it similar to standard POSIX setsockopt
[18:18] mikko the example passes: rc = zmq_setsockopt (socket, ZMQ_AFFINITY, 2, sizeof (int64_t));
[18:19] sustrik coincidence
[18:19] sustrik it should look like this:
[18:19] sustrik int64_t affinity =2;
[18:19] sustrik zmq_setsockopt (s, ZMQ_AFFINITY, &affinity, sizeof (affinity));
[18:20] mikko yep
[18:20] mikko seems like all other examples are correct (camera, chat etc)
[18:20] mikko i seem to have a wikidot account already
[18:20] mikko 'mkoppanen'
[18:20] sustrik wait a sec
[18:20] mikko it must be because liblqr was hosted on wikidot
[18:21] mikko the website
[18:21] Skaag I never had one, made it now
[18:21] sustrik Skaag: you want access too?
[18:22] Skaag sure
[18:22] sustrik mikko: you are invited
[18:22] sustrik jst accept and you can alter the page
[18:22] Skaag I'll probably mostly fix typos if I find them, or rephrase sentences to clarify
[18:22] Skaag and maybe contribute some code samples, if applicable
[18:22] Skaag I'm 'skaag' on wikidot
[18:23] sustrik Skaag: you are working on PHP as well?
[18:23] Skaag PHP, C++ and Java
[18:23] sustrik ok, feel free to change the page of the binding you are working on
[18:23] Skaag thanks
[18:23] sustrik however, when changing generic content first discuss it on thr mailing listy
[18:24] sustrik Skaag: you are invited
[18:25] Skaag awesome
[18:25] Skaag I assume you copied the python page content to the php binding page for convenience?
[18:28] sustrik Skaag: yes, to give you a template
[18:28] Skaag awesome. it's helpful.
[18:28] sustrik however, feel free to modify it in any way
[18:28] sustrik once you have something ready ping me and I'll link your page from the main page
[18:29] mikko will do
[18:29] mikko thanks a lot
[20:35] sustrik mikko: still there?
[20:41] mikko sustrik: yes
[22:15] mikko initial round on docs
[22:15] mikko at least doesnt say py0MQ anymore :)
[22:15] Skaag awesome work
[22:15] Skaag I will test and give feedback