IRC Log

Friday June 4, 2010

[Time] NameMessage
[07:38] lvh Hey.
[07:39] lvh What's the reccomended way to get ZeroMQ running on Debian stable?
[07:39] lvh I've tried manually downloading the libzmq0/zeromq packages, but they require a version of libuuid1 much higher than that in Debian 5.0.
[07:39] lvh Upgrade to testing?
[08:09] pieterh @lvh: grab the git and build from source
[08:18] lvh git master, or some other branch?
[08:23] sustrik lvh: depends on what you want
[08:23] sustrik officially released version is on branch 2.06
[08:24] sustrik trunk is going to be released shortly as version 2.0.7
[08:44] lvh So, for production use, just use the latest release branch. Okay, thanks :-)
[08:50] sustrik you are welcome
[10:58] mato re
[10:58] mato sustrik: ack, ok, so MORE instead of TBC, you're right
[10:58] mato sustrik: and as for the pgm examples, i'll happily remove those
[11:38] jugg does multiple duplicate ZMQ_SUBSCRIBE keep a reference count and require ZMQ_UNSUBSCRIBE for each duplicate subscription?
[11:46] mato it's in the doc for ZMQ_UNSUBSCRIBE:
[11:46] mato If the socket has several instances of the same filter attached the ZMQ_UNSUBSCRIBE option shall remove only one instance, leaving the rest in place and functional.
[12:14] jugg mato: somehow I missed that (I have no idea how). thanks.
[12:41] sustrik mato: re
[12:42] mato sustrik: hi
[12:42] sustrik hi
[12:42] mato i have a problem with the helpers in that they need uint64_t, or long long
[12:43] mato if i use long long C++ complains that it's not part of the standard :-(
[12:43] sustrik yes, include stdint.hpp
[12:43] sustrik it's ugly but it should work
[12:43] sustrik both under gcc and nsvc
[12:43] sustrik msvc
[12:50] mato sustrik: zmqd annoys me, i don't think it should be in the 2.0.7 release at all
[12:51] sustrik you would like to remove it from the trunk?
[12:51] sustrik isn't it sufficient to just left it undocumented?
[12:51] mato people will see that this "zmqd" thing is being built
[12:51] mato and will get the wrong idea
[12:52] sustrik it has to get in once anyway
[12:52] sustrik either now or later
[12:52] mato sustrik: d'you think we could remove it, make the release and then put it back? :-)
[12:52] mato sustrik: or at least move it under devices/ and call it zmq_device or something vague like that
[12:52] sustrik if you care so much as to do the work, i won't object
[12:53] mato sustrik: i'll do it, but what about the above option?
[12:53] mato what is better?
[12:53] sustrik what would be the point?
[12:53] sustrik either leave it or remove it
[12:53] sustrik shrug
[12:54] mato well the point is it should replace the existing devices...
[12:55] mato the current situation will just confuse people
[12:55] mato so let's remove it now
[12:55] mato and put it back later
[12:55] mato sustrik: ack?
[12:55] sustrik ack
[13:01] CIA-17 zeromq2: 03Martin Lucina 07master * rd844a90 10/ (Makefile.am configure.in zmqd/Makefile.am zmqd/zmqd.cpp): zmqd: Removing for now, not ready for 2.0.7 (+6 more commits...) - http://bit.ly/9iefIH
[13:08] mato sustrik: ok, all done and committed
[13:08] mato sustrik: time to start breaking stuff
[13:08] sustrik ok
[13:08] sustrik let me remove the args from zmq_init
[13:09] mato you do that, and remove all the deprecated stuff from zmq.h
[13:09] mato and don't forget to update the perf tests
[13:09] sustrik and devices
[13:09] sustrik yes
[13:24] CIA-17 zeromq2: 03Martin Sustrik 07master * r67ca7dc 10/ (10 files in 6 dirs): obsolete API elements removed - this commit breaks backward compatibility - http://bit.ly/cLVjTO
[13:24] sustrik mato: done
[13:24] sustrik going to check on win32
[13:24] mato sustrik: yup, i will check here on Linux, Solaris, NetBSD
[13:35] CIA-17 zeromq2: 03Martin Lucina 07master * r621d741 10/ src/atomic_ptr.hpp : Fix Solaris/NetBSD breakage in atomic_ptr.hpp - http://bit.ly/c24C6Q
[13:47] CIA-17 zeromq2: 03Martin Sustrik 07master * r9279938 10/ (6 files in 6 dirs): MSVC build fixed - http://bit.ly/aNzO1F
[13:50] sustrik mato: msvc build is fixed
[13:51] mato sustrik: great
[13:51] sustrik however, i believe it braks mingw build
[13:51] mato why?
[13:51] sustrik i would expect some include files will be missing
[13:51] mato huh?
[13:52] sustrik msvc build uses it's own platform.hpp
[13:52] sustrik which includes win socket headers
[13:52] sustrik this collided with #include <windows.h> in helpers.cpp
[13:52] sustrik so i removed it
[13:52] sustrik however, afaiu mingw build uses standard platform.hpp
[13:52] mato can windows.h not be included 2x?
[13:53] sustrik iirc the point in that winsock has to be included _before_ windows.h
[13:54] mato i have no idea how a "plain" mingw build works
[13:54] sustrik neither do i
[13:54] mato and i suspect that no one does that, or if they do, we've never supported it explicity
[13:54] sustrik what about your win build?
[13:54] mato given that a cross-build using mingw does work, then i see no reason for a native one not to
[13:55] mato but yes, i should re-check that the cross build still works
[13:55] sustrik please, do
[13:55] sustrik i've changed the source
[13:55] mato sustrik: in fact, maybe helpers.h should just include platform.hpp from ../src
[13:55] mato would that work for msvc?
[13:56] sustrik i suppose so
[13:56] sustrik ok, change it that way and I retest with MSVC
[13:57] mato sustrik: why did you change the snprintf in xmlParser back to sprintf?
[13:58] mato sustrik: hang on
[13:58] mato sustrik: why do you care about whether or not helpers.hpp includes windows.h?
[13:58] sustrik no snprintf in msvc
[13:59] sustrik because it breaks the build
[13:59] mato i don't understand why
[13:59] mato the perf tests do not use winsock
[13:59] sustrik but you'll get conflicts
[14:00] sustrik redefinition of symbols
[14:00] lvh Hey! I wanted to get Twisted support for 0MQ a while ago but that kind of fizzled out due to a combination of health problems and technical issues nobody agreed on. I'm going to try again because I think 0MQ support in Twisted would be good for both 0MQ and Twisted (duh). What I think was the critical point of failure was that the interested ZMQ folks didn't really understand Twisted's gory guts and vice versa (Twisted people not
[14:00] lvh understanding what ZMQ really wanted to do). I'm going to reopen the discussion on the Twisted side, but step one is educating people on just how ZMQ (wants to) work. Is there someone who could do a quick (less than 5m) review of the email I'm going to send to the TWisted mailing list to figure out if there are any grave technical mistakes in my understanding of ZMQ?
[14:00] lvh Wow, that was longer than I thought it was.
[14:00] mato lvh: maybe it belongs on the mailing list? :-)
[14:01] lvh Which one?
[14:01] sustrik zeromq-dev
[14:01] sustrik i'll review it
[14:01] mato sustrik: ah, you get these because stdint.hpp includes platform.hpp, which includes windows.h
[14:01] sustrik others may comment as well
[14:01] mato sustrik: right?
[14:01] sustrik mato: right
[14:01] lvh sustrik: Good idea. I'll polish it up some more and then mail zeromq-dev :-)
[14:01] sustrik but i am sure the ordering also matters
[14:01] sustrik lvh: goodo
[14:02] mato grr
[14:02] sustrik something like:
[14:02] sustrik #include <windows.h>
[14:03] sustrik #include <winsock2.h>
[14:03] sustrik fail
[14:03] sustrik other way round it's ok
[14:03] sustrik i may be wrong though
[14:03] sustrik it's just a dim memory of this kind of problem
[14:03] mato i think you're doing the wrong thing with the msvc platform.hpp
[14:03] sustrik ?
[14:03] mato it should not be including windows.h IMO
[14:04] sustrik what would you propose?
[14:04] mato i would propose that winsock2.h and windows.h are included inside ifdefs by those components that actually use them
[14:04] mato since that is what you're doing with all other system headers
[14:04] sustrik hm, that's lot of files
[14:05] sustrik but yes, it can work that way
[14:05] mato yes, but that is consistent with all other platforms
[14:05] sustrik right
[14:05] sustrik let me do it
[14:05] sustrik see you in 10 mins
[14:05] mato just stick it inside #ifdef _WIN32
[14:10] guido_g poor boy :)
[14:30] CIA-17 zeromq2: 03Martin Sustrik 07master * rcf048bb 10/ (8 files in 3 dirs): platform.hpp for MSVC contains only very basic stuff - http://bit.ly/aLcqx0
[14:31] sustrik mato: done, try now
[14:33] mato sustrik: trying...
[14:36] mato sustrik: well, at least you haven't broken it on linux :)
[14:38] sustrik :)
[14:42] mato sustrik: i get a bunch of warnings from the cross build about NOMINMAX being redefined in windows.hpp, those weren't there before
[14:43] sustrik try to fix it :)
[14:43] mato what is NOMINMAX?
[14:43] sustrik this is exactly why i haven't dared to mess with helper functions :)
[14:43] sustrik windows defines it's own min and max
[14:44] sustrik as macros
[14:44] sustrik those collide with std::min and std::max
[14:44] sustrik so what you do is you define NOMINMAX
[14:44] sustrik then include windows.h
[14:44] mato maybe it just needs a #ifndef NOMINMAX, #define
[14:44] mato let me try that
[14:44] sustrik this prevents definition of windows' min & max
[14:46] jugg I have a REQ socket connected to an endpoint. I send a message, and then wait in recv(). However, if the REP socket bound to the endpoint receives the message but dies/disconnects before sending the response, the REQ socket sits forever in recv() and never returns creating a dead lock. Because when the REP socket comes back online, it doesn't have an requests in its queue so never sends a reply.
[14:47] sustrik jugg: yes, that's a known problem
[14:47] sustrik we are going to fix that by resending the request
[14:48] sustrik at the moment the only thing you can do to avoid it is to poll for reply (zmq_poll + timeout)
[14:48] sustrik when reply doesn't arrive in time just consider it lost and resend
[14:48] sustrik however, all this should be done inside 0mq
[14:49] sustrik it will, but it'll take some more time to get there
[14:49] jugg ok, so a near-term fix isn't to be expected?
[14:50] sustrik mato: i would say you are going to implement it, within the scope of your current project, no?
[14:50] mato sustrik: guess so
[14:52] sustrik jugg: looks like fix is to expected in near term
[14:52] jugg :)
[14:55] mato ah, my bad for not using the correct magic incantation
[15:02] CIA-17 zeromq2: 03Martin Lucina 07master * r94dfe13 10/ src/windows.hpp :
[15:02] CIA-17 zeromq2: Fix MINGW build
[15:02] CIA-17 zeromq2: Mingw seems to define NOMINMAX, so don't redefine it if already defined - http://bit.ly/9k7J9l
[15:03] mato sustrik: ok, the cross build almost works; requires hand-patching a pgm include file because steven messed up, but i guess we don't care about that right now
[15:03] mato sustrik: and AFAIK steven doesn't care about v2 of openpgm anyway
[15:04] mato i just looked and he seems to be developing a version 3 *and* a version 5 :-)
[15:04] sustrik yes, ok
[15:04] sustrik what next?
[15:05] mato let me think
[15:05] mato 1. write up a draft of the release announcement
[15:05] mato 2. put that into NEWS in the distro, moving the non-changelog bits also out of ChangeLog and into NEWS
[15:06] mato 3. check that the Debian packaging is sane and put myself in as maintainer
[15:06] mato I think that's all; then we can release
[15:07] sustrik ok, i can do 1
[15:07] mato yes please
[15:07] mato I will do 2 in the mean time
[15:07] mato when you have a draft of 1 stick it at the top of NEWS and I'll review it
[15:07] mato ack?
[15:08] sustrik ack
[15:13] CIA-17 zeromq2: 03Martin Lucina 07master * rbaf659f 10/ (ChangeLog NEWS): Move news from ChangeLog into NEWS - http://bit.ly/dDsRc0
[15:13] mato sustrik: look at NEWS, model what you write on the same as is there for 2.0.6
[15:13] mato leaving out the bits that don't apply
[15:44] mato sustrik: how are you getting on?
[15:45] mato sustrik: the debian packaging will have to follow next week, i have a bunch of questions i need to ask adrian
[15:45] mato sustrik: since i'm not quite sure that i've got the workflow right
[15:45] mato sustrik: so as soon as we have a release announcement we're good to go
[15:50] sustrik on phone
[15:51] sustrik wait a sec
[15:51] mato no problem
[15:59] jugg http://www.zeromq.org/bindings:c "C++ binding is the native API of ØMQ project" ?
[16:00] mato ah, that should be changed
[16:00] mato the C API is the "native" API
[16:00] mato I think what sustrik meant is that the "C++ binding is included with the core 0MQ distribution"
[16:01] mato oh, i see
[16:01] mato i'll fix it
[16:26] sustrik mato: re
[16:26] mato sustrik: i'm still here...
[16:27] sustrik ok, back to announcement :)
[16:27] mato waiting for your draft of what goes in NEWS for 2.0.7
[16:27] mato then i will review and add my stuff
[16:27] sustrik ack
[16:27] sustrik what NEWS are you speaking about?
[16:27] sustrik the file seems to be empty
[16:27] mato pull git
[16:28] mato sustrik: I moved the "interesting" text from ChangeLog into NEWS
[16:28] mato leaving the ChangeLog with just... the changes :-)
[16:28] sustrik got it
[16:31] lvh I've sent that mail to zeromq-dev if anyone's interested
[16:31] lvh (but email is an asynchronous process, this is a notification not an interrupt ;-))
[16:31] sustrik mato: pt1
[16:31] sustrik * Multi-hop REQ/REP pattern is fully supprted. This feature allows to put
[16:31] sustrik a device(s) between REP and REP sockets thus allowing for scenarios like
[16:31] sustrik multi-threaded server, shared service queue etc. The whole infrastructure
[16:31] sustrik is transparent to the applications.
[16:32] lvh Huh, I'm not longer a member so I need moderator approval. Wonder how that happened.
[16:32] sustrik lvh: i've approved you
[16:32] sustrik your email
[16:32] sustrik maybe you are subscribed with a different email address?
[16:33] sustrik where's part 1 btw?
[16:33] lvh No, I just checked, for some reason I unsubscribed. No idea why, I don't remember doing that
[16:33] lvh sustrik: Well, that was a month or so back when we were talking about how to implement it
[16:33] lvh that fizzled out
[16:33] sustrik ah, i see
[16:34] mato sustrik: write it all up, put it in git, i will review it there
[16:34] lvh so it should really be "try 2" not "part 2"
[16:34] mato sustrik: pizza has just arrived, am going to eat, brb
[16:34] sustrik mato: ok
[16:34] lvh Whaaa? now mailman is telling me the confirmation string is bad
[16:34] lvh Oh, wiat, never mind, I'm just being an idiot.
[16:35] sustrik :)
[16:35] jugg is there any benefits/tradeoffs documentation/discussion on io_threads?
[16:36] lvh I don't know ZMQ but I know the mechanics it uses
[16:36] lvh I really don't see why you would ever want 2 io_threads unless you can somehow shove long-term blocking disk io into the io thread
[16:37] lvh which afaik doesn't happen very often (does it?)
[16:37] sustrik lvh: two possibilities
[16:37] sustrik 1. if your app is extremely network I/O intensive
[16:38] sustrik 2. if you need extra low latency
[16:38] lvh I can sort-of understand 2
[16:38] lvh I don't understand why eg epoll would start behaving better for 1
[16:38] sustrik in latter case running i/o threads on two different cpu cores can prevent latency peaks
[16:38] lvh yep
[16:38] sustrik it's not about epoll
[16:39] lvh It's about 1 CPU moving stuff around vs 2 CPUs doing that?
[16:39] sustrik rather about the associated work (such as copying data from kernel to user space)
[16:39] lvh right
[16:39] lvh I guess, I haven't done the benchmarks and I'm assuming you have :)
[16:39] sustrik yes :)
[16:40] lvh is there a way to shove blocking disk IO into the io thread in zmq?
[16:40] lvh I mean a *sane* way.
[16:41] lvh Oh, by the way: how does "Is fully distributed: no central servers to crash, millions of WAN and LAN nodes." apply to things like zmq_forwarder/zmq_queue? AFAICT it doesn't, really?
[16:42] lvh My use case involves decoupling M producers from N consumers. That's what we use AMQP for now.
[16:45] lvh (The three things blocking us from moving to ZeroMQ at the moment are: it doesn't do Twisted yet, Thrift support uncertain for a lot of implementations (even though that's more of a Thrift problem than a ZeroMQ problem), and reliable delivery is a bit hazy)
[16:50] CIA-17 zeromq2: 03Martin Sustrik 07master * r9b8f902 10/ NEWS : initial version of 2.0.7 NEWS - http://bit.ly/a8mp7Z
[16:50] sustrik mato: committed "new features" and "interface"
[16:50] sustrik i'll leave "build" part to you
[16:51] sustrik lvh: yes, forwarder/queue is meant
[16:51] mato sustrik: ok, going to look at it now
[16:51] sustrik these are not real "central servers"
[16:51] sustrik rather dumb routers
[16:52] sustrik what's thrift?
[16:52] lvh sustrik: Do you know Protocol Buffers?
[16:52] sustrik yes
[16:52] lvh sustrik: Basically, Thrift is something that you describe data structures and services to
[16:52] lvh sustrik: Oh, even better
[16:52] lvh sustrik: thrift = protobufs + rpc
[16:52] sustrik ok, got it
[16:53] lvh sustrik: you describe services and it generates code for a whole bunch of languages
[16:53] sustrik what does it have to do with 0mq?
[16:53] lvh sustrik: it's not 0mq's problem, but I want to put the binary format in zmq messages
[16:53] sustrik can the transport layer be migrated to whatever messaging implementation you like?
[16:53] lvh sustrik: a lot of thrift bindings dont give you the format directly
[16:53] lvh sustrik: yes unless those implementations suck
[16:53] lvh sustrik: like I said, that's not 0mq's problem
[16:53] sustrik ok, i see
[16:54] sustrik reliability: yes
[16:54] lvh sustrik: they just hook it directly into their service code
[16:54] sustrik it'll get better soon though
[16:54] lvh the python thrift code does the right thing, it essentially gives you objects that have serialization methods that return raw bytestrings
[16:54] lvh "raw" makes no sense there, you get the idea
[16:55] sustrik sure
[16:55] lvh sustrik: Cool :-) We can wait, AMQP works fine in production now anywya
[16:57] mato sustrik: put devices between REP and REP sockets, you mean REQ and REP sockets?
[17:01] sustrik mato: yes
[17:01] sustrik typo
[17:03] sustrik mato: i have to leave now
[17:03] sustrik anything else i should do?
[17:04] mato sustrik: well, if you're happy with me uploading everything and sending out the release announcement, then no
[17:04] sustrik great
[17:04] mato ok then
[17:04] sustrik just a note
[17:04] sustrik when sending an announcement
[17:05] mato yes?
[17:05] sustrik note that API/ABI have changed and that it may take few days for language bindings to catch up
[17:05] mato yes, good point
[17:06] mato i'm changing your text a bit, do you mind?
[17:07] sustrik sure, do so
[17:07] versificateur hello
[17:07] sustrik leaving now
[17:07] sustrik see you later
[17:07] mato see you
[17:08] versificateur does the java bindings (jzmq) support rcv multipart??
[17:08] versificateur mato:??
[17:09] versificateur sustrik:??
[17:09] mato versificateur: i don't use jzmq, sorry
[17:09] mato but there's no reason why it shouldn't
[17:09] versificateur :( thanks
[17:10] mato versificateur: if it doesn't work then report a bug :)
[17:10] versificateur okay
[17:21] CIA-17 zeromq2: 03Martin Lucina 07master * re8a9614 10/ NEWS : Update NEWS for 2.0.7 release - http://bit.ly/9Hb3hZ
[17:31] CIA-17 zeromq2: 03Martin Lucina 07master * r5c97ff9 10/ NEWS : More NEWS for 2.0.7 - http://bit.ly/cwE7pj
[17:31] CIA-17 zeromq2: 03Martin Lucina 07master * r8f51a10 10/ ChangeLog : Update ChangeLog for v2.0.7 - http://bit.ly/cgL2VL
[18:35] versificateur please could you tell what this error refer to??
[18:35] versificateur Assertion failed: !inpipe && !outpipe (pair.cpp:47)

Web site design and content is copyright © 2010-2011 Peter Alexander, ProDataLab. Site content licensed under the Creative Commons Attribution-Share Alike 3.0 License. ØMQ is copyright © Copyright © 2007-2011 iMatix Corporation and Contributors. Contact us for professional support. ØMQ is free software licensed under the LGPL. ØMQ, ZeroMQ, and 0MQ are trademarks of iMatix Corporation. Terms of Use ? Privacy Policy