Saturday November 27, 2010

[Time] NameMessage
[02:52] foca hi all :) is there a way to, using pubsub, have the SUB subscribe to multiple queues? ideally some sort of wildcard?
[11:52] CIA-20 zeromq2: 03Martin Sustrik 07master * r0bc2a05 10/ .gitignore :
[11:52] CIA-20 zeromq2: .gitignore modified to ignore test executables
[11:52] CIA-20 zeromq2: Signed-off-by: Martin Sustrik <> -
[13:17] mikko good morning
[13:17] mikko or well, more like afternoon
[13:48] sustrik mikko: good afternoon
[14:08] mikko
[14:08] mikko getting there
[14:12] sustrik what's that?
[14:14] mikko adding sun studio flags for -pedantic, -Werror and -Wall
[14:14] mikko they have their own way to do the flags
[14:15] mikko sun studio is actually a pretty good compiler
[14:16] mikko seems to be a lot stricter about things than the others
[14:17] sustrik aha, i see
[14:17] mikko although it complains about things such as: "pub.cpp", line 68: Warning: Conversion of 64 bit type value to "int" causes truncation.
[14:18] mikko maybe have to disable -Werror for now
[14:29] Guthur msvc complains about those 64bit conversions as well
[14:32] sustrik mikko: yes, possibly
[14:32] sustrik in the long term the warnings should be fixed though
[14:32] sustrik Guthur: that's win64, right?
[14:34] Guthur sustrik, Yeah on win64
[14:35] Guthur But even when compiling a 32bit binary iirc
[14:35] Guthur it was saying possible conversion i think
[14:35] sustrik i don't see the warning on my win32
[14:35] Guthur let me check here
[14:35] Guthur will just take a minute (or ten) to fire up the VM and VS2010
[14:36] mikko hm
[14:36] mikko i wonder if it would be easy to automate MSVC builds
[14:39] sustrik mikko: dunno, but you would at least need win VM
[14:39] sustrik does hudson allow to run remote builds?
[14:41] mikko yeah
[14:41] mikko not sure how easy MSVC is to automate
[14:42] mikko and the current build box is ia32
[14:42] mikko so win64 issues wouldnt be visible
[14:42] mikko it's ESX so it could run a windows vm in theory
[14:43] sustrik well, if it's possible to automate it i'd be willing to donate some money to buy a win license for the build machine
[14:45] Guthur sustrik, Yeah it is only x64 builds
[14:45] Guthur on MSVS
[15:06] mikko sustrik: there is a small typo in MAINTAINERS
[15:06] mikko "Component: Application Threads and Differnet Socket Types"
[15:06] mikko Different*
[15:29] smandy Hi all, have a very noob question. On stackoverflow someone mentioned using omq as an in-process message queue, i.e. to replace traditional threads'n'locks synchronization. would this make appropriate? what cuold you send between producers/consumers? - pointers? data?
[15:29] smandy ^make^be^
[15:30] smandy or would that be considered overkill?
[15:32] mikko smandy: you cand pointers inside the process
[15:32] mikko smandy: and not overkill at all
[15:33] mikko smandy: you might find this interesting:
[15:33] smandy mikko: thanks
[15:33] mikko smandy: 0MQ suits very well for threaded programming if there is a need to share data
[15:34] mikko rather than lock and syncronise etc etc send messages between threads
[15:34] smandy I'm a bit of a java guy and I've been looking for an 'actor-like' framework for C/C++ ( along the lines of scala or erlang, actors). Looks like zeromqs abstractions are similar - would that be a reasonable statement to make?
[15:35] mikko smandy: yes, i think that is a fair statement
[15:36] mikko and with 0MQ you can distribute your process fairly easily as well
[15:36] mikko changing from inproc:// to tcp:// should be fairly trivial
[15:37] mikko naturally you need to marshall the data using tcp but apart from that it should be just switching the transport
[15:37] smandy so inproc you would share pointers, out of proc you need to have a serialization framework of some sort. Is there a 'sanctioned' way of doing this or do you call out to something like protocol buffers/thrift.
[15:38] mikko smandy: 0MQ doesn't have serialisation built-in
[15:38] mikko i remember someone talking about protobuffs
[15:39] mikko but any serialisation framework should work fine
[15:42] smandy got it. Thanks a lot for your quick and helpful answers :-)
[15:43] Guthur if you you are sharing pointers you are probably sharing global state, which kind of negates the message passing benefits
[15:44] Guthur unless you are letting go of the pointer on one end, I suppose
[15:47] mikko it depends on the situation as well
[15:48] smandy Guthur: right, could a smart pointer be passed around maybe? The intention would be to use the system in an erlang style - i.e. don't access global state, but if you're in the same address space, marshalling the data seems wasteful.
[15:49] mikko smandy: i don't know much about C++ tbh
[15:50] mikko smandy: the problematic thing is that if you give the pointer to N threads and each one of them write to it
[15:51] mikko smandy: naturally you can hand the ownership of the pointer to a thread that you hand it over to
[15:51] smandy mikko: I see, i think the intention would be to pass around immutable data, or at least have the contract be that the receiving threads can't muck around with what they've been passed...
[15:52] mikko i guess something like reference counting works well for that scenario
[15:52] smandy mikko: my thoughts exactly :-)
[15:53] smandy so to pass a pointer you just write the bytes of the pointer to the socket?
[15:54] mikko smandy: you wrap the pointer to a zeromq message
[17:48] cremes i see that the new LINGER, RECONNECT_IVL and BACKLOG sockopts are ints...
[17:49] cremes can we just make them int64s so that we don't have to worry about varying sizes of ints on different platforms?
[17:54] mikko cremes: we should
[17:54] cremes mikko: i'll open a ticket
[17:55] mikko although is the different int sizes a problem?
[17:55] mikko as you would have a binary for the platform
[17:55] mikko in which scenario would this cause problems?
[17:55] cremes it is a problem when running inside of a scripting environment
[17:55] mikko binding compiled on 64bit and libzmq on 32bit?
[17:56] mikko cremes: what do you mean by that?
[17:56] cremes a scripting environment can represent integers however it likes, so even on a 64-bit build it could use 32-bit integers for numbers smaller than 2^32
[17:57] cremes so imagine this...
[17:57] cremes libzmq is built 64-bit
[17:57] cremes the scripting runtime is also built 64-bit
[17:57] mikko but surely you would cast the given integer to int before passing to libzmq
[17:57] cremes the scripting environment uses 32-bit integers
[17:58] cremes passing an integer from that environment to the 64-bit library will mismatch
[17:58] cremes why should i have to cast? just define the parameter properly so that there is *no* question
[17:58] mikko it the scripting engine is built as 64 then sizeof(int) == sizeof(int)
[17:59] cremes internally yes, but who knows how it will represent the values to its FFI library?
[17:59] mikko if the scripting engine uses 32bit integer (such as int32_t) you still have to case when passing to int64_t
[17:59] cremes this is a potential bug; that's all i'm saying
[17:59] cremes yes
[18:00] mikko isn't this problem present with any integer type then?
[18:01] cremes mikko: it's only a problem for those options that can change depending on how the library was built
[18:01] cremes we can avoid all of this by just defining it as int64_t
[18:02] cremes if you look at the other sockopts, they are all defined specifically as uint64_t or int64_t
[18:02] cremes this is likely an oversight
[18:03] mikko i think it is
[18:09] Guthur For clrzmq2 I definitely prefer to see int64_t as opposed int
[21:21] CIA-20 zeromq2: 03Martin Sustrik 07master * r325dd2f 10/ (src/thread.cpp src/thread.hpp tests/test_shutdown_stress.cpp):
[21:21] CIA-20 zeromq2: Functions passed to pthread_create are declared as extern "C"
[21:21] CIA-20 zeromq2: So far these were declared as C++ static functions which
[21:21] CIA-20 zeromq2: was incorrect and caused warnings with SunStudio.
[21:21] CIA-20 zeromq2: Signed-off-by: Martin Sustrik <> -
[21:49] mikko sustrik_: thanks!
[22:20] shaun510 newbie question: I have a system I'd like to convert over to zeromq (currently it's using a hand-rolled tcp connection management system), however...
[22:20] shaun510 the components each maintain a tcp connection and stream data back and forth bidirectionally.
[22:21] shaun510 which doesn't really fit the 1:1 request/response model.
[22:21] shaun510 most messages don't require a response, for instance.
[22:22] shaun510 what messaging model in zeromq fits this type of usage?
[22:42] mikko shaun510: do you need occasional responses?
[22:42] shaun510 in a sense, yes
[22:42] mikko have you looked at XREQ/XREP?
[22:43] shaun510 the components will be sending data back and forth, but that data might be unrelated.
[22:43] mikko hmm
[22:43] mikko maybe two sockets in that case?
[22:43] mikko PUSH/PULL to both directions
[22:43] shaun510 this might be more of an architecture problem... yeah, two sockets would make more sense.
[22:43] shaun510 one socket for outbound, one for inbound, then.
[22:43] mikko what does the application do?
[22:44] shaun510 it's a jabber/XMPP server
[22:44] shaun510 a pretty far-removed variant of jabberd2, specifically.
[22:45] mikko
[22:45] mikko does this describe the use-case even remotely?
[22:47] shaun510 oooo..
[22:47] shaun510 yes.
[22:49] shaun510 I was wondering about doing that with XREQ/XREP pairs, but it seemed a bit counter to what they were designed for.
[22:49] shaun510 which made a bit nervous.
[22:49] mikko maybe do a small prototype?
[22:49] mikko you should get a prototype running in no time
[23:03] shaun510 building a prototype right now. :)
[23:04] shaun510 I'll let you know how it goes, assuming my internet keeps working (which is looking unlikley)
[23:06] mikko cool