ZeroMq IRC Log

Saturday November 27, 2010

[Time] Name	Message
[02:52] foca	hi all :) is there a way to, using pubsub, have the SUB subscribe to multiple queues? ideally some sort of wildcard?
[11:52] CIA-20	zeromq2: 03Martin Sustrik 07master * r0bc2a05 10/ .gitignore :
[11:52] CIA-20	zeromq2: .gitignore modified to ignore test executables
[11:52] CIA-20	zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/eSO8gq
[13:17] mikko	good morning
[13:17] mikko	or well, more like afternoon
[13:48] sustrik	mikko: good afternoon
[14:08] mikko	https://gist.github.com/42c775a7b6b9f67ed04f
[14:08] mikko	getting there
[14:12] sustrik	what's that?
[14:14] mikko	adding sun studio flags for -pedantic, -Werror and -Wall
[14:14] mikko	they have their own way to do the flags
[14:15] mikko	sun studio is actually a pretty good compiler
[14:16] mikko	seems to be a lot stricter about things than the others
[14:17] sustrik	aha, i see
[14:17] mikko	although it complains about things such as: "pub.cpp", line 68: Warning: Conversion of 64 bit type value to "int" causes truncation.
[14:18] mikko	maybe have to disable -Werror for now
[14:29] Guthur	msvc complains about those 64bit conversions as well
[14:32] sustrik	mikko: yes, possibly
[14:32] sustrik	in the long term the warnings should be fixed though
[14:32] sustrik	Guthur: that's win64, right?
[14:34] Guthur	sustrik, Yeah on win64
[14:35] Guthur	But even when compiling a 32bit binary iirc
[14:35] Guthur	it was saying possible conversion i think
[14:35] sustrik	i don't see the warning on my win32
[14:35] Guthur	let me check here
[14:35] Guthur	will just take a minute (or ten) to fire up the VM and VS2010
[14:36] mikko	hm
[14:36] mikko	i wonder if it would be easy to automate MSVC builds
[14:39] sustrik	mikko: dunno, but you would at least need win VM
[14:39] sustrik	does hudson allow to run remote builds?
[14:41] mikko	yeah
[14:41] mikko	not sure how easy MSVC is to automate
[14:42] mikko	and the current build box is ia32
[14:42] mikko	so win64 issues wouldnt be visible
[14:42] mikko	it's ESX so it could run a windows vm in theory
[14:43] sustrik	well, if it's possible to automate it i'd be willing to donate some money to buy a win license for the build machine
[14:45] Guthur	sustrik, Yeah it is only x64 builds
[14:45] Guthur	on MSVS
[15:06] mikko	sustrik: there is a small typo in MAINTAINERS
[15:06] mikko	"Component: Application Threads and Differnet Socket Types"
[15:06] mikko	Different*
[15:29] smandy	Hi all, have a very noob question. On stackoverflow someone mentioned using omq as an in-process message queue, i.e. to replace traditional threads'n'locks synchronization. would this make appropriate? what cuold you send between producers/consumers? - pointers? data?
[15:29] smandy	^make^be^
[15:30] smandy	or would that be considered overkill?
[15:32] mikko	smandy: you cand pointers inside the process
[15:32] mikko	smandy: and not overkill at all
[15:33] mikko	smandy: you might find this interesting: http://www.zeromq.org/blog:multithreading-magic
[15:33] smandy	mikko: thanks
[15:33] mikko	smandy: 0MQ suits very well for threaded programming if there is a need to share data
[15:34] mikko	rather than lock and syncronise etc etc send messages between threads
[15:34] smandy	I'm a bit of a java guy and I've been looking for an 'actor-like' framework for C/C++ ( along the lines of scala or erlang, actors). Looks like zeromqs abstractions are similar - would that be a reasonable statement to make?
[15:35] mikko	smandy: yes, i think that is a fair statement
[15:36] mikko	and with 0MQ you can distribute your process fairly easily as well
[15:36] mikko	changing from inproc:// to tcp:// should be fairly trivial
[15:37] mikko	naturally you need to marshall the data using tcp but apart from that it should be just switching the transport
[15:37] smandy	so inproc you would share pointers, out of proc you need to have a serialization framework of some sort. Is there a 'sanctioned' way of doing this or do you call out to something like protocol buffers/thrift.
[15:38] mikko	smandy: 0MQ doesn't have serialisation built-in
[15:38] mikko	i remember someone talking about protobuffs
[15:39] mikko	but any serialisation framework should work fine
[15:42] smandy	got it. Thanks a lot for your quick and helpful answers :-)
[15:43] Guthur	if you you are sharing pointers you are probably sharing global state, which kind of negates the message passing benefits
[15:44] Guthur	unless you are letting go of the pointer on one end, I suppose
[15:47] mikko	it depends on the situation as well
[15:48] smandy	Guthur: right, could a smart pointer be passed around maybe? The intention would be to use the system in an erlang style - i.e. don't access global state, but if you're in the same address space, marshalling the data seems wasteful.
[15:49] mikko	smandy: i don't know much about C++ tbh
[15:50] mikko	smandy: the problematic thing is that if you give the pointer to N threads and each one of them write to it
[15:51] mikko	smandy: naturally you can hand the ownership of the pointer to a thread that you hand it over to
[15:51] smandy	mikko: I see, i think the intention would be to pass around immutable data, or at least have the contract be that the receiving threads can't muck around with what they've been passed...
[15:52] mikko	i guess something like reference counting works well for that scenario
[15:52] smandy	mikko: my thoughts exactly :-)
[15:53] smandy	so to pass a pointer you just write the bytes of the pointer to the socket?
[15:54] mikko	smandy: you wrap the pointer to a zeromq message
[17:48] cremes	i see that the new LINGER, RECONNECT_IVL and BACKLOG sockopts are ints...
[17:49] cremes	can we just make them int64s so that we don't have to worry about varying sizes of ints on different platforms?
[17:54] mikko	cremes: we should
[17:54] cremes	mikko: i'll open a ticket
[17:55] mikko	although is the different int sizes a problem?
[17:55] mikko	as you would have a binary for the platform
[17:55] mikko	in which scenario would this cause problems?
[17:55] cremes	it is a problem when running inside of a scripting environment
[17:55] mikko	binding compiled on 64bit and libzmq on 32bit?
[17:56] mikko	cremes: what do you mean by that?
[17:56] cremes	a scripting environment can represent integers however it likes, so even on a 64-bit build it could use 32-bit integers for numbers smaller than 2^32
[17:57] cremes	so imagine this...
[17:57] cremes	libzmq is built 64-bit
[17:57] cremes	the scripting runtime is also built 64-bit
[17:57] mikko	but surely you would cast the given integer to int before passing to libzmq
[17:57] cremes	the scripting environment uses 32-bit integers
[17:58] cremes	passing an integer from that environment to the 64-bit library will mismatch
[17:58] cremes	why should i have to cast? just define the parameter properly so that there is no question
[17:58] mikko	it the scripting engine is built as 64 then sizeof(int) == sizeof(int)
[17:59] cremes	internally yes, but who knows how it will represent the values to its FFI library?
[17:59] mikko	if the scripting engine uses 32bit integer (such as int32_t) you still have to case when passing to int64_t
[17:59] cremes	this is a potential bug; that's all i'm saying
[17:59] cremes	yes
[18:00] mikko	isn't this problem present with any integer type then?
[18:01] cremes	mikko: it's only a problem for those options that can change depending on how the library was built
[18:01] cremes	we can avoid all of this by just defining it as int64_t
[18:02] cremes	if you look at the other sockopts, they are all defined specifically as uint64_t or int64_t
[18:02] cremes	this is likely an oversight
[18:03] mikko	i think it is
[18:09] Guthur	For clrzmq2 I definitely prefer to see int64_t as opposed int
[21:21] CIA-20	zeromq2: 03Martin Sustrik 07master * r325dd2f 10/ (src/thread.cpp src/thread.hpp tests/test_shutdown_stress.cpp):
[21:21] CIA-20	zeromq2: Functions passed to pthread_create are declared as extern "C"
[21:21] CIA-20	zeromq2: So far these were declared as C++ static functions which
[21:21] CIA-20	zeromq2: was incorrect and caused warnings with SunStudio.
[21:21] CIA-20	zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/i1JeTr
[21:49] mikko	sustrik_: thanks!
[22:20] shaun510	newbie question: I have a system I'd like to convert over to zeromq (currently it's using a hand-rolled tcp connection management system), however...
[22:20] shaun510	the components each maintain a tcp connection and stream data back and forth bidirectionally.
[22:21] shaun510	which doesn't really fit the 1:1 request/response model.
[22:21] shaun510	most messages don't require a response, for instance.
[22:22] shaun510	what messaging model in zeromq fits this type of usage?
[22:42] mikko	shaun510: do you need occasional responses?
[22:42] shaun510	in a sense, yes
[22:42] mikko	have you looked at XREQ/XREP?
[22:43] shaun510	the components will be sending data back and forth, but that data might be unrelated.
[22:43] mikko	hmm
[22:43] mikko	maybe two sockets in that case?
[22:43] mikko	PUSH/PULL to both directions
[22:43] shaun510	this might be more of an architecture problem... yeah, two sockets would make more sense.
[22:43] shaun510	one socket for outbound, one for inbound, then.
[22:43] mikko	what does the application do?
[22:44] shaun510	it's a jabber/XMPP server
[22:44] shaun510	a pretty far-removed variant of jabberd2, specifically.
[22:45] mikko	http://www.mail-archive.com/zeromq-dev@lists.zeromq.org/msg02501.html
[22:45] mikko	does this describe the use-case even remotely?
[22:47] shaun510	oooo..
[22:47] shaun510	yes.
[22:49] shaun510	I was wondering about doing that with XREQ/XREP pairs, but it seemed a bit counter to what they were designed for.
[22:49] shaun510	which made a bit nervous.
[22:49] mikko	maybe do a small prototype?
[22:49] mikko	you should get a prototype running in no time
[23:03] shaun510	building a prototype right now. :)
[23:04] shaun510	I'll let you know how it goes, assuming my internet keeps working (which is looking unlikley)
[23:06] mikko	cool