ZeroMq IRC Log

Saturday October 9, 2010

[Time] Name	Message
[00:23] jsimmons	what would be the result of having multiple zmq contexts in a process?
[05:48] FreeNslaved	grim is live ..freekers ball...taking request on channel 3 over at http://www.reallibertymedia.com
[05:56] CIA-20	zeromq2: 03Martin Sustrik 07maint * ref8db78 10/ (4 files in 4 dirs): (log message trimmed)
[05:56] CIA-20	zeromq2: Version macros added
[05:56] CIA-20	zeromq2: Macro ZMQ_VERSION represents the current version of 0MQ
[05:56] CIA-20	zeromq2: Macro ZMQ_MAKE_VERSION(major,minor,patch) allows to create
[05:56] CIA-20	zeromq2: a representation of the specified version.
[05:56] CIA-20	zeromq2: The versions can be compared using simple <, >, ==, etc.
[05:56] CIA-20	zeromq2: operators.
[06:05] CIA-20	zeromq2: 03Martin Sustrik 07master * ref8db78 10/ (4 files in 4 dirs): (log message trimmed)
[06:05] CIA-20	zeromq2: Version macros added
[06:05] CIA-20	zeromq2: Macro ZMQ_VERSION represents the current version of 0MQ
[06:05] CIA-20	zeromq2: Macro ZMQ_MAKE_VERSION(major,minor,patch) allows to create
[06:05] CIA-20	zeromq2: a representation of the specified version.
[06:05] CIA-20	zeromq2: The versions can be compared using simple <, >, ==, etc.
[06:05] CIA-20	zeromq2: operators.
[06:05] CIA-20	zeromq2: 03Martin Sustrik 07master * r318cdd1 10/ (4 files in 4 dirs):
[06:05] CIA-20	zeromq2: Merge branch 'maint'
[06:05] CIA-20	zeromq2: * maint:
[06:05] CIA-20	zeromq2: Version macros added
[06:05] CIA-20	zeromq2: Conflicts:
[06:05] CIA-20	zeromq2: builds/msvc/platform.hpp
[06:05] CIA-20	zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/br4sKH
[08:47] mikko	sustrik: a lot of builds failing against master
[08:56] pieterh	mikko: particular example?
[09:01] mikko	pieterh: http://valokuva.org:8080/
[09:01] mikko	jzmq_master, erlzmq_master
[09:01] mikko	seem to fail against master
[09:02] pieterh	hey, you got Hudson working :-)
[09:02] mikko	here is the build output: http://valokuva.org:8080/job/jzmq_master/17/console
[09:02] mikko	yeah, getting hudson running took ~2 minutes
[09:02] mikko	adding builds another two hours
[09:02] pieterh	you should have announced this, it's great!
[09:02] mikko	but since it's been grinding pretty steadily
[09:02] mikko	its still a prototype
[09:02] mikko	gotta run, bbl
[09:02] pieterh	cyl
[09:03] pieterh	Socket.cpp:103: error: âZMQ_TYPEâ was not declared in this scope
[10:51] dv	hello
[10:51] dv	i've tried out publish/subscribe with PGM,
[10:51] dv	and wonder about one thing: with PGM multicast, is it possible to have multiple publishers?
[10:56] pieterh	dv_: afaik, yes
[10:57] pieterh	it is not documented
[10:57] pieterh	in the zmq documentation...
[10:59] dv	cool
[10:59] dv	so I can use this for announce/discovery
[11:00] guido_g	multicast is inherently n:m (publisher:subscribers)
[11:00] dv	yeah
[11:00] guido_g	where n and m can be > 1
[11:00] dv	glad to hear zeromq didnt pose any restrictions on that
[11:01] dv	now i'm stuck with an amqp based solution in the company project which uses TCP for announce & discovery, scaling n^2
[11:02] Guthur	well now you can use that experience as a case study on why zmq would be better, hehe
[11:02] dv	heh, yes
[11:02] dv	oh, and can I use the zmq forwarder with PGM publish/subscribe?
[11:03] dv	to tunnel this to another LAN, for instance
[11:03] dv	(over a WAN)
[11:03] guido_g	sure
[11:04] dv	hm. perhaps i can find the budget to rewrite this component. 0mq sounds like a perfect fit.
[11:04] dv	right now, the amqp server is a single point of failure, and eats tons of CPU % when many messages are transmitted
[11:04] Guthur	how much could javascript and HTML support zmq?
[11:04] guido_g	just attach a multicast subsriber and let it forward the traffic it sees via tcp
[11:04] Guthur	HTML5
[11:05] guido_g	Ã¸mq uses a native wire format, so you need a gateway for websocket
[11:05] guido_g	javascript is implementation dependant
[11:06] guido_g	a single js engine might incorporate Ã¸mq, browsers won't i guess
[11:06] dv	i also have possibly found a use for 0mq in a completely different field, not related to the company project at all:
[11:06] dv	messaging between multithreaded subsystems of a game
[11:07] Guthur	Would nice to have browser frontend connecting directly to services served up via 0mq
[11:07] guido_g	Guthur: no, would be an even worse security nightmare then now
[11:08] Guthur	but 'they' do like their browser frontends so
[11:08] dv	guido_g: he doesnt say connect to ALL services
[11:08] dv	perhaps just some frontends
[11:09] dv	some authorization layer of sorts
[11:09] guido_g	an open port is an open port and can (an will) be exploited
[11:09] dv	this is always true for any kind of service
[11:09] Guthur	guido_g, I am more thinking of over company intranets
[11:10] guido_g	Guthur: same thing, see how and from where most of the data is stolen/missused etc.
[11:11] Guthur	guido_g, well we work in a distributed society, I can't see how you avoid these things
[11:11] guido_g	by not connecting everything to every thing else and _hoping_ that it will stay ok
[11:12] guido_g	like it is donw today
[11:12] guido_g	*done
[11:12] dv	i briefly talked to a security guy once, and how difficult it is sometimes to deploy any secure layers. his answer was that if its not difficult, then its insecure. :)
[11:12] guido_g	sigh
[11:15] Guthur	guido_g, I know my company are looking to leverage datamining techniques, which is to be facilitated by a common transport layer connecting an extremely wide variety of data sources which will then be correlated by a middle layer and served up where appropriate, this of course will include browser based frontends
[11:16] Guthur	If zeromq was to be used, for arguments sake, should it only be used in the Transport layer
[11:18] guido_g	if you need browsers and other client side apps, it'll take a while (and cost an enormous amount of money)
[11:18] guido_g	but for (not client facing) backends it'll be fine soon
[11:19] Guthur	guido_g, ok, cheers, food for thought
[11:19] guido_g	np
[11:20] guido_g	i'm also in the process of convincing myself to introduce Ã¸mq in project that lingers here
[11:21] Guthur	I was thinking that it would be a better alternative to a lot of the Java EE based solutions I am seeing
[11:21] guido_g	full ack
[11:22] Guthur	I'm a little disappoint the Java 0mq isn't quite as mature as some of the other bindings
[11:22] guido_g	and given the wide variety of bindings even now you're quite safe for the future
[11:22] guido_g	hehehe
[11:22] dv	btw i guess several people have started to write RPC solutions on top of 0mq. is this correct?
[11:23] guido_g	i fear yes
[11:23] guido_g	poor lost souls
[11:23] dv	i dont mind RPC, as long as you keep it simple (!)
[11:23] guido_g	i do
[11:23] dv	i dread a CORBA 2.0
[11:23] guido_g	it doesn't scale
[11:23] dv	do you know the concept of signals?
[11:24] dv	like, qt signals, libsigc++/boost signals?
[11:24] dv	in c# they are called delegates
[11:24] guido_g	rpc is by definition synchronous, like req/rep in Ã¸mq
[11:24] dv	i'm thinking about writing something similar, but over 0mq,
[11:24] dv	which would be more like pub/sub
[11:25] guido_g	there is req/rep
[11:25] dv	signal emission -> send
[11:25] dv	signal callback invoke -> receive
[11:25] guido_g	sounds more like event-driven
[11:25] guido_g	which is not rpc
[11:25] dv	yeah, true, i use the term very loosely
[11:25] guido_g	the "procedure call" is the problem
[11:26] dv	qt signals are close to what I mean. you have an "emit mysignal(params);" expression, which can marshal the params, stuff them in a message,
[11:26] dv	and upon the next main loop iteration, the corresponding callbacks (called slots) are invoked with the unmarshaled params
[11:26] guido_g	ahhh... messaging for the unaware masses
[11:27] dv	in an earlier call, you connect the signal(s) with the slot(s)
[11:27] dv	works beautifully in practice.
[11:28] dv	boost/libsigc++ signals work a bit differently, they dont marshal anything, the signals internally contain a list of callbacks, which are all invoked when the signal is emitted
[11:28] dv	which is fast, but not thread safe, and does not work over different hosts
[11:28] dv	well, enter 0mq publish/subscribe.
[11:28] guido_g	right
[11:30] mikko	pieterh: this far hudson has been a joy to test with
[11:30] Guthur	guido_g, You mentioned that 0mq would be fine 'soon' for backends, what do you feel is most required to make it ready for 'prime time', so to speak?
[11:31] pieterh	mikko: strange, things build fine from the repo
[11:32] pieterh	guthur: there are a couple of things missing or not fully explored
[11:32] guido_g	Guthur: a bit better error (or should i say condition) reporting
[11:32] mikko	pieterh: it might be something to do with the zeromq2 master build failing
[11:32] mikko	i removed the make check step and retrying
[11:32] mikko	it's waiting for mato's rpath patch
[11:33] pieterh	mikko: I just tried that rebuild by hand and it worked...
[11:33] guido_g	Guthur: much better introspection
[11:33] mikko	pieterh: http://valokuva.org:8080/ zeromq2 master building now and it should kick off all bindings afterwards
[11:34] pieterh	Guthur: better handling of dynamic networks, e.g. heartbeating at socket level
[11:34] Guthur	hehe, sounds challenging
[11:34] mikko	it would be nice to get structured information about peer disconnection as well
[11:34] pieterh	e.g. req/rep just stops working if server crashes
[11:34] guido_g	Guthur: most of the parts are there
[11:34] mikko	subscribe to an inproc and get struct zmq_event_t or something when a peer disconnects
[11:34] mikko	or connects
[11:34] pieterh	also the design and names of xrep/xreq/rep/req need more work IMO
[11:35] pieterh	etc. but all these areas are being actively addressed
[11:35] pieterh	it's quite amazing how many people are working on different parts of 0MQ
[11:36] guido_g	yes, true open source project it is
[11:37] pieterh	guido_g: indeed :-)
[11:40] mikko	now everything builds apart from perl against master branch
[12:10] dv	hm it seems the publisher buffers in the beginning
[12:10] dv	i have to send several messages until the subscribes start receiving anything
[12:11] guido_g	there is an example on how to synchronize pub/sub in the guide
[12:25] pieterh	btw if anyone wants an easy way to write texts like the Guide: http://github.com/imatix/gitdown
[12:25] pieterh	uses github as a kind of publishing medium...
[12:26] dv	hmm the "invalid argument" bug with epgm connections is still there
[13:49] xraid	hi, speaking of guides . where can i find the source to http://www.zeromq.org/code:examples-exchange
[13:54] xraid	also Videoconferencing and Instant Messaging source seems to be amiss ... ?
[14:02] xraid	Gitdown seems very good thanks ... typo "Github is written and maintained by Pieter Hintjens." -- should prolly read Gitdown ...
[15:50] pieterh	seems we lost the source code for those examples somehow
[15:51] pieterh	chat example is here: http://github.com/zeromq/zeromq-chat
[16:05] xraid	yeah got that buts its funky having codesamples sans code ...
[16:06] xraid	and i think i seen the vodeo one perhaps in an earlier source dist ?
[16:12] mikko	finally. hudson moved to dedicated 4 core system
[16:31] pieterh	xraid: these are hanging somewhere, I'll ask around
[16:32] xraid	k thanks
[17:21] CIA-14	zeromq2: 03Christian Gudrian 07master * rf5030a9 10/ (AUTHORS src/select.cpp):
[17:21] CIA-14	zeromq2: Execute the timers before pollset initialisation.
[17:21] CIA-14	zeromq2: Since executing the timers might modify the source pollsets we have
[17:21] CIA-14	zeromq2: to defer the initialisation until after the timers have executed.
[17:21] CIA-14	zeromq2: Signed-off-by: Christian Gudrian <christian.gudrian@fluidon.com> - http://bit.ly/cGdRxZ
[18:48] mikko	lestrrat: zeromq-perl build fails now against maint as well
[18:48] mikko	Error: Cannot parse ALIAS definitions from 'ZMQ_MAKE_VERSION(major, = ZMQ_MAKE_VERSION(major,' in const-xs.inc, line 11