ZeroMq IRC Log

Tuesday July 19, 2011

[Time] Name	Message
[01:29] rev	whoever set up this irc client on the zmq webpage - thanks a lot! my office won't let me install an irc client so this is great.
[01:55] michelp	rev, glad you could join us :)
[13:46] ptrb	any problem to re-close an already closed socket?
[13:47] mikko	why does it need to be re-closed?
[13:47] sustrik	sure, it's a problem
[13:47] sustrik	the socket is deallocated during first close
[13:48] sustrik	second close accesses non-existent object
[13:48] ptrb	I see. Does C++ destructor trigger a close()?
[13:48] sustrik	yes
[13:48] ptrb	I see.
[13:49] sustrik	however, in C++ binding destructor tests whether socket was closed explicitly before
[13:49] sustrik	and if so, it doesn do zmq_close()
[13:49] ptrb	So, assuming RAII ownership model of a socket, I should never clo... oh, alright.
[13:50] ptrb	Very good! Thanks.
[14:27] amirpc	Hey guys I'm putting together an architecture document for a proposal at work and one thing I'm looking to put in there is current production use cases, to try to head of dissenters before they can raise the issue - does anyone know where I can find some good examples?
[14:28] mikko	amirpc_: you have read the guide i assume?
[14:28] mikko	one company i know using zeromq is loggly
[14:28] mikko	they speak quite openly about it
[14:28] amirpc	mikko: Yep, I have read the guide but I read code samples, etc a long time ago - is there prod info in it? gonna go check
[14:29] mikko	what sort of info are you looking for?
[14:29] mikko	just people using zeromq in production?
[14:30] amirpc	Yeah, if I don't have examples then the change fearers will rapidly take the conversation in a bad direction
[14:30] amirpc	just other people I can say who are using it
[14:33] mikko	you can probbaly spot such a companies from mailing-lists
[14:34] amirpc	okay I'll give that a shot
[14:34] taotetek	amirpc_: we (aggregate knowledge) use zeromq in production
[14:34] taotetek	amirpc_: we have a few blog posts up about it on our corporate blog if that's helpful
[14:35] taotetek	amirpc_: http://blog.aggregateknowledge.com/
[14:36] taotetek	amirpc_: we use rsyslog as an "artery" to steam log data to services that then distribute log data to various parsing and analaysis systems via zeromq
[14:36] taotetek	s/steam/stream
[14:37] pieterh	amirpc_: there are a number of 'stories' on the zeromq.org site: http://www.zeromq.org/story:_start
[14:37] pieterh	there could be more, we've not been very good about collecting them
[14:37] taotetek	oh hi pieterh :)
[14:37] pieterh	hi Brian!
[14:38] pieterh	ohai
[14:38] taotetek	pieterh: just sitting down to work on some prototypes and play more with jruby + zeromq
[14:38] pieterh	saw that, nice...
[14:38] taotetek	pieterh: heading out to HQ in california in a couple of weeks trying to get some good examples together for the dev team
[14:39] pieterh	how many devs?
[14:40] amirpc	pieterh: Do you know what our largest scale deployment is? They'll ask about that, we've got a few million users a day and this will be spread across prob 200 individual nodes, ever seen anything that big?
[14:40] taotetek	pieterh: we've got hmmm.. just 4 engineers in the dev group, well plus the cto does a lot of coding and I do a lot of coding
[14:40] taotetek	pieterh: so I guess 6 or so of us writing code on a regular basis
[14:41] pieterh	amirpc_: no idea what the largest scale 0MQ deployment is, but I know for a fact some people are aiming at 10M+node networks
[14:41] pieterh	taotetek: have you read http://unprotocols.org/blog:13?
[14:41] amirpc	okay if I wanted to find more info about that, mailing list is best or?
[14:41] taotetek	pieterh: oh and ken writes code of course when he's contracting for us
[14:41] pieterh	amirpc_: ask on zeromq-dev... but many large deployments are discrete about their use of 0MQ
[14:42] pieterh	taotetek: sounds fun... :)
[14:42] taotetek	pieterh: yup! it was a nice article
[14:42] amirpc	okay well I just really want to use zeromq in my project but I'm going up against a decent number of people who are afraid of change, so any advice you can give would be helpful :)
[14:42] pieterh	i've used this approach a few times now, it works well to go from hello world to real prototypes
[14:43] pieterh	amirpc_: if you are fighting emotional arguments (fear), don't come with technical answers
[14:43] taotetek	pieterh: we used a version of "urban protocol" for our first workflow system speaking of unprotocols ;)
[14:44] pieterh	taotetek: :-) nice
[14:44] taotetek	pieterh: experimenting with majordomo now
[14:44] amirpc	lol I appreciate that, but it's from technical people so they fear-monger using antiquated examples of how change has introduced instaiblities before
[14:44] pieterh	I did this with a team, wrote a protocol and we then made stacks
[14:44] pieterh	and then we switched to a OHAI form for the protocol and they loved it
[14:45] pieterh	amirpc_: there are some _very_ large firms starting to use 0MQ
[14:45] taotetek	pieterh: yup. it made people laugh and got some "ICANHAZ?" jokes flying around for a few days
[14:45] taotetek	amirpc_: we don't have a "wide" system (not a lot of nodes), but we run a lot of data through it
[14:45] pieterh	amirpc_: the truth is that technical change is always risky
[14:46] pieterh	there is no point in arguing otherwise, accept it, work with it
[14:46] taotetek	amirpc_: 500 to 700 million "events" a day
[14:46] pieterh	so to reduce the risk of change, you need a number of strategies
[14:46] pieterh	such as rapid prototyping, portability of OS and language, and wide, supportive community of users
[14:47] taotetek	amirpc_: I'll tell you this - it hasn't been as entirely smooth as we'd hoped (what ever is), and we did run into a surprise here or there in the library that threw us a curve ball or two
[14:47] taotetek	amirpc_: but we've not yet ever regretted the decision to go with zeromq as a core technology
[14:47] pieterh	amirpc_: essentially, a message-based architecture is extremely good at reducing risk
[14:48] amirpc	taotetek: Are you allowed to say what company it is, and/or what curve balls you've seen?
[14:48] taotetek	amirpc_: http://blog.aggregateknowledge.com/ that's us
[14:48] pieterh	amirpc_: where are you based?
[14:49] amirpc	SF
[14:49] taotetek	amirpc_: one of the curves was just conceptual. it took a little bit to fully grok that zeromq is not a messaging solution so much as it is a toolkit to enable you to build your solutions
[14:50] pieterh	You might want to contact dotcloud, they're quite large 0MQ users in SF
[14:50] pieterh	they built their cloud architecture using AMQP/RabbitMQ and then moved to 0MQ
[14:50] taotetek	amirpc_: the second was that the "high water mark" settings don't quite do entirely what we first thought
[14:50] pieterh	taotetek: if you need it, I have a good little credit-based flow control model working
[14:51] taotetek	amirpc_: e.g. with push / pull pipeline you will still end up with quite a few messages in transit in the buffers no matter what you set the HWM at
[14:51] pieterh	avoids the need for HWMs on router-based architectures
[14:51] taotetek	pieterh: I'd be interested in seeing it
[14:51] taotetek	pieterh: the latest things we're working on are using REQ / REP and just allowing the workers to ask for more jobs when they want them
[14:51] taotetek	pieterh: avoids routing, avoids HWM issues, etc
[14:52] amirpc	What I want to use it for is less for the queue and more for the durability
[14:52] pieterh	taotetek: yes, Majordomo
[14:52] amirpc	and flexibility to allow novel architectures to arrise
[14:52] pieterh	amirpc_: you do need to read the Guide and internalize the "language"
[14:53] taotetek	pieterh: yup, the next step we're taking is majordomo - allowing workers to register the services they can handle, etc
[14:53] amirpc	I've read the guide and implemented several projects with zeromq already actually
[14:53] taotetek	pieterh: our first production system was a bunch of task workers that all perform the exact same task so it was a good, simple place to start.
[14:53] taotetek	pieterh: now that people feel comfortable with that we'll evolve
[14:54] pieterh	taotetek: I'm posting that CBFC example now, with some explanation
[14:54] pieterh	amirpc_: if you get to the stage where you need some evangelism / explanation, I'm sometimes in SFO, and happy to travel there
[14:55] taotetek	pieterh: excellent, looking forward to reading it
[14:56] amirpc	pieterh: That's good to hear. I probably will either sink or swim today @ 5pm cst, hehe. If they don't get hung up on zeromq, this project has 10s of millions of dollars behind it and staggering reach
[14:59] chrismurf	Hello all.
[15:00] pieterh	taotetek: http://unprotocols.org/blog:15
[15:02] pieterh	chrismurf: hi
[15:02] taotetek	excellent, thanks pieterh
[15:03] pieterh	it requires bidirectional sockets, so router-dealer is the usual choice
[15:03] chrismurf	Are there architectural obstacles to implementing a non-reliable multicast protocol in zeroMQ? I would like the option to use UDP Multicast for some messages, and wonder if that's something that can be done /in/ 0mq.
[15:03] pieterh	with the new generic sockets in 0MQ/3.x, this will work even better
[15:03] sustrik	chrismurf: no, there's no real problem
[15:04] pieterh	chrismurf: I have a UDP framework but it's rather complex still - https://github.com/imatix/vtx/tree/master/v3
[15:04] pieterh	sustrik: lol
[15:04] chrismurf	sustrik, we have been using LCM (code.google.com/p/lcm/) which is UDPM based
[15:04] sustrik	is that UDP multicast
[15:04] chrismurf	yes
[15:04] sustrik	pieter: i though it was UDP unicast
[15:05] pieterh	sustrik: that's UDP unicast but it's not that different
[15:05] chrismurf	considering switching to 0mq and google protobuf, but I'm not sure of all the consequences
[15:05] pieterh	it already for example does broadcast connects (i.e. connect to any IP address)
[15:05] sustrik	chrismurf: you can take src/pgm_sender.hpp/.cpp and src/pgm_receiver.hpp/.cpp as a template
[15:05] sustrik	remove the PGM stuff and replace it by UDP
[15:05] sustrik	should be easy
[15:05] chrismurf	sustrik, thanks - perfect.
[15:06] fintler	chrismurf: if you plan on doing a ton of rpc, consider thrift
[15:06] chrismurf	fintler, I'll look at it, but I'm not RPC-heavy
[15:06] chrismurf	mostly just data sharing
[15:07] chrismurf	IPC on an autonomous underwater robot
[15:07] fintler	proto buffers is probably better for just data
[15:09] chrismurf	pieterh, sustrik, fintler, thanks!
[15:09] pieterh	chrismurf: what kind of traffic volume are you doing?
[15:09] chrismurf	well, the problem is that some of the network cables go through wet-mateable cables
[15:10] chrismurf	which can have HORRIBLE SNR
[15:10] chrismurf	and we have gigabit ethernet based cameras, trying to go through a 100Mb link, through wet-mated connectors, toss in a little seawater...
[15:11] chrismurf	steady state traffic isn't huge
[15:11] pieterh	do you need pubsub, or is point-to-point OK?
[15:12] pieterh	so look at the vtx_test_udp.c source in that project I pointed to
[15:12] chrismurf	it's a research vehicle, so we're trying to switch to pubsub, so that it is easy to write standalone applications that can consume data without running in a critical process
[15:12] pieterh	it demonstrates all the 0MQ socket types over UDP
[15:12] pieterh	including pub-sub, which is lossy
[15:12] pieterh	request-reply will do basic reliability
[15:13] pieterh	it should be simple to plug into your code:
[15:13] pieterh	#include "vtx.c"
[15:13] pieterh	#include "vtx_udp.c"
[15:13] chrismurf	pieterh, thanks
[15:13] pieterh	it is really not meant for public use yet, but it handles nicely in tests
[15:13] pieterh	also it's not fast, does blocking writes
[15:14] pieterh	however it effectively gives you 0MQ sockets connected via UDP in all the different patterns
[15:14] chrismurf	okay
[15:15] chrismurf	I have been happy with the way LCM handles UDPM (ie - invisibly) so I may also look at how complex their implementation is (both are LGPL)
[15:16] pieterh	I guess the benefit of using 0MQ would be you get a single API and framework for all different kinds of transport
[15:16] pieterh	UDP to your seawater cameras, TCP to other boxes on the network, inproc to other threads in the process
[15:17] chrismurf	pieterh, sort of, yeah - we're looking at using TCP for some critical messaging (ABORT!) but generally udpm/pubsub for message passing so things remain modular
[15:18] chrismurf	we have a mass-o-threads right now, which is impossible to test, debug, and develop for
[15:18] chrismurf	and message passing is a mix of UDP and TCP and shared memory
[15:18] pieterh	chrismurf: 0MQ makes that really simple
[15:18] chrismurf	so tracking data flow is impossible
[15:18] pieterh	indeed
[15:19] chrismurf	so we want to switch to a bunch of separate processes, with pubsub between them.
[15:19] chrismurf	but udpm falls down nicely as traffic goes up, and I don't know enough about pgm
[15:19] chrismurf	and there are scary sounding caveats about loopback interfaces and the like on the 0mq website ;-)
[15:20] chrismurf	I've found we start running code on one pc, and then it's on two, and then it's on one again, etc... udpm pub/sub means we don't have to care.
[15:21] chrismurf	why are there no unreliable transports in 0mq now? Just lack of need by the developers?
[20:59] mikko	pieterh: here?
[20:59] pieterh	mikko: hi!
[20:59] mikko	looks like 2-1 and 2-2 builds on win are failing
[20:59] pieterh	just writing my book
[20:59] mikko	not sure if the builds are wrong or a new patch
[21:00] pieterh	I saw that, not sure how that's possible, seems int64_t isn't defined...
[21:00] mikko	ermm, 2-2 only
[21:00] pieterh	czmq or zeromq2-2?
[21:00] mikko	mailbox.obj : error LNK2001: unresolved external symbol "public: void __thiscall zmq::signaler_t::recv(void)" (?recv@signaler_t@zmq@@QAEXXZ)
[21:00] mikko	http://build.zero.mq/job/libzmq2-2_MSVC-win7/217/console
[21:01] mikko	http://build.zero.mq/job/libzmq2-2_mingw32-win7/207/
[21:01] mikko	looks like one of those backports broke it
[21:02] pieterh	hmm...
[21:02] pieterh	LINK : warning LNK4098: defaultlib 'MSVCRTD' conflicts with use of other libs; use /NODEFAULTLIB:library
[21:02] mikko	thats fine
[21:03] mikko	openpgm debug build
[21:03] pieterh	normally there are the same backports to 2.1 as 2.2
[21:03] mikko	http://build.zero.mq/job/libzmq2-2_mingw32-win7/207/console
[21:03] mikko	it happens with mingw32 as well
[21:03] mikko	signaler.cpp: In member function 'void zmq::signaler_t::send()':
[21:03] mikko	signaler.cpp:123:54: error: invalid conversion from 'unsigned char' to 'const char'
[21:03] pieterh	let me check that...
[21:04] mikko	another one is signaler.cpp: In member function 'void zmq::signaler_t::recv()':
[21:04] mikko	signaler.cpp:197:54: error: invalid conversion from 'unsigned char' to 'char'
[21:05] pieterh	ok, so I'd expect it to fail on 2.1 as well
[21:05] mikko	let me check 2.1
[21:05] pieterh	same source on 2.1 and 2.2 for this file
[21:05] mikko	http://build.zero.mq/job/libzmq2-1_mingw32-win7/217/console
[21:05] mikko	same fail
[21:06] pieterh	ok, I see the problem, code was fixed in 3.x but not in the back port
[21:06] pieterh	30 seconds...
[21:07] rev	mikko: are you working on http://code.google.com/p/hudson/?
[21:08] mikko	rev_: nope
[21:08] pieterh	mikko: that should do it now
[21:08] pieterh	thanks for catching this...
[21:08] mikko	rev_: i've done occasional patches for jenkins
[21:09] rev	mikko: ok i saw 'hudson' in the build output and just wondered.
[21:09] mikko	rev_: it's a jenkins build cluster for zeromq
[21:09] mikko	runs a few operating systems and compilers
[21:09] mikko	http://www.zeromq.org/docs:builds
[21:10] rev	mikko: gotcha
[21:21] devinus	what message format would you guys use for RPC?
[21:24] mikko	devinus: depends
[21:24] mikko	devinus: i was just reading this http://news.ycombinator.com/item?id=2491997
[21:26] devinus	msgpack was something i had in mind
[21:26] devinus	right now we're doing jsonrpc over http
[21:27] devinus	thinking of using zeromq for the service
[21:34] whack	I've found msgpack quite nice doing rpc in C
[21:34] whack	only because the msgpack library makes it easy to write data structures; I could just as soon use xdr but fewer languages support that