ZeroMq IRC Log

Wednesday March 31, 2010

[Time] Name	Message
[00:14] davelee	hi. So i've installed zeromq. There doesn't seem to be any docs about how to actually start the service ... ?
[00:14] davelee	(i'm on snow leopard FWIW)
[00:15] davelee	there's a zmq_queue binary which wants a config file ..
[05:45] sustrik	davelee: what are you trying to do?
[06:44] davelee	sustrik: I was just trying to get "up and running"
[06:45] davelee	to get the ruby gem demo / test running
[06:45] sustrik	let me see...
[06:45] davelee	i'm assuming that, like rabbitmq, there's a server you need to run ... ?
[06:45] davelee	or is it just a p2p scenario
[06:45] sustrik	both
[06:45] sustrik	it depends on what you want to do
[06:46] sustrik	if you just want to test it, run the perf tests
[06:46] sustrik	rbzmq/perf
[06:46] sustrik	someting like:
[06:47] sustrik	ruby local_thr tcp://127.0.0.1:5555 1 10000
[06:47] sustrik	then:
[06:47] sustrik	ruby remote_thr.rb tcp://127.0.0.1:5555 1 10000
[06:47] davelee	in different terminals (ie, are they communicating)?
[06:48] sustrik	yes, in different terminals
[06:48] sustrik	or you can try it on two boxes
[06:48] sustrik	you have to supply real IP addresses in the latter case of course
[06:48] sustrik	(or names)
[06:49] davelee	ok, that's the part which confused me. I ran ruby local_lat.rb tcp://127.0.0.1:5555 1 100
[06:49] sustrik	yeah, you need the other endpoint
[06:49] davelee	and it did nothing - i was expecting it'd want to connect to a broker / etc and complain, at which point I'd go start something
[06:49] davelee	yep rgr
[06:51] davelee	ok, that seemed to work; local_thr.rb closed with a message about throughput; remote_thr.rb remains running
[06:51] sustrik	yup
[06:53] davelee	so ... are there any other examples of how to actually use the ruby gem ?
[06:53] davelee	the amqp gem is a bit arcane but reasonably easy to figure out by looking at the examples
[06:53] davelee	zmq gem has these perf scripts, and i've got a basic idea from looking at them
[06:54] davelee	oh right, time to look @ the C bindings ...
[06:55] davelee	in that case, I'll assume there's not yet a higher-level ruby library - it's just a wrapper around the C libs?
[06:56] davelee	anyway, gotta go. thanks for the help
[09:05] mato	sustrik: take a look at my proposals regarding MSG_MORE when you get a moment
[09:05] mato	sustrik: i think they are nice and consistent
[09:07] pquerna	hi, for $work we use lots of txAMQP and rabbitmq, but am unhappy with various parts. I'm trying to gork zmq, but really it seems to just be a transport layer, abit a cool one that abstracts more than most tcp based transports.
[09:07] pquerna	It seems hinted at that you need to build your own mechnisims for configuration management and distribution of how to communicate between machines
[09:08] pquerna	is this something zmq intends to do more of, I mean I look at for example http://api.zeromq.org/zmq_queue.html , and it makes me feel like those more higher level services just don't exist yet?
[09:09] mato	pquerna: the device implementations do exist, but they're still considered early days, so are not documented
[09:09] mato	pquerna: however, there are people using e.g. zmq_queue to successfully implement shared queues
[09:10] mato	and yes, zmq devices are something that we will be focusing on
[09:12] pquerna	okay, and what about the brokerless design -- i read throguh http://www.zeromq.org/whitepapers:brokerless and yeha, I've done this before with my own wire format about 2 jobs back :) but doesn't seem like there are bits on the open source side of zmq that do it yet (?)
[09:13] pquerna	or rather, its left as an exercise to the reader...
[09:13] mato	well, it depends on what you need from the broker
[09:13] mato	all the bits are in place, and if you look at recent discussion on the mailing list, you'll see that there is talk of embedding devices inside applications
[09:14] mato	so instead of zmq_queue being a binary, it's a call to e.g. zmq_queue_device(...) inside your app
[09:14] pquerna	ah
[09:14] mato	which means that you decide where your "broker" bits go, if you need them
[09:15] pquerna	ya, we have a mix of use cases, some of them definately could use somehting as simple as that.
[09:15] pquerna	most of ours are either single writer, lots of readers, or single reader, lots of writers, both of which can fit that just fine.
[09:16] mato	yep, and you can probably do a lot of that without any devices at all
[09:16] mato	since 0mq sockets let you talk to multiple endpoints
[09:17] mato	take a look at the zmq_socket documentation, it describes the messaging patterns you can use
[09:17] mato	http://api.zeromq.org/zmq_socket.html
[09:17] mato	(yes, it's terse, but it's a reference so you should get the idea)
[09:18] mato	you might also want to skim the LWN article if you haven't read it already
[09:18] mato	http://lwn.net/Articles/370307/
[09:19] pquerna	one thing I don't quite understand is error handling, say in the simplest case of a p2p tcp connection -- how much does zmq retry underneath, is there any way to hook in my own logging subsystem to say 'hey tcp ain't working to host xxx'
[09:19] mato	no there isn't, and that's kind of deliberate
[09:19] mato	since even with tcp you don't get a real/instant notification of e.g. "the other end went away"
[09:19] pquerna	sure
[09:20] pquerna	(unless you wait for a reply)
[09:20] mato	precisely
[09:21] mato	which you can also do with 0mq, but the point is that a 0mq socket lets you access an asynchronous queue
[09:21] pquerna	right, we just do lots of... multi-data center stuff on cloud servers.
[09:21] pquerna	the network can.. be... sucky.
[09:21] mato	so you want to detect and respond to that suckiness somehow?
[09:21] mato	right?
[09:21] pquerna	also I guess no native encryption.. so vpn/ipsec everything?
[09:22] pquerna	ya, at least log it a bit, ideally we switch whcih endpoint we are using
[09:22] mato	no encryption, so yes, vpn/ipsec... designing good encryption is hard and vpn/ipsec is proven and understood
[09:22] mato	we try not to reinvent the wheel
[09:22] pquerna	:)
[09:23] mato	as for detecting the network...
[09:23] mato	what i would suggest is a side channel / monitoring messages using 0mq
[09:23] mato	so you periodically ping the other end and test latency
[09:24] mato	and if you start getting bad figures back your application can take evasive action/start paging netops people/whatever...
[09:26] pquerna	thanks for answering all my questions
[09:26] mato	also, flow control (high/low water mark) on sockets has been implemented in 0mq
[09:26] pquerna	yes, thank goodness
[09:26] pquerna	i actually want a mode where it throws away exploding queues, but meh
[09:27] mato	so that can also be used to good effect and may be enough for some applications
[09:27] mato	pquerna: you're welcome
[09:27] mato	pquerna: have fun, tell a friend, come back, contribute, sponsor, etc etc :-)
[09:37] sustrik	mato: test env is running
[09:37] mato	sustrik: thanks
[11:12] mato	sustrik: ok, pgm on trunk has regressed. see my emails
[11:12] mato	sustrik: i don't think there's anything else i can do now
[11:13] sustrik	mato: ok, seen that
[11:13] mato	sustrik: will have to wait for feedback from steven
[11:13] mato	sustrik: so i'll shut down the servers...
[11:13] sustrik	can i switch the boxes off now?
[11:13] sustrik	ok
[11:14] mato	you can unplug it once they all switch off
[12:21] sjampoo	Hi, i do not completely understand zmq_init(appthreads, iothreads), are there any pointers with more information? What happens when i access a context with more threads then appthreads specified? Do i need to worry about iothreads, are there any guidelines for its size?
[12:23] sustrik	sjampoo: if you've specified say 3 application threads
[12:23] sustrik	and have sockets opened from 3 threads
[12:23] sustrik	and try to open a socket from 4th thread
[12:23] sustrik	you'll get EMTHREAD error
[12:24] sustrik	as for I/O threads think of them as of CPU cores
[12:24] sustrik	say you have 8-core box
[12:24] sustrik	you want 2 cores to handle to background part of 0MQ work
[12:24] sustrik	use 2 I/O threads then
[12:25] sjampoo	Ok I understand IOThreads now, thanks.
[12:25] sjampoo	Concerning appthreads, i am planning to use it in a singe event loop with multiple light weight threads.
[12:25] sustrik	each thread will have a socket open, right?
[12:25] sjampoo	So i will want to open multiple ZMQ sockets on a single context and use them non-blocking, should i increase the app threads number to the amount of sockets?
[12:26] sustrik	it's number of threads you'll be using 0MQ from
[12:26] sustrik	so if you are using it from 1 thread, specify 1
[12:26] sustrik	is you have your event loop thread + 5 worker threads, specify 6
[12:27] sjampoo	aha
[12:27] sjampoo	Would 10.000 be a weird big number?
[12:28] sustrik	whoa, what kind of application are you writing?
[12:28] sustrik	HPC cluster?
[12:28] sjampoo	I am planning to integrate it with websockets where each user would get its own ZMQ thread.
[12:28] sustrik	you are planning for 10000 threads on a single box?
[12:29] sustrik	how many CPU cores there are?
[12:29] sjampoo	10000 lightweight threads
[12:29] sjampoo	(with stackless Python)
[12:29] sustrik	ah, you mean python pseudo-threads?
[12:29] sustrik	or green threads
[12:29] sjampoo	yes.
[12:29] sjampoo	Green Threads
[12:30] sjampoo	Tasklets really
[12:30] sustrik	afaik all python green threads map to a single OS thread
[12:30] sustrik	so specify 1
[12:30] sjampoo	Ok.
[12:31] sjampoo	So that appthread does in no way limit the amount of sockets i can open is my real question i think.
[12:31] sjampoo	(I mean the appthread setting to zmq_init)
[12:33] sustrik	no, it's just number of threads
[12:33] sustrik	even if you have 1 thread
[12:34] sustrik	you can open 1000 sockets
[12:34] sjampoo	ok thanks for clearing that up for me
[12:34] sustrik	np
[12:34] sjampoo	ZMQ is really nice, did some benchmarks its really fast
[12:34] sustrik	we've tried :)
[12:41] sjampoo	I also love the design of the website and the documentation. But it took me some time to understand the level at which ZMQ operates, which is pretty low compared to other messaging systems. But, really like it for that.
[12:46] sustrik	sjampoo: if you have any suggestion how to organise the documentation better
[12:47] sustrik	give us a hint
[12:47] sustrik	what kind of info would you have preferred to see first?
[12:47] sustrik	etc.
[12:48] sjampoo	i'll give it some thought and might get back to it on the mailing list
[12:49] sustrik	sjampoo: thanks
[12:49] sjampoo	I think the main thing that i know understand is that ZMQ isn't really a messaging system but an awesome socket implementation on which you can build your own MS.
[12:51] sustrik	it depends on the exact meaning of "messaging system"
[12:51] sustrik	the problem is that the terminology is pretty vague
[12:52] sustrik	so what we thought of was ditching all the "messaging middleware" terminology on the front page
[12:52] sustrik	and instead presenting what you can do with the product
[12:53] sustrik	something like:
[12:53] sustrik	"write a server in 10 lines of code"
[12:54] sjampoo	I completely agree with that. The problem with my specific case is, that I started looking at ZMQ after i read about a comparison on the SecondLife wiki, it is different than RabbitMQ or ActiveMQ.
[12:54] sustrik	yup, but the difference is hard to explain, we've been struggling with explaining it for years
[12:55] sustrik	i mean, explaining it in 1-2 paragrahs on the frontpage
[12:55] sustrik	so that all kinds of people that come to the website get at least a dim idea of what the hell is it about
[13:16] sjampoo	I think a good analogy would be that where ActiveMQ/RabbitMQ gives you a post office, ZMQ gives you couriers. While being lightweight and fast you do need to take care of them yourself.
[13:21] sustrik	hm, makes sense
[13:21] sustrik	what about "a new layer of Internet stack"
[13:22] sustrik	+ a picture showing Ethernet, IP, TCP, UDP and 0MQ on top of that?
[13:24] sjampoo	yes that makes sense, i personally think of ZMQ as TCP/IP covered in sugar. All the flexibility without the headaches.
[13:25] sjampoo	I think it makes a perfect combination with high level languages such as Python.
[13:36] sjampoo	I really think it is important to stress the level at which ZMQ operates. I remember that i read the article: 'broker vs brokerless' a while ago and after finishing it started looking for broker/directory service options.
[13:37] sjampoo	Now that i understand what ZMQ provides and doesn't provide it makes sense.
[13:50] sustrik	ok, i'll try to improve the frontpage...
[14:13] mato	sustrik: i had an idea
[14:13] mato	sustrik: are you around?
[14:37] CIA-5	zeromq2: 03Martin Hurton 07master * r37fd1a7 10/ src/rep.cpp : Handle full-pipe for REP sockets more gracefully - http://bit.ly/aAszD1
[16:10] sjampoo	Hmm what is causing the huge interest spike in ZMQ at the moment? I see lots of people bookmarking it at delicious today and yesterday.
[16:14] sjampoo	ah, probably the 'AMQP fundamentally flawed' article by Hintjes (which hit Hacker News)
[16:16] squeeky	few people tweeted about it.
[16:16] sjampoo	yah saw that too
[19:30] mikko	howdy