Wednesday March 31, 2010

[Time] NameMessage
[00:14] davelee hi. So i've installed zeromq. There doesn't seem to be any docs about how to actually start the service ... ?
[00:14] davelee (i'm on snow leopard FWIW)
[00:15] davelee there's a zmq_queue binary which wants a config file ..
[05:45] sustrik davelee: what are you trying to do?
[06:44] davelee sustrik: I was just trying to get "up and running"
[06:45] davelee to get the ruby gem demo / test running
[06:45] sustrik let me see...
[06:45] davelee i'm assuming that, like rabbitmq, there's a server you need to run ... ?
[06:45] davelee or is it just a p2p scenario
[06:45] sustrik both
[06:45] sustrik it depends on what you want to do
[06:46] sustrik if you just want to test it, run the perf tests
[06:46] sustrik rbzmq/perf
[06:46] sustrik someting like:
[06:47] sustrik ruby local_thr tcp:// 1 10000
[06:47] sustrik then:
[06:47] sustrik ruby remote_thr.rb tcp:// 1 10000
[06:47] davelee in different terminals (ie, are they communicating)?
[06:48] sustrik yes, in different terminals
[06:48] sustrik or you can try it on two boxes
[06:48] sustrik you have to supply real IP addresses in the latter case of course
[06:48] sustrik (or names)
[06:49] davelee ok, that's the part which confused me. I ran ruby local_lat.rb tcp:// 1 100
[06:49] sustrik yeah, you need the other endpoint
[06:49] davelee and it did nothing - i was expecting it'd want to connect to a broker / etc and complain, at which point I'd go start something
[06:49] davelee yep rgr
[06:51] davelee ok, that seemed to work; local_thr.rb closed with a message about throughput; remote_thr.rb remains running
[06:51] sustrik yup
[06:53] davelee so ... are there any other examples of how to actually use the ruby gem ?
[06:53] davelee the amqp gem is a bit arcane but reasonably easy to figure out by looking at the examples
[06:53] davelee zmq gem has these perf scripts, and i've got a basic idea from looking at them
[06:54] davelee oh right, time to look @ the C bindings ...
[06:55] davelee in that case, I'll assume there's not yet a higher-level ruby library - it's just a wrapper around the C libs?
[06:56] davelee anyway, gotta go. thanks for the help
[09:05] mato sustrik: take a look at my proposals regarding MSG_MORE when you get a moment
[09:05] mato sustrik: i think they are nice and consistent
[09:07] pquerna hi, for $work we use lots of txAMQP and rabbitmq, but am unhappy with various parts. I'm trying to gork zmq, but really it seems to just be a transport layer, abit a cool one that abstracts more than most tcp based transports.
[09:07] pquerna It seems hinted at that you need to build your own mechnisims for configuration management and distribution of how to communicate between machines
[09:08] pquerna is this something zmq intends to do more of, I mean I look at for example , and it makes me feel like those more higher level services just don't exist yet?
[09:09] mato pquerna: the device implementations do exist, but they're still considered early days, so are not documented
[09:09] mato pquerna: however, there are people using e.g. zmq_queue to successfully implement shared queues
[09:10] mato and yes, zmq devices are something that we will be focusing on
[09:12] pquerna okay, and what about the brokerless design -- i read throguh and yeha, I've done this before with my own wire format about 2 jobs back :) but doesn't seem like there are bits on the open source side of zmq that do it yet (?)
[09:13] pquerna or rather, its left as an exercise to the reader...
[09:13] mato well, it depends on what you *need* from the broker
[09:13] mato all the bits are in place, and if you look at recent discussion on the mailing list, you'll see that there is talk of embedding devices inside applications
[09:14] mato so instead of zmq_queue being a binary, it's a call to e.g. zmq_queue_device(...) inside your app
[09:14] pquerna ah
[09:14] mato which means that you decide where your "broker" bits go, if you need them
[09:15] pquerna ya, we have a mix of use cases, some of them definately could use somehting as simple as that.
[09:15] pquerna most of ours are either single writer, lots of readers, or single reader, lots of writers, both of which can fit that just fine.
[09:16] mato yep, and you can probably do a lot of that without any devices at all
[09:16] mato since 0mq sockets let you talk to multiple endpoints
[09:17] mato take a look at the zmq_socket documentation, it describes the messaging patterns you can use
[09:17] mato
[09:17] mato (yes, it's terse, but it's a reference so you should get the idea)
[09:18] mato you might also want to skim the LWN article if you haven't read it already
[09:18] mato
[09:19] pquerna one thing I don't quite understand is error handling, say in the simplest case of a p2p tcp connection -- how much does zmq retry underneath, is there any way to hook in my own logging subsystem to say 'hey tcp ain't working to host xxx'
[09:19] mato no there isn't, and that's kind of deliberate
[09:19] mato since even with tcp you don't get a real/instant notification of e.g. "the other end went away"
[09:19] pquerna sure
[09:20] pquerna (unless you wait for a reply)
[09:20] mato precisely
[09:21] mato which you can also do with 0mq, but the point is that a 0mq socket lets you access an asynchronous queue
[09:21] pquerna right, we just do lots of... multi-data center stuff on cloud servers.
[09:21] pquerna the network can.. be... sucky.
[09:21] mato so you want to detect and respond to that suckiness somehow?
[09:21] mato right?
[09:21] pquerna also I guess no native encryption.. so vpn/ipsec everything?
[09:22] pquerna ya, at least log it a bit, ideally we switch whcih endpoint we are using
[09:22] mato no encryption, so yes, vpn/ipsec... designing good encryption is hard and vpn/ipsec is proven and understood
[09:22] mato we try not to reinvent the wheel
[09:22] pquerna :)
[09:23] mato as for detecting the network...
[09:23] mato what i would suggest is a side channel / monitoring messages using 0mq
[09:23] mato so you periodically ping the other end and test latency
[09:24] mato and if you start getting bad figures back your application can take evasive action/start paging netops people/whatever...
[09:26] pquerna thanks for answering all my questions
[09:26] mato also, flow control (high/low water mark) on sockets has been implemented in 0mq
[09:26] pquerna yes, thank goodness
[09:26] pquerna i actually want a mode where it throws away exploding queues, but meh
[09:27] mato so that can also be used to good effect and may be enough for some applications
[09:27] mato pquerna: you're welcome
[09:27] mato pquerna: have fun, tell a friend, come back, contribute, sponsor, etc etc :-)
[09:37] sustrik mato: test env is running
[09:37] mato sustrik: thanks
[11:12] mato sustrik: ok, pgm on trunk has regressed. see my emails
[11:12] mato sustrik: i don't think there's anything else i can do now
[11:13] sustrik mato: ok, seen that
[11:13] mato sustrik: will have to wait for feedback from steven
[11:13] mato sustrik: so i'll shut down the servers...
[11:13] sustrik can i switch the boxes off now?
[11:13] sustrik ok
[11:14] mato you can unplug it once they all switch off
[12:21] sjampoo Hi, i do not completely understand zmq_init(appthreads, iothreads), are there any pointers with more information? What happens when i access a context with more threads then appthreads specified? Do i need to worry about iothreads, are there any guidelines for its size?
[12:23] sustrik sjampoo: if you've specified say 3 application threads
[12:23] sustrik and have sockets opened from 3 threads
[12:23] sustrik and try to open a socket from 4th thread
[12:23] sustrik you'll get EMTHREAD error
[12:24] sustrik as for I/O threads think of them as of CPU cores
[12:24] sustrik say you have 8-core box
[12:24] sustrik you want 2 cores to handle to background part of 0MQ work
[12:24] sustrik use 2 I/O threads then
[12:25] sjampoo Ok I understand IOThreads now, thanks.
[12:25] sjampoo Concerning appthreads, i am planning to use it in a singe event loop with multiple light weight threads.
[12:25] sustrik each thread will have a socket open, right?
[12:25] sjampoo So i will want to open multiple ZMQ sockets on a single context and use them non-blocking, should i increase the app threads number to the amount of sockets?
[12:26] sustrik it's number of threads you'll be using 0MQ from
[12:26] sustrik so if you are using it from 1 thread, specify 1
[12:26] sustrik is you have your event loop thread + 5 worker threads, specify 6
[12:27] sjampoo aha
[12:27] sjampoo Would 10.000 be a weird big number?
[12:28] sustrik whoa, what kind of application are you writing?
[12:28] sustrik HPC cluster?
[12:28] sjampoo I am planning to integrate it with websockets where each user would get its own ZMQ thread.
[12:28] sustrik you are planning for 10000 threads on a single box?
[12:29] sustrik how many CPU cores there are?
[12:29] sjampoo 10000 lightweight threads
[12:29] sjampoo (with stackless Python)
[12:29] sustrik ah, you mean python pseudo-threads?
[12:29] sustrik or green threads
[12:29] sjampoo yes.
[12:29] sjampoo Green Threads
[12:30] sjampoo Tasklets really
[12:30] sustrik afaik all python green threads map to a single OS thread
[12:30] sustrik so specify 1
[12:30] sjampoo Ok.
[12:31] sjampoo So that appthread does in no way limit the amount of sockets i can open is my real question i think.
[12:31] sjampoo (I mean the appthread setting to zmq_init)
[12:33] sustrik no, it's just number of threads
[12:33] sustrik even if you have 1 thread
[12:34] sustrik you can open 1000 sockets
[12:34] sjampoo ok thanks for clearing that up for me
[12:34] sustrik np
[12:34] sjampoo ZMQ is really nice, did some benchmarks its really fast
[12:34] sustrik we've tried :)
[12:41] sjampoo I also love the design of the website and the documentation. But it took me some time to understand the level at which ZMQ operates, which is pretty low compared to other messaging systems. But, really like it for that.
[12:46] sustrik sjampoo: if you have any suggestion how to organise the documentation better
[12:47] sustrik give us a hint
[12:47] sustrik what kind of info would you have preferred to see first?
[12:47] sustrik etc.
[12:48] sjampoo i'll give it some thought and might get back to it on the mailing list
[12:49] sustrik sjampoo: thanks
[12:49] sjampoo I think the main thing that i know understand is that ZMQ isn't really a messaging system but an awesome socket implementation on which you can build your own MS.
[12:51] sustrik it depends on the exact meaning of "messaging system"
[12:51] sustrik the problem is that the terminology is pretty vague
[12:52] sustrik so what we thought of was ditching all the "messaging middleware" terminology on the front page
[12:52] sustrik and instead presenting what you can do with the product
[12:53] sustrik something like:
[12:53] sustrik "write a server in 10 lines of code"
[12:54] sjampoo I completely agree with that. The problem with my specific case is, that I started looking at ZMQ after i read about a comparison on the SecondLife wiki, it is different than RabbitMQ or ActiveMQ.
[12:54] sustrik yup, but the difference is hard to explain, we've been struggling with explaining it for years
[12:55] sustrik i mean, explaining it in 1-2 paragrahs on the frontpage
[12:55] sustrik so that all kinds of people that come to the website get at least a dim idea of what the hell is it about
[13:16] sjampoo I think a good analogy would be that where ActiveMQ/RabbitMQ gives you a post office, ZMQ gives you couriers. While being lightweight and fast you do need to take care of them yourself.
[13:21] sustrik hm, makes sense
[13:21] sustrik what about "a new layer of Internet stack"
[13:22] sustrik + a picture showing Ethernet, IP, TCP, UDP and 0MQ on top of that?
[13:24] sjampoo yes that makes sense, i personally think of ZMQ as TCP/IP covered in sugar. All the flexibility without the headaches.
[13:25] sjampoo I think it makes a perfect combination with high level languages such as Python.
[13:36] sjampoo I really think it is important to stress the level at which ZMQ operates. I remember that i read the article: 'broker vs brokerless' a while ago and after finishing it started looking for broker/directory service options.
[13:37] sjampoo Now that i understand what ZMQ provides and doesn't provide it makes sense.
[13:50] sustrik ok, i'll try to improve the frontpage...
[14:13] mato sustrik: i had an idea
[14:13] mato sustrik: are you around?
[14:37] CIA-5 zeromq2: 03Martin Hurton 07master * r37fd1a7 10/ src/rep.cpp : Handle full-pipe for REP sockets more gracefully -
[16:10] sjampoo Hmm what is causing the huge interest spike in ZMQ at the moment? I see lots of people bookmarking it at delicious today and yesterday.
[16:14] sjampoo ah, probably the 'AMQP fundamentally flawed' article by Hintjes (which hit Hacker News)
[16:16] squeeky few people tweeted about it.
[16:16] sjampoo yah saw that too
[19:30] mikko howdy