Friday January 28, 2011

[Time] NameMessage
[00:54] stodge Do I need to install openpgm for zeroconf to support the protocol?
[01:06] stodge Or do I compile zeromq with pgm support
[06:37] stockMQ Hi I am using pgmping to do some tests
[06:37] stockMQ Anyone here using it?
[06:44] sustrik mikko: what size did you set up?
[06:52] CIA-21 zeromq2: 03Martin Sustrik 07master * r1b15eba 10/ src/socket_base.cpp :
[06:52] CIA-21 zeromq2: Fixed the problem of subscription forwarding and PGM interaction
[06:52] CIA-21 zeromq2: Signed-off-by: Martin Sustrik <> -
[07:01] stockMQ ??
[07:05] Evet zeromq client-side javascript binding would be great
[07:06] Evet s/javascript/html5/
[07:34] sustrik Evet: afaiu java script doesn't allow for invoking C binaries
[07:35] Evet sustrik: protocol compatibility with websockets isnt enough?
[07:37] sustrik well, you need some code
[07:38] sustrik compatibility is an abstract concept
[07:38] sustrik no much help
[09:21] boothead Hi all, is there an easy (or possible) way to get a list of a socket's connections or binds... I'm using pyzmq
[09:24] thoward hello
[09:24] thoward anyone here familiar with pyzmq on windows?
[09:25] boothead thoward - i've built it using VS, but that's about it!
[09:25] mikko thoward: not very
[09:25] mikko thoward: but there are many pyzmq+win users here
[09:26] thoward i've had success building it on windows and so far there have been no problems there..
[09:26] thoward i'm now trying to deploy it to a box that doesn't have a compiler on it..
[09:26] thoward i'm having no success with that
[09:39] sustrik what's the error?
[09:39] sustrik mikko: how large buffer did you used on bsd?
[09:39] thoward sustrik: DLL Load failed
[09:40] sustrik bad paths?
[09:40] thoward doesn't say which DLL. have tried the normal tricks (updated path to include location of libzmq.dll and copied the dll all over the place
[09:40] sustrik you can also try to check the dll with depeds.exe
[09:40] sustrik depends.exe
[09:40] sustrik it'll show missing dlls in red
[09:41] thoward you mean dependencies of libzmq.dll?
[09:41] sustrik yes
[09:41] mikko sustrik: let me check
[09:41] mikko sustrik:
[09:41] mikko kern.ipc.maxsockbuf=4000000
[09:41] mikko net.inet.tcp.sendbuf_max=16777216
[09:41] mikko net.inet.tcp.recvbuf_max=16777216
[09:42] mikko are those right buffers?
[09:42] sustrik nope
[09:42] sustrik it's socket pair
[09:42] sustrik so no TCP is involved
[09:42] sustrik it's unix domain socket afaict
[09:42] mikko sustrik: kern.ipc.maxsockbuf ?
[09:42] sustrik no idea
[09:43] sustrik maybe you can retrieve the buffer size in mailbox.cpp
[09:43] sustrik when the socketpair is created
[09:43] sustrik and print it out
[09:44] sustrik that way you can check whether the setting had the desirable effect
[10:12] thoward yay! i got it working.
[10:12] thoward thanks to:
[10:12] thoward
[10:25] sustrik nice
[10:47] sustrik no issues i am aware of
[10:47] sustrik what version is that?
[10:48] sustrik it looks like the I/O threads are still working, receiving messages
[10:48] sustrik but the application thread doesn't get notified about the traffic
[10:48] sustrik it looks like device problem imo
[10:49] sustrik i was just deducing because of the growing memory you referred to
[10:50] sustrik hm
[10:50] sustrik if it was completely stuck i would just remain on stable mem usage
[10:51] sustrik i assume there's no way to reproduce the problem?
[10:52] sustrik i guess you'll have to attach debugger to the stuck device and find out what's going on then
[10:55] sustrik yes, first have a look at backtrace
[10:55] sustrik all the threads in the process
[10:56] sustrik using gdb?
[10:56] sustrik info threads
[10:57] sustrik that will give you list of all threads
[10:57] sustrik then thread 1
[10:57] sustrik to switch to thread 1
[10:57] sustrik thread 2
[10:57] sustrik to switch to thread 2 etc.
[10:57] sustrik np
[11:04] mikko on the topic of gdb this might be interesting as well
[11:08] sustrik nice, i wasn't aware -ggdb3 thing
[11:35] zchrish I am building a zeromq application and would like to stress test it. Can anyone point me to a free or low-cost server farm whereby I can connect 1000-10000 clients?
[11:40] zchrish I have a set of server(s) that I want to test and would like to use this server farm to simulate a "client cloud". I realize it isn't terribly realistic because, in use, my application would be accessed by people on the internet across the world.
[11:40] zchrish But at least I would like to test the "number" of client connects.
[11:47] zchrish Has anyone done this before? If so, what types of issues should I expect and, therefore, plan on. For example, I expect my server to hang from time-2-time and will need to kill and restart it but I don't necessarily have a strategy in mind yet.
[11:57] CIA-21 zeromq2: 03Martin Sustrik 07sub-forward * r95356d6 10/ (5 files):
[11:57] CIA-21 zeromq2: Re-creating subscription pipes after disconnect
[11:57] CIA-21 zeromq2: Signed-off-by: Martin Sustrik <> -
[12:06] sustrik zchrish: no idea, sorry
[12:09] zchrish sustrik: Has anyone built an application that serves 1000-100000 client yet? I will be using a combination of REQ/REP and PUB/SUB to communicate with my central servers and these clients.
[12:10] sustrik dermoth|home just mentioned having 3000 connections
[12:13] zchrish Oh, that's great and encouraging. I am staying close (read exact) to the model patterns described in "The Guide" to prepare my server software. I saw a company named "Sabalcore" that rents computer time.
[12:14] zchrish Anyway, it is somewhat of an unknown to me now what types of issues will pop-up since the zeromq software is designed to shield this level of detail from my view which is ok.
[12:54] mikko i think Skaag has a fairly large test environment
[12:54] mikko and plans to use it on large production environment
[12:55] mikko if i remember correctly
[12:55] Skaag it's already being used right now
[12:55] Skaag our test cluster has 16 ~ 20 nodes
[12:55] mikko wow, that happened quick
[12:55] Skaag not really, 2 weeks ;0
[12:55] Skaag :-)
[12:56] mikko i'm used to working with enterprises
[12:56] mikko where two weeks is spent on planning the planning phase
[12:56] Skaag right, you're 100% correct ;)
[12:57] Skaag we calculated bottlenecks, planned our message sizes and structure carefully, and just went ahead with it
[12:57] zchrish I am using a master that will fan-out to a configurable number of intermdiaries. Is there a rule of thumb to know when to add another node? I suppose I can either (1) theoretically determine the load or (2) watch the activity and add another node based on heuristics.
[12:57] Skaag and I have to say it's looking very good now
[12:58] Skaag zchrish: the way we plan to do it, is based on actual load + some other factors (strategic)
[12:58] zchrish I see; thank you.
[13:00] mikko
[13:00] mikko cool
[13:06] Skaag yes :)
[13:07] Skaag my next task, is to replicate the functionality on our own system
[13:07] Skaag and then make some really cool and interactive web based widgets with that
[13:07] Skaag that involve live video streaming, etc.
[13:08] Skaag and the next big step is synchronizing flash objects between many media servers, and that will really be game changing for the media industry.
[13:08] Skaag hopefully, all this can also happen in the next 2 ~ 3 weeks.
[13:11] sustrik changing the media industry in 2-3 weeks, wow! :)
[13:12] sustrik anyway, the idea is to build large-scale distribution trees for media delivery?
[13:12] mikko i think UDT would of interest for the media industry
[13:13] sustrik my impression was that UDT makes sense when you are trying to transfer you 10TB database over the WAN
[13:13] sustrik no idea about UDT & media
[13:13] sustrik UDT & real time to be more precise
[13:14] Skaag video distribution, we do already, with Wowza
[13:14] Skaag WowzaMediaServer is a replacement for Adobe's FMS
[13:14] Skaag but imagine a developer wants to create a large scale application right now - he can't really...
[13:15] Skaag he's limited by what a single FMS can provide
[13:15] sustrik what's FMS?
[13:15] Skaag when you talk to Adobe about FMS, they talk to you in terms of hundreds of channels
[13:15] Skaag FMS = Flash Media Server
[13:15] Skaag with all its cool object synchronization, etc.
[13:15] sustrik i see
[13:15] Skaag So I want to attack that + also do this for html5 with json object sync
[13:16] Skaag you send the server some update to an object, all other subscribers receive it at once
[13:16] Skaag like really
[13:16] Skaag I wonder what they use in their backends in place of zmq
[13:18] sustrik aha
[13:18] sustrik so basically large-scale delivery of PUB/SUB streams
[13:19] sustrik have you seen my article about scaling 0mq-like system to internet scale?
[13:19] sustrik
[13:21] sustrik one of the points is that with such a system any participant can add new devices to the distribution tree
[13:21] sustrik thus making it infinitely scalable
[13:22] sustrik this applies not only to the provider of the service but also to end-user, ISPs etc. virtually everyone
[14:03] Skaag I need to read it because i'm not sure about the infinity thing :)
[14:23] CIA-21 zeromq2: 03Martin Sustrik 07sub-forward * rf267287 10/ (src/ src/matcher.cpp src/matcher.hpp):
[14:23] CIA-21 zeromq2: Simple ad hoc message matcher added
[14:23] CIA-21 zeromq2: Signed-off-by: Martin Sustrik <> -
[15:12] cremes zchrish: the applications i built with 0mq use around 9k sockets at their steady state
[15:12] cremes the main thing i learned from building this can be broken out to 3 pieces
[15:13] cremes 1. on linux/osx, make sure your OS buffers are sized correctly (maxsockbuf & friends)
[15:13] cremes 2. 0mq needs to be rebuilt to allow for more than the default of 512 sockets
[15:13] cremes 3. make sure your code cleans up after itself (e.g. closes sockets) when a 0mq resource is no longer needed
[15:20] zchrish Great; thank you.
[15:26] Skaag sustrik: ok for the infinity thing, every device you add, adds its own (albeit small) delay factor, then yes, in theory infinity is possible
[15:26] Skaag we do this right now with video
[15:26] Skaag every media server repeats a single channel from a publishing box
[15:27] Skaag and publishes it to thousands of people
[15:27] sustrik Skaag: great
[15:28] sustrik the scalability thing is the "big deal" with 0mq
[15:28] sustrik we've had different messaging systems up to now
[15:28] sustrik but afaik none of them was designed specifically to be scalable to global level
[15:29] sustrik so i would be interested to hear about any scalability results you may get
[15:31] Skaag well - our next step is quite interesting
[15:31] Skaag the challenge is to decide how to route between the forwarder devices intelligentl
[15:31] Skaag and by intelligently, I mean, taking into account latency, cost of traffic, etc.
[15:31] Skaag and to do all this almost in real-time
[15:32] sustrik yes, it's a bandwidth/latency trade-off
[15:33] sustrik depends on the user requirements i guess
[15:41] francois Does anybody see issues with sending 500,000+ messages using ZMQ_SNDMORE? I have a unit of work that's based on 500,000 and more records. I want either all of it or none of it to proceed.
[15:43] sustrik the message is stored in the memory
[15:43] francois Until SNDMORE is left unset?
[15:43] sustrik so all the 1/2M parts have to fit into memroy
[15:43] sustrik yes
[15:44] francois Thanks for the info
[15:45] sustrik np
[20:08] francois I have a ZMQ_PUSH socket with a ZMQ_HWM set to 1. My code can call zmq_send multiple times, even though no peers are connected. I created the socket using zmq_socket(ctx, ZMQ_PUSH); and zmq_connect(socket, ENDPOINT). What am I doing wrong? Calling getsockopt right before zmq_send correctly reports the HWM set to 1.
[20:09] mikko francois_: what is the behaviour you are seeing?
[20:09] mikko the messages are accepted by the socket?
[20:09] francois mikko:
[20:09] francois mikko: That's the sending side, the one doing zmq_send
[20:10] mikko that's the output of the sending code
[20:10] francois Right, and I want the caller of zmq_send to block
[20:10] mikko what version of zeromq?
[20:10] francois 2.0.10
[20:11] mikko it's been a while since i've ran 2.0.x
[20:11] mikko but let me test with 2.1.x quickly
[20:11] francois Thanks
[20:11] mikko the expected behaviour is to block
[20:11] francois That was my understanding from the documentation
[20:14] mikko francois_: are you setting the HWM before connecting?
[20:14] francois Ah no: I call zmq_socket, zmq_connect, then zmq_setsockopt
[20:14] francois Let me try that
[20:14] mikko thats the most common thing causing this sort of behaviour
[20:14] mikko the HWM affects subsequent connects/binds
[20:14] mikko i think that has been fixed in documentation for the later versions
[20:15] mikko you need zmq_socket, zmq_setsockopt, zmq_connect
[20:15] francois Yay: correctly blocked.
[20:15] francois Thanks again!
[20:16] mikko no problem
[20:18] francois That's why my ZMQ_IDENTITY was left unset then.
[20:28] _jrideout is there a way to identify the originating host that published a message on the subscription side, when using pub/sub?
[20:31] mikko _jrideout: no, not really
[20:31] cremes _jrideout: no, the 0mq lib doesn't expose that; you would need to include that information as part of the message payload
[20:31] mikko _jrideout: what defines originating host?
[20:32] _jrideout ip, or hostname would work
[20:32] _jrideout even an aribtrary identifier declared on the publish side
[20:32] mikko _jrideout: what about if there is a publisher and a device in the middle that does fan-out?
[20:33] cremes _jrideout: again, no; you would need to do this at the application level
[20:33] mikko is the originator the device or the actual publisher?
[20:33] _jrideout my current implentation transfers the hostname with a delimeter as the first set of bytes in the message, but this sends a ton of redundant data
[20:33] _jrideout what would be the distinction between device and publisher?
[20:33] _jrideout application and machine?
[20:33] mikko _jrideout: conceptual
[20:34] _jrideout in my case, they are one and the same
[20:34] mikko but as cremes said, zeromq doesnt expose this information
[20:35] _jrideout is there an internal location where this information could be accessed? I wouldn't mind modifying the library
[20:40] _jrideout could you point me to the .cpp file where messages are put onto the fq/pipe?
[20:43] cremes _jrideout: why modify the application to send this data out when you can do it at the application layer without changing the library?
[20:43] cremes what you are proposing doesn't make any sense to me
[20:51] _jrideout cremes: I'm not sure what distinguishes the application and application layer in your above comment. I have numerous publishing clients connecting to a single subscribing server. The server needs to know which publisher sent each particular message. My current implementation sends a message such as "hostname|messageBody" this allows the server to take appropriate action based on the hostname. It seems like a waste of bytes, though, to send the s
[20:52] cremes why don't you do a pair of pub/sub sockets for each publisher? you'll know where you connected to when you
[20:52] cremes start them, so no need to transmit any data at all
[20:53] cremes pubA connects to subA
[20:53] cremes pubB connects to subB
[20:53] cremes pubC connects to subC
[20:53] cremes when you get a message on subA, you know it came from pubA
[20:53] _jrideout cremes: would I need to use a different port for each pub/sub pair?
[20:54] cremes of course
[20:54] cremes you can't bind multiple sockets to the same port
[20:55] cremes i'm just trying to point out other solutions so you don't make a lot of additional work for yourself
[20:55] cremes by digging into the library's innards
[20:55] _jrideout cremes: of course, thank you, i appreciate it
[20:55] cremes sure
[20:56] cremes another alternative is to publish your "hostname|messageBody" format every X messages where X could be a large number
[20:56] cremes or publish it every X seconds
[20:56] cremes lots of possibilities here...
[20:58] _jrideout cremes: the larger, batched message probably makes the most sense for my application. thanks for the idea -i'll explore that a bit more
[23:14] mikko Steve-o: building
[23:14] mikko Steve-o: i created a wrapper csproj file that sets the properties and includes the zeromq .sln file
[23:15] Steve-o nifty
[23:24] Steve-o mikko: looks like it worked
[23:25] mikko nope
[23:25] mikko the constant is not defined for some reason
[23:25] mikko debugging atm
[23:26] Steve-o its not inherited
[23:27] mikko that was my first stab
[23:27] mikko writing the xml by hand
[23:27] mikko trying to create one using visual studio now
[23:29] Steve-o visual studio is odd at times
[23:29] Steve-o building the openpgm examples using cl.exe on the command line breaks something, but studio works fine
[23:33] mikko because if i edit the csproj file the next checkout will wipe it
[23:33] mikko unless there is some local patch hackery or so