ZeroMq IRC Log

Friday January 28, 2011

[Time] Name	Message
[00:54] stodge	Do I need to install openpgm for zeroconf to support the protocol?
[01:06] stodge	Or do I compile zeromq with pgm support
[06:37] stockMQ	Hi I am using pgmping to do some tests
[06:37] stockMQ	Anyone here using it?
[06:44] sustrik	mikko: what size did you set up?
[06:52] CIA-21	zeromq2: 03Martin Sustrik 07master * r1b15eba 10/ src/socket_base.cpp :
[06:52] CIA-21	zeromq2: Fixed the problem of subscription forwarding and PGM interaction
[06:52] CIA-21	zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/eCdQFd
[07:01] stockMQ	??
[07:05] Evet	zeromq client-side javascript binding would be great
[07:06] Evet	s/javascript/html5/
[07:34] sustrik	Evet: afaiu java script doesn't allow for invoking C binaries
[07:35] Evet	sustrik: protocol compatibility with websockets isnt enough?
[07:37] sustrik	well, you need some code
[07:38] sustrik	compatibility is an abstract concept
[07:38] sustrik	no much help
[09:21] boothead	Hi all, is there an easy (or possible) way to get a list of a socket's connections or binds... I'm using pyzmq
[09:24] thoward	hello
[09:24] thoward	anyone here familiar with pyzmq on windows?
[09:25] boothead	thoward - i've built it using VS, but that's about it!
[09:25] mikko	thoward: not very
[09:25] mikko	thoward: but there are many pyzmq+win users here
[09:26] thoward	i've had success building it on windows and so far there have been no problems there..
[09:26] thoward	i'm now trying to deploy it to a box that doesn't have a compiler on it..
[09:26] thoward	i'm having no success with that
[09:39] sustrik	what's the error?
[09:39] sustrik	mikko: how large buffer did you used on bsd?
[09:39] thoward	sustrik: DLL Load failed
[09:40] sustrik	bad paths?
[09:40] thoward	doesn't say which DLL. have tried the normal tricks (updated path to include location of libzmq.dll and copied the dll all over the place
[09:40] sustrik	you can also try to check the dll with depeds.exe
[09:40] sustrik	depends.exe
[09:40] sustrik	it'll show missing dlls in red
[09:41] thoward	you mean dependencies of libzmq.dll?
[09:41] sustrik	yes
[09:41] mikko	sustrik: let me check
[09:41] mikko	sustrik:
[09:41] mikko	kern.ipc.maxsockbuf=4000000
[09:41] mikko	net.inet.tcp.sendbuf_max=16777216
[09:41] mikko	net.inet.tcp.recvbuf_max=16777216
[09:42] mikko	are those right buffers?
[09:42] sustrik	nope
[09:42] sustrik	it's socket pair
[09:42] sustrik	so no TCP is involved
[09:42] sustrik	it's unix domain socket afaict
[09:42] mikko	sustrik: kern.ipc.maxsockbuf ?
[09:42] sustrik	no idea
[09:43] sustrik	maybe you can retrieve the buffer size in mailbox.cpp
[09:43] sustrik	when the socketpair is created
[09:43] sustrik	and print it out
[09:44] sustrik	that way you can check whether the setting had the desirable effect
[10:12] thoward	yay! i got it working.
[10:12] thoward	thanks to:
[10:12] thoward	http://www.lfd.uci.edu/~gohlke/pythonlibs/
[10:25] sustrik	nice
[10:47] sustrik	no issues i am aware of
[10:47] sustrik	what version is that?
[10:48] sustrik	it looks like the I/O threads are still working, receiving messages
[10:48] sustrik	but the application thread doesn't get notified about the traffic
[10:48] sustrik	it looks like device problem imo
[10:49] sustrik	i was just deducing because of the growing memory you referred to
[10:50] sustrik	hm
[10:50] sustrik	if it was completely stuck i would just remain on stable mem usage
[10:51] sustrik	i assume there's no way to reproduce the problem?
[10:52] sustrik	i guess you'll have to attach debugger to the stuck device and find out what's going on then
[10:55] sustrik	yes, first have a look at backtrace
[10:55] sustrik	all the threads in the process
[10:56] sustrik	using gdb?
[10:56] sustrik	info threads
[10:57] sustrik	that will give you list of all threads
[10:57] sustrik	then thread 1
[10:57] sustrik	to switch to thread 1
[10:57] sustrik	thread 2
[10:57] sustrik	to switch to thread 2 etc.
[10:57] sustrik	np
[11:04] mikko	on the topic of gdb this might be interesting as well http://blog.ksplice.com/2011/01/8-gdb-tricks/
[11:08] sustrik	nice, i wasn't aware -ggdb3 thing
[11:35] zchrish	I am building a zeromq application and would like to stress test it. Can anyone point me to a free or low-cost server farm whereby I can connect 1000-10000 clients?
[11:40] zchrish	I have a set of server(s) that I want to test and would like to use this server farm to simulate a "client cloud". I realize it isn't terribly realistic because, in use, my application would be accessed by people on the internet across the world.
[11:40] zchrish	But at least I would like to test the "number" of client connects.
[11:47] zchrish	Has anyone done this before? If so, what types of issues should I expect and, therefore, plan on. For example, I expect my server to hang from time-2-time and will need to kill and restart it but I don't necessarily have a strategy in mind yet.
[11:57] CIA-21	zeromq2: 03Martin Sustrik 07sub-forward * r95356d6 10/ (5 files):
[11:57] CIA-21	zeromq2: Re-creating subscription pipes after disconnect
[11:57] CIA-21	zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/eeeiXr
[12:06] sustrik	zchrish: no idea, sorry
[12:09] zchrish	sustrik: Has anyone built an application that serves 1000-100000 client yet? I will be using a combination of REQ/REP and PUB/SUB to communicate with my central servers and these clients.
[12:10] sustrik	dermoth\|home just mentioned having 3000 connections
[12:13] zchrish	Oh, that's great and encouraging. I am staying close (read exact) to the model patterns described in "The Guide" to prepare my server software. I saw a company named "Sabalcore" that rents computer time.
[12:14] zchrish	Anyway, it is somewhat of an unknown to me now what types of issues will pop-up since the zeromq software is designed to shield this level of detail from my view which is ok.
[12:54] mikko	i think Skaag has a fairly large test environment
[12:54] mikko	and plans to use it on large production environment
[12:55] mikko	if i remember correctly
[12:55] Skaag	it's already being used right now
[12:55] Skaag	our test cluster has 16 ~ 20 nodes
[12:55] mikko	wow, that happened quick
[12:55] Skaag	not really, 2 weeks ;0
[12:55] Skaag	:-)
[12:56] mikko	i'm used to working with enterprises
[12:56] mikko	where two weeks is spent on planning the planning phase
[12:56] Skaag	right, you're 100% correct ;)
[12:57] Skaag	we calculated bottlenecks, planned our message sizes and structure carefully, and just went ahead with it
[12:57] zchrish	I am using a master that will fan-out to a configurable number of intermdiaries. Is there a rule of thumb to know when to add another node? I suppose I can either (1) theoretically determine the load or (2) watch the activity and add another node based on heuristics.
[12:57] Skaag	and I have to say it's looking very good now
[12:58] Skaag	zchrish: the way we plan to do it, is based on actual load + some other factors (strategic)
[12:58] zchrish	I see; thank you.
[13:00] mikko	http://twitter.com/taotetek/statuses/30949638219505664
[13:00] mikko	cool
[13:06] Skaag	yes :)
[13:07] Skaag	my next task, is to replicate the pubnub.com functionality on our own system
[13:07] Skaag	and then make some really cool and interactive web based widgets with that
[13:07] Skaag	that involve live video streaming, etc.
[13:08] Skaag	and the next big step is synchronizing flash objects between many media servers, and that will really be game changing for the media industry.
[13:08] Skaag	hopefully, all this can also happen in the next 2 ~ 3 weeks.
[13:11] sustrik	changing the media industry in 2-3 weeks, wow! :)
[13:12] sustrik	anyway, the idea is to build large-scale distribution trees for media delivery?
[13:12] mikko	i think UDT would of interest for the media industry
[13:13] sustrik	my impression was that UDT makes sense when you are trying to transfer you 10TB database over the WAN
[13:13] sustrik	no idea about UDT & media
[13:13] sustrik	UDT & real time to be more precise
[13:14] Skaag	video distribution, we do already, with Wowza
[13:14] Skaag	WowzaMediaServer is a replacement for Adobe's FMS
[13:14] Skaag	but imagine a developer wants to create a large scale application right now - he can't really...
[13:15] Skaag	he's limited by what a single FMS can provide
[13:15] sustrik	what's FMS?
[13:15] Skaag	when you talk to Adobe about FMS, they talk to you in terms of hundreds of channels
[13:15] Skaag	FMS = Flash Media Server
[13:15] Skaag	with all its cool object synchronization, etc.
[13:15] sustrik	i see
[13:15] Skaag	So I want to attack that + also do this for html5 with json object sync
[13:16] Skaag	you send the server some update to an object, all other subscribers receive it at once
[13:16] Skaag	like pubnub.com really
[13:16] Skaag	I wonder what they use in their backends in place of zmq
[13:18] sustrik	aha
[13:18] sustrik	so basically large-scale delivery of PUB/SUB streams
[13:19] sustrik	have you seen my article about scaling 0mq-like system to internet scale?
[13:19] sustrik	http://www.250bpm.com/hits
[13:21] sustrik	one of the points is that with such a system any participant can add new devices to the distribution tree
[13:21] sustrik	thus making it infinitely scalable
[13:22] sustrik	this applies not only to the provider of the service but also to end-user, ISPs etc. virtually everyone
[14:03] Skaag	I need to read it because i'm not sure about the infinity thing :)
[14:23] CIA-21	zeromq2: 03Martin Sustrik 07sub-forward * rf267287 10/ (src/Makefile.am src/matcher.cpp src/matcher.hpp):
[14:23] CIA-21	zeromq2: Simple ad hoc message matcher added
[14:23] CIA-21	zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/dSNqlA
[15:12] cremes	zchrish: the applications i built with 0mq use around 9k sockets at their steady state
[15:12] cremes	the main thing i learned from building this can be broken out to 3 pieces
[15:13] cremes	1. on linux/osx, make sure your OS buffers are sized correctly (maxsockbuf & friends)
[15:13] cremes	2. 0mq needs to be rebuilt to allow for more than the default of 512 sockets
[15:13] cremes	3. make sure your code cleans up after itself (e.g. closes sockets) when a 0mq resource is no longer needed
[15:20] zchrish	Great; thank you.
[15:26] Skaag	sustrik: ok for the infinity thing, every device you add, adds its own (albeit small) delay factor, then yes, in theory infinity is possible
[15:26] Skaag	we do this right now with video
[15:26] Skaag	every media server repeats a single channel from a publishing box
[15:27] Skaag	and publishes it to thousands of people
[15:27] sustrik	Skaag: great
[15:28] sustrik	the scalability thing is the "big deal" with 0mq
[15:28] sustrik	we've had different messaging systems up to now
[15:28] sustrik	but afaik none of them was designed specifically to be scalable to global level
[15:29] sustrik	so i would be interested to hear about any scalability results you may get
[15:31] Skaag	well - our next step is quite interesting
[15:31] Skaag	the challenge is to decide how to route between the forwarder devices intelligentl
[15:31] Skaag	and by intelligently, I mean, taking into account latency, cost of traffic, etc.
[15:31] Skaag	and to do all this almost in real-time
[15:32] sustrik	yes, it's a bandwidth/latency trade-off
[15:33] sustrik	depends on the user requirements i guess
[15:41] francois	Does anybody see issues with sending 500,000+ messages using ZMQ_SNDMORE? I have a unit of work that's based on 500,000 and more records. I want either all of it or none of it to proceed.
[15:43] sustrik	the message is stored in the memory
[15:43] francois	Until SNDMORE is left unset?
[15:43] sustrik	so all the 1/2M parts have to fit into memroy
[15:43] sustrik	yes
[15:44] francois	Thanks for the info
[15:45] sustrik	np
[20:08] francois	I have a ZMQ_PUSH socket with a ZMQ_HWM set to 1. My code can call zmq_send multiple times, even though no peers are connected. I created the socket using zmq_socket(ctx, ZMQ_PUSH); and zmq_connect(socket, ENDPOINT). What am I doing wrong? Calling getsockopt right before zmq_send correctly reports the HWM set to 1.
[20:09] mikko	francois_: what is the behaviour you are seeing?
[20:09] mikko	the messages are accepted by the socket?
[20:09] francois	mikko: https://gist.github.com/800852
[20:09] francois	mikko: That's the sending side, the one doing zmq_send
[20:10] mikko	that's the output of the sending code
[20:10] francois	Right, and I want the caller of zmq_send to block
[20:10] mikko	what version of zeromq?
[20:10] francois	2.0.10
[20:11] mikko	it's been a while since i've ran 2.0.x
[20:11] mikko	but let me test with 2.1.x quickly
[20:11] francois	Thanks
[20:11] mikko	the expected behaviour is to block
[20:11] francois	That was my understanding from the documentation
[20:14] mikko	francois_: are you setting the HWM before connecting?
[20:14] francois	Ah no: I call zmq_socket, zmq_connect, then zmq_setsockopt
[20:14] francois	Let me try that
[20:14] mikko	thats the most common thing causing this sort of behaviour
[20:14] mikko	the HWM affects subsequent connects/binds
[20:14] mikko	i think that has been fixed in documentation for the later versions
[20:15] mikko	you need zmq_socket, zmq_setsockopt, zmq_connect
[20:15] francois	Yay: correctly blocked.
[20:15] francois	Thanks again!
[20:16] mikko	no problem
[20:18] francois	That's why my ZMQ_IDENTITY was left unset then.
[20:28] _jrideout	is there a way to identify the originating host that published a message on the subscription side, when using pub/sub?
[20:31] mikko	_jrideout: no, not really
[20:31] cremes	_jrideout: no, the 0mq lib doesn't expose that; you would need to include that information as part of the message payload
[20:31] mikko	_jrideout: what defines originating host?
[20:32] _jrideout	ip, or hostname would work
[20:32] _jrideout	even an aribtrary identifier declared on the publish side
[20:32] mikko	_jrideout: what about if there is a publisher and a device in the middle that does fan-out?
[20:33] cremes	_jrideout: again, no; you would need to do this at the application level
[20:33] mikko	is the originator the device or the actual publisher?
[20:33] _jrideout	my current implentation transfers the hostname with a delimeter as the first set of bytes in the message, but this sends a ton of redundant data
[20:33] _jrideout	what would be the distinction between device and publisher?
[20:33] _jrideout	application and machine?
[20:33] mikko	_jrideout: conceptual
[20:34] _jrideout	in my case, they are one and the same
[20:34] mikko	but as cremes said, zeromq doesnt expose this information
[20:35] _jrideout	is there an internal location where this information could be accessed? I wouldn't mind modifying the library
[20:40] _jrideout	could you point me to the .cpp file where messages are put onto the fq/pipe?
[20:43] cremes	_jrideout: why modify the application to send this data out when you can do it at the application layer without changing the library?
[20:43] cremes	what you are proposing doesn't make any sense to me
[20:51] _jrideout	cremes: I'm not sure what distinguishes the application and application layer in your above comment. I have numerous publishing clients connecting to a single subscribing server. The server needs to know which publisher sent each particular message. My current implementation sends a message such as "hostname\|messageBody" this allows the server to take appropriate action based on the hostname. It seems like a waste of bytes, though, to send the s
[20:52] cremes	why don't you do a pair of pub/sub sockets for each publisher? you'll know where you connected to when you
[20:52] cremes	start them, so no need to transmit any data at all
[20:53] cremes	pubA connects to subA
[20:53] cremes	pubB connects to subB
[20:53] cremes	pubC connects to subC
[20:53] cremes	when you get a message on subA, you know it came from pubA
[20:53] _jrideout	cremes: would I need to use a different port for each pub/sub pair?
[20:54] cremes	of course
[20:54] cremes	you can't bind multiple sockets to the same port
[20:55] cremes	i'm just trying to point out other solutions so you don't make a lot of additional work for yourself
[20:55] cremes	by digging into the library's innards
[20:55] _jrideout	cremes: of course, thank you, i appreciate it
[20:55] cremes	sure
[20:56] cremes	another alternative is to publish your "hostname\|messageBody" format every X messages where X could be a large number
[20:56] cremes	or publish it every X seconds
[20:56] cremes	lots of possibilities here...
[20:58] _jrideout	cremes: the larger, batched message probably makes the most sense for my application. thanks for the idea -i'll explore that a bit more
[23:14] mikko	Steve-o: building
[23:14] mikko	Steve-o: i created a wrapper csproj file that sets the properties and includes the zeromq .sln file
[23:15] Steve-o	nifty
[23:24] Steve-o	mikko: looks like it worked
[23:25] mikko	nope
[23:25] mikko	the constant is not defined for some reason
[23:25] mikko	debugging atm
[23:26] Steve-o	its not inherited
[23:27] mikko	that was my first stab
[23:27] mikko	writing the xml by hand
[23:27] mikko	trying to create one using visual studio now
[23:29] Steve-o	visual studio is odd at times
[23:29] Steve-o	building the openpgm examples using cl.exe on the command line breaks something, but studio works fine
[23:33] mikko	because if i edit the csproj file the next checkout will wipe it
[23:33] mikko	unless there is some local patch hackery or so