ZeroMq IRC Log

Monday December 20, 2010

[Time] Name	Message
[03:28] davidstrauss	I'm trying to figure out why my first attempt at a C++ echo/client implementation isn't exchanging any messages.
[03:28] davidstrauss	http://pastie.org/1390975
[03:28] davidstrauss	(the code is pastied)
[03:35] Steve-o	isn't rep supposed to bind?
[03:36] davidstrauss	Steve-o, I've tried that
[03:36] davidstrauss	Steve-o, Converting the connect to a bind on the echo.cpp doesn't make it work
[03:37] Steve-o	k
[03:37] davidstrauss	Steve-o, I have nearly identical code in Python working
[03:37] Steve-o	aha, nice
[03:38] davidstrauss	Steve-o, The echo.cpp version starts and then just waits forever for a message
[03:38] davidstrauss	Steve-o, And the client.cpp code "sends" a message without error.
[03:38] davidstrauss	Steve-o, But echo.cpp doesn't seem to get it
[03:39] davidstrauss	Steve-o, If I can get this working, I would happily publish this as a public example
[03:39] Steve-o	local_lat uses REP sockets
[03:39] davidstrauss	Steve-o, local_lat?
[03:39] Steve-o	latency performance test tool
[03:40] Steve-o	>> https://github.com/zeromq/zeromq2/blob/master/perf/local_lat.cpp
[03:40] Steve-o	with remote_lat being the REQ client
[03:41] davidstrauss	Steve-o, Unfortunately, that uses the C API, even though the code is C++
[03:43] Steve-o	the bind on the REQ socket should be something like tcp://eth0:5555
[03:43] davidstrauss	Steve-o, I need to bind on the REQ socket? Not just connect?
[03:44] Steve-o	bind only
[03:44] Steve-o	actually forget the eth0 thing, according to http://api.zeromq.org/zmq_tcp.html using the IP address should work too
[03:44] davidstrauss	Steve-o, All the demos I see have the ZMQ_REP server bind and the ZMQ_REQ connect
[03:44] Steve-o	change line 11 to socket.bind("tcp://127.0.0.1:5000");
[03:45] Steve-o	sorry, getting confused
[03:45] davidstrauss	Steve-o, I can't have the server and client both bind
[03:45] Steve-o	REQ should connect only, REP should bind only, right?
[03:45] davidstrauss	Steve-o, That's the impression i'm under, and it's not working for me.
[03:45] davidstrauss	Steve-o, Nor is it giving any errors.
[03:47] Steve-o	mmm, don't you have to do a bit more work in the echo.cpp loop
[03:47] Steve-o	it looks like you are re-using one message object all the time
[03:47] davidstrauss	Steve-o, How so?
[03:48] davidstrauss	Steve-o, Even reusing the same message object should work at least once
[03:48] Steve-o	yes, I would think so
[03:48] davidstrauss	Steve-o, The code never reaches "cout << "Message: " << (char *) message.data() << endl;"
[03:48] Steve-o	stuck in recv?
[03:49] davidstrauss	Steve-o, yes, blocking on getting a message
[03:49] davidstrauss	Steve-o, I just see "Waiting on message" when I run it
[03:51] Steve-o	but Python version is working?
[03:52] davidstrauss	Steve-o, yes
[03:52] davidstrauss	Steve-o, http://pastie.org/1391002
[03:53] Steve-o	try adding a socket.recv() in the client.cpp
[03:53] Steve-o	I'm guessing the client is terminating too quickly
[03:54] davidstrauss	Steve-o, woo hoo
[03:55] davidstrauss	Steve-o, the works!
[03:55] davidstrauss	that*
[03:55] Steve-o	which version of 0mq is this?
[03:56] davidstrauss	Steve-o, 2.0.10
[03:56] Steve-o	ok 2.1.0 will wait for you in zmq_term
[03:56] Steve-o	2.0 will exit out early
[03:57] davidstrauss	Steve-o, i see
[03:57] Steve-o	ZMQ_LINGER option
[03:57] Steve-o	otherwise you need to sleep or some other hack
[03:57] davidstrauss	Steve-o, honestly, it's kind of silly to use REP/REQ without the client waiting on the reply
[03:58] davidstrauss	Steve-o, I was just trying to get things working
[03:58] Steve-o	it also affects other sockets, like trying to PUB on message and quit
[03:58] Steve-o	which makes command line utilities a bit useless
[04:01] Steve-o	REQ/REP limits you on speed to RTT, you need other socket types for faster performance
[04:04] davidstrauss	Steve-o, to RTT?
[04:04] Steve-o	round-trip-time on the LAN/MAN/WAN
[04:05] Steve-o	0mq excels when you can batch up messages
[04:06] Steve-o	depends on application architecture and requirements
[04:07] Steve-o	whether you wish to allow several outstanding requests for example
[04:18] davidstrauss	Steve-o, Cool. So, I have PHP, Python, and C++ interoperating on the REQ/REP pattern. I'll try to clean up the code and publish it.
[04:19] davidstrauss	Steve-o, thanks for your help
[04:26] Steve-o	np
[04:28] davidstrauss	Is there any good comparison between using Thrift's built-in RPC support vs. using ZeroMQ?
[04:32] Steve-o	I've seen it come up, but nothing detailed apart from discussing it
[04:35] Steve-o	I think there was a protobuf vs thrift as message format above zeromq
[04:35] Steve-o	nothing on the actual RPC implementation
[04:36] Steve-o	lol, thrift over 0mq: http://www.mail-archive.com/thrift-dev@incubator.apache.org/msg08473.html
[04:36] davidstrauss	Steve-o, Well, it is necessary to pick some sort of format for messages.
[04:37] davidstrauss	Steve-o, I'm looking at pure Thrift vs. Thrift serialization + ZeroMQ
[04:37] Steve-o	ok, I like protobufs as it is easy to roll
[04:38] Steve-o	still a bit backward compared with TIBCO quick forms though
[04:38] davidstrauss	Steve-o, Yeah, but I'm working with primarily PHP clients, and Protobufs only have experimental PHP support with no accelerated serialization
[04:39] Steve-o	ok
[04:40] Steve-o	You also have JSON and derivatives to pick from then
[04:40] davidstrauss	Steve-o, JSON isn't great for C++ servers, nor is it a terribly efficient format :-/
[04:41] Steve-o	but it's easy for PHP, JavaScript, and humans to read :D
[04:41] Steve-o	and there are libraries for it everywhere
[04:42] davidstrauss	Steve-o, agreed
[04:42] Steve-o	it's less cumbersome than XDR anyhow
[04:44] Steve-o	you could roll your own PHP extension with fixed scope,
[04:48] Steve-o	Google lacks dynamic forms, everything is static compiled
[08:16] sustrik	hi mikko: what segfault?
[10:22] mikko	sustrik: https://gist.github.com/17fb87b54e83832354b5
[10:24] sustrik	mikko: looks like a deallocation of invalid message
[10:24] sustrik	do you have a test program?
[10:25] mikko	the way i reproduce is by creating a rep socket
[10:25] mikko	then telnet in
[10:25] mikko	press ctrl+c in telnet
[10:25] mikko	and enter twice
[10:25] sustrik	sorry, what's enter twice?
[10:25] mikko
[10:25] mikko
[10:25] sustrik	in telnet?
[10:25] mikko	yes
[10:26] sustrik	let me try...
[10:26] mikko	i've been too lazy to check the exact bytes that combo sends :)
[10:26] sustrik	doesn't matter
[10:27] sustrik	reproduced
[10:27] sustrik	thanks!
[10:30] mikko	np
[10:31] mikko	https://gist.github.com/21ee1c85b1b2f6cbba66
[10:31] mikko	that's from my gdb session last night
[10:47] CIA-21	zeromq2: 03Martin Sustrik 07master * r112d0cd 10/ src/decoder.cpp :
[10:47] CIA-21	zeromq2: Fix the segfault when over-sized message is rejected.
[10:47] CIA-21	zeromq2: The in_progress message inside of decoder was left
[10:47] CIA-21	zeromq2: uninitialised in such a case.
[10:47] CIA-21	zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/grKk2b
[10:53] mikko	sustrik: gj, that was quick
[11:12] sustrik	:)
[12:05] CIA-21	zeromq2: 03Martin Sustrik 07master * r51d2d9b 10/ src/decoder.cpp :
[12:05] CIA-21	zeromq2: Yet one more fix for oversized messages.
[12:05] CIA-21	zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/ez2JYr
[18:41] myraft	mikko - I haven't been back here for a while. Quick update - got past the initial Java HelloWorld queue setup. The problem was the way I was running the Java code on Ubuntu.
[18:42] myraft	Thanks for the help the other day.
[18:44] mikko	no problem
[18:45] mikko	what is the problem of running java code on ubuntu?
[19:01] myraft	mikko - the guide explicitly states to calls to run it with -Djava.library.path="/usr/local/lib". I did sudo ldconfig , which in theory should have loaded it and be avaiable for any program to use (at least that was my impression). Once I set that as VM Argument - everythigng started working.
[19:09] mikko	nice!
[19:10] mikko	so the error message was correct after all
[19:10] mikko	good to know, now we know what to tell the next guy asking
[19:25] dv	woot
[19:25] dv	i can now send asynchronous function calls through zeromq
[19:32] mikko	dv_: cool!
[19:32] mikko	dv_: what language are you working in?
[19:32] dv	its not rpc, mind you
[19:33] dv	more like boost signals/libsigc++/qt signals
[19:33] dv	mikko, right now its for c++ and ruby
[19:33] dv	i plan on adding support for more languages. especially javascript through websockets
[19:33] mikko	im doing something similar in my current project
[19:33] mikko	i got "intercomm" for communicating between threads
[19:33] mikko	sending shutdown messages etc
[19:33] dv	on one side, i do send_call("foobar", params);
[19:33] dv	on the other side, i previously registered a "foobar" callback
[19:34] mikko	nice
[19:34] dv	and it transmits the params etc. through the wire
[19:34] dv	the c++ side uses a lot of metaprogramming fu :)
[19:34] mikko	nice, nice
[19:35] dv	do you send more complex types, such as arrays?
[19:35] mikko	no, my use-case is very simple
[19:35] dv	i want to limit things, that is, primitive types, and maybe binary buffers
[19:35] mikko	just sending ints
[19:35] dv	but more heavyweight transmissions need their dedicated channel anyway in my opinion
[19:36] mikko	im sending enums like HTTPD_SHUTDOWN, SEND_STATS etc
[19:36] mikko	fairly simple
[19:36] dv	yeah, usually this kind of communication makes up the majority of it all
[19:36] mikko	i got a libevent based httpd that pushes data down to 0MQ, which is consumed by classifier nodes and then aggregated from classifiers in aggregator nodes
[19:37] mikko	in=in to
[19:40] dv	btw. in case you do something with epgm,
[19:40] dv	try using your IP address instead of the NIC name
[19:41] dv	sometimes openpgm seems to have problems with the nic's name, this issue was discussed a bit in the mailing list a while ago