ZeroMq IRC Log

Monday October 3, 2011

[Time] Name	Message
[04:41] bitcycle	Hey all. I'm having trouble with push/pull dropping messages when children processes are pushing messages back to a single parent process. Can someone advise me on how to get it to not drop them without purposefully slowing down the children?
[05:27] sharp_wan	hi, does anyone know how to compile the hwserver and hwclient in C?
[05:27] sharp_wan	i'm using feroda 15
[05:29] sharp_wan	i've tried the build all command in my terminal but it say bash:build:command not found
[05:39] sharp_wan	i need a guide on how to complie the example i download from github
[05:39] sharp_wan	compile*
[05:39] sharp_wan	i tried build it but not owrking
[05:49] sustrik	gcc -pthread -luuid -o name name.c libzmq.a
[05:50] sustrik	or similar
[06:18] sustrik	bitcycle: what socket type are you using?
[06:19] sustrik	the pub/sub on top of TCP sounds like a good fit
[06:20] bitcycle	So, at this point, I'm using a PUSH (child processes) => PULL (parent process) and a REQ (child requesting to exit) => REP (parent giving permission to exit). One moment while I paste the code.
[06:21] bitcycle	http://paste.pound-python.org/show/13314/
[06:24] sustrik	why not simply pub/sub?
[06:24] sustrik	ah, right
[06:24] sustrik	termination
[06:24] sustrik	yes, req/rep can be used for that
[06:24] sustrik	what's the problem then?
[06:28] bitcycle	sustrik: Just debugging through it, at this point. No worries. If I get truly stumped I'll msg.
[06:28] sustrik	ok
[06:37] sharp_wan	how to install zeromq in linux/fedora?
[06:37] sharp_wan	i use yum install zeromq
[06:37] sustrik	dunno
[06:37] sharp_wan	but when i try to compile my program, it say error cannot find the zmq.h file
[06:37] sustrik	but you can download it from zeromq.org
[06:38] sustrik	your paths are probably not pointing to the right dir
[06:38] sharp_wan	the instruction is not clear
[06:38] sharp_wan	yea
[06:38] sharp_wan	thats what i'm thinking
[06:38] sharp_wan	how to resolve that?
[06:43] bitcycle	sustrik: I've pasted the output and the test script that I've got right now... basically the parent process isn't getting the exit request that the child makes. Any suggestions on that one?
[06:45] sustrik	that's a child or a parent?
[06:46] bitcycle	The output has the name of the logger: child or parent.
[06:48] bitcycle	sustrik: parent: Reading from poller => child: Sending exit request => parent: still reading from the poller
[06:49] sustrik	parent requests children to exit?
[06:49] sustrik	exit_req = context.socket(zmq.REQ)
[06:49] bitcycle	oh. I'm a dumbass.
[06:50] bitcycle	Well, maybe not. I miss-named the variable, but I'm using the right zmq socket type.
[06:52] bitcycle	sustrik: I've updated the paste with the corrected variable names. http://paste.pound-python.org/show/13317/
[06:53] sustrik	uou mean the parent sends requests to children to exit?
[06:53] sustrik	exit_req = context.socket(zmq.REQ)
[06:55] bitcycle	sustrik: Well, the way that I've originally coded it is to have the child wait to exit until the parent gives the OK. This is the guard against the push socket being closed when the child exits too quickly.
[06:55] sustrik	or is it the client code?
[06:55] sustrik	the REQ should be on the client side
[06:55] sustrik	the REP should be on the server side
[06:56] bitcycle	sustrik: the script I pasted includes both the client and server. The clients are child processes, and the parent is the main script process.
[06:57] bitcycle	Try running it, and you'll see that: "python test.py"
[06:57] sustrik	do you terminate the library correctly
[06:58] sustrik	if not, pending outbound messages may be dropped when the process exits
[06:58] bitcycle	sustrik: The socket & context are closed & terminated (respectively) upon process exit.
[06:58] bitcycle	Its part of the python zmq bindings.
[06:58] sustrik	ok
[06:58] sustrik	then it's a bug
[06:59] sustrik	please, strip down the code to minimal test case
[06:59] sustrik	and create a ticket in the bug tracker
[07:00] bitcycle	sustrik: Do you mean its a bug that the poller doesn't get the response?
[07:00] bitcycle	sustrik: or rather that the poller doesn't get the request.
[07:00] sustrik	it's a bug that message send to REQ doesn't arrive at connected REP when the sending peer is closed in correct manner
[07:01] bitcycle	Yes, I agree with you there. I'll try a few more things in the morning before I do, though. Are there major problems with REQ/REP sockets, in zmq v2.1?
[07:01] sustrik	shouldn't be
[07:01] sustrik	it's out there for a long time and pretty stable
[07:06] sustrik	mikko: i think i got it!
[07:13] bitcycle	sustrik: Do you think it'd be alright for me to attach the test script to a request for help on the mailing list?
[07:14] sustrik	sure
[07:14] sustrik	but if it's a bug, the minimal test case is crucial
[07:18] bitcycle	For sure. That's where the test script will come in handy.
[07:18] bitcycle	Anyways, I'm heading to bed. G'night.
[07:33] kfuglsang	Hi all. I'm about to start a study project in which I will be using a device running the .NET Micro Framework 4.1. Would anyone know if the ZeroMQ .NET bindings can be used on this platform?
[07:34] kfuglsang	I haven't been able to find any information about what requirements ZeroMQ and the .NET bindings have to the platform on which it will run
[07:36] sustrik	.net bindings are just a thing wrapper on top on native zmq library
[07:36] sustrik	so first try compiling zeromq as such for your platform
[07:37] kfuglsang	yhym, okay. Thanks. I will try that when I receive the device.
[07:44] mikko	sustrik: cool!
[07:45] mikko	sustrik: what is the problem?
[08:10] sustrik	mikko: it's shutdown mechanism on the sender side
[08:10] sustrik	there's a message queue
[08:10] sustrik	and a buffer to batch messages into before sending
[08:11] sustrik	when shutting down the system waits for the queue to be empty
[08:11] sustrik	but it doesn't check whether the batch buffer was entirely flushed to the network
[09:16] mikko	sustrik: cool
[09:16] mikko	sustrik: is this just for router type?
[09:16] mikko	ermm, dealer
[09:16] mikko	or all sockets
[09:17] sustrik	all sockets
[09:17] sustrik	it's generic
[09:29] mikko	sustrik: this is good progress
[09:30] mikko	well done for being able to strip down the test case
[09:31] sustrik	i'm looking into how to solve it
[09:31] sustrik	some kind of handshake between session and engine during the termination phase
[09:58] tuffaha	Hello, I was looking at the python binding and it seems they only support zmq 2.1.x can I use them with 2.2 or do I have to use C?
[09:58] mikko	tuffaha: why do you want zeromq 2.2?
[09:59] tuffaha	mikko, isn't it the latest stable version?
[09:59] tuffaha	or should I be using 2.1?
[09:59] mikko	2.1 is the latest stable version
[09:59] mikko	http://www.zeromq.org/intro:get-the-software
[10:00] tuffaha	mikko: thanks :)
[10:06] Odipides	I'm getting a bit of a glitch under windows (XP) with multiple servers on the same end point. Can't find anything in the guide or the API about this being a problem, but when the second server is run the bind returns -1, and errno is 100 (which is undefined) in errno.h)
[10:07] mikko	Odipides: are you trying to bind to same port multiple times?
[10:07] Odipides	Yup
[10:07] mikko	that's not supported
[10:08] Odipides	The samples in the guide seem to do that so I figured it was allowed
[10:08] mikko	Odipides: which samples?
[10:09] Odipides	If you look at figure 19 in the guide, the code sample below shows a req/rep server
[10:10] Odipides	the diagram shows multiple instances of the server but I didn't notice any machine boundaries
[10:12] mikko	Odipides: the REQs and REPs are each different services
[10:13] mikko	you can only bind once to a port on same interface
[10:13] mikko	just like with normal sockets
[10:13] mikko	did you check the returned errno with zmq_strerror ?
[10:13] Odipides	Understood. I just thought it might be possible to run all the 'REP' services on the same node
[10:13] mikko	Odipides: you can
[10:14] mikko	Odipides: but they have to use different ports
[10:14] Odipides	No, didn't get the strerror yet, I figured it might be because of using the single port
[10:14] Odipides	No prob, though. The sample code confused me because I thought I had missed out some config or setsockopt step for sharing the port
[10:23] Odipides	By the way, is the pgm endpoint supported on Windows platforms? I'm writing a portability layer and I'm in the process of generating a matrix of endpoint types * platform * socket types. My test harness returns an error on the pgm type so it's currently going down as 'Not on Win platforms'. I seem to recall reading that it isn't supported but I may have dreamt it
[10:49] mikko	Odipides: yes it is
[10:50] mikko	Odipides: you need to build zeromq with openpgm support
[12:27] djc	I'm hitting Assertion failed: cmd.type == command_t::done (ctx.cpp:148)
[12:27] djc	any hint on what might be wrong here?
[12:28] djc	it happens in a fairly run-of-the-mill script
[12:42] sustrik	djc_: aren't you using single socket from multiple threads?
[13:41] mikko	sustrik: can you attach the test cases to LIBZMQ-264 ?
[14:03] sustrik	mikko: done
[14:06] mikko	sustrik: thanks
[14:24] djc	sustrik: we're only using one thread, from pyzmq
[14:27] sustrik	djc: then using a handle after it have been closed may be a reason
[14:27] sustrik	or memory overwrite
[14:43] djc	sustrik: both of those seem unlikely
[14:44] djc	and the latter would be a pyzmq bug, I guess?
[14:46] sustrik	ack
[14:46] sustrik	anyway, create minimal test case and report the bug
[14:53] djc	hmm, we don't have a consistent repro scenario yet
[14:53] djc	but we'll try
[18:46] Steve-o	so the rate limiter disappeared from 3.0, what gives?
[18:50] mikko	have you checked history?
[18:54] Steve-o	looks like a mistake
[18:55] Steve-o	found collateral damage in 2.1.2 and I sent a patch to fix in March, but didn't follow through on 3.
[18:56] Steve-o	it's because PGM_TXW_MAX_RTE is used to size the transmit window * and * the rate limiter
[18:56] Steve-o	the commit that broke it was to change the definition of the transmit window size
[18:58] Steve-o	mmm, github is a bit limited, I cannot appear to navigate through a files history
[19:05] Steve-o	ok, I'll add patches for 3 & 4 ...
[19:15] Steve-o	mikko: I built OpenPGM 5.1.118 with OSX, I have a hunch from one client that Snow Leopard doesn't support Autotools' silent rules
[19:15] Steve-o	need to add patches to update every branch
[19:26] mikko	Steve-o: i remember that issue from the past
[19:27] mikko	Steve-o: silent rules seem to work fine for me
[19:27] mikko	im on 10.6.8
[19:28] mikko	autoconf (GNU Autoconf) 2.68
[19:28] Steve-o	one client complained about extra output, so I'm not sure
[19:29] Steve-o	I need to start testing 5.2 for Lion support though
[19:30] Steve-o	I've ported two unit test modules to Google Mock & Google Test and seems to work well, suitable for automation
[19:31] Steve-o	need to try building on Windows though, which is the real benefit compared to the previous libcheck
[19:32] Steve-o	I think I'm creating more problems with C99 and C++2003 incompatibilities though
[19:32] Steve-o	I only just got real, fake, and mock objects working for C++ wrapping C
[19:33] Steve-o	namespace hell
[19:33] Steve-o	:(
[19:39] Steve-o	I think I have to force my C89 patches to also work with C++2003 as I rebuild everything with C++ in order to fit into a Pgm::external namespace
[20:02] mikko	Steve-o: are you rebuilding openpgm with C++ ?
[20:03] Steve-o	for testing
[20:03] Steve-o	because Googletest provides an automation interface for C++ projects
[20:06] Steve-o	and some pretty neat mock concepts
[20:06] Steve-o	similar to jquery
[20:07] lsousa	hello there, I'm new to this zmq stuff. I'm loving it!
[20:07] Steve-o	eventually I'll look at adding GoogleTest & GoogleMock to zmq, it definitely needs something inline,
[20:08] lsousa	but I couldn't solve something I quess is trivial, even after doing the "Missing Message Problem Solver" tutorial
[20:09] lsousa	my question is quite simple, I guess. I'm writting a program using the pub/sub stuff
[20:09] lsousa	and I'd like to know what is the best way for making a client answer the publisher server
[20:10] lsousa	when the server sends a message to one or more clients, everything works fine, but the contrary is not working
[20:10] lsousa	to receive the messages on the server side, I've created another socket and connected to the same address I've binded the server
[20:10] lsousa	is this correct?
[20:12] michelp	lsousa, yep
[20:12] michelp	PUB/SUB is one way
[20:13] michelp	if you want the client to talk to the server, you have to have another socket going the other way
[20:13] lsousa	michelp: thank you for your answer :)
[20:14] lsousa	michelp: could you please suggest me what is the best protocol to use it? once each part of my program is running on a different process
[20:15] michelp	lsousa, well, if you want the subscriber to "answer" what the publisher published to it, then REQ/REP would work well
[20:15] michelp	the publisher would need to be polling or blocking on those sockets to get the answers, that design is up to you
[20:16] michelp	i guess the subscriber could also PUSH to the subscriber that is PULL
[20:16] michelp	lsousa, you definitely want to read the entire guide, take it slow and actually try out the examples if you can, then the right patterns will jump out at you
[20:18] michelp	but it sounds like waht you want is two socket pairs, on the publisher use 'PUB' and 'PULL' and on the subscriber, use 'SUB' and 'PUSH'
[20:19] michelp	when the subscriber gets data on the SUB, do the work, then PUSH the answer
[20:19] michelp	then publisher PUBs data to the workers, and periodically PULLs the responses
[20:19] michelp	that should at least get you started
[20:20] lsousa	humm, this way seems to fit very well on my app
[20:20] michelp	the 'ventilator' examples in the guide use these socket types, so definitely read that part of the guide again
[20:20] lsousa	michelp: hehe, yep, I'm looking at it right now
[20:21] lsousa	thank you michelp! I've followed your suggestion and, at least in my test scripts, things worked like a charm
[20:22] lsousa	I'll now integrate it to my main program! :)
[20:22] michelp	awesome, good luck
[20:22] lsousa	thank you!
[21:07] mikko	sustrik: alive?

ZeroMq Home

Monday October 3, 2011