Monday October 3, 2011

[Time] NameMessage
[04:41] bitcycle Hey all. I'm having trouble with push/pull dropping messages when children processes are pushing messages back to a single parent process. Can someone advise me on how to get it to not drop them without purposefully slowing down the children?
[05:27] sharp_wan hi, does anyone know how to compile the hwserver and hwclient in C?
[05:27] sharp_wan i'm using feroda 15
[05:29] sharp_wan i've tried the build all command in my terminal but it say bash:build:command not found
[05:39] sharp_wan i need a guide on how to complie the example i download from github
[05:39] sharp_wan compile*
[05:39] sharp_wan i tried build it but not owrking
[05:49] sustrik gcc -pthread -luuid -o name name.c libzmq.a
[05:50] sustrik or similar
[06:18] sustrik bitcycle: what socket type are you using?
[06:19] sustrik the pub/sub on top of TCP sounds like a good fit
[06:20] bitcycle So, at this point, I'm using a PUSH (child processes) => PULL (parent process) and a REQ (child requesting to exit) => REP (parent giving permission to exit). One moment while I paste the code.
[06:21] bitcycle
[06:24] sustrik why not simply pub/sub?
[06:24] sustrik ah, right
[06:24] sustrik termination
[06:24] sustrik yes, req/rep can be used for that
[06:24] sustrik what's the problem then?
[06:28] bitcycle sustrik: Just debugging through it, at this point. No worries. If I get truly stumped I'll msg.
[06:28] sustrik ok
[06:37] sharp_wan how to install zeromq in linux/fedora?
[06:37] sharp_wan i use yum install zeromq
[06:37] sustrik dunno
[06:37] sharp_wan but when i try to compile my program, it say error cannot find the zmq.h file
[06:37] sustrik but you can download it from
[06:38] sustrik your paths are probably not pointing to the right dir
[06:38] sharp_wan the instruction is not clear
[06:38] sharp_wan yea
[06:38] sharp_wan thats what i'm thinking
[06:38] sharp_wan how to resolve that?
[06:43] bitcycle sustrik: I've pasted the output and the test script that I've got right now... basically the parent process isn't getting the exit request that the child makes. Any suggestions on that one?
[06:45] sustrik that's a child or a parent?
[06:46] bitcycle The output has the name of the logger: child or parent.
[06:48] bitcycle sustrik: parent: Reading from poller => child: Sending exit request => parent: still reading from the poller
[06:49] sustrik parent requests children to exit?
[06:49] sustrik exit_req = context.socket(zmq.REQ)
[06:49] bitcycle oh. I'm a dumbass.
[06:50] bitcycle Well, maybe not. I miss-named the variable, but I'm using the right zmq socket type.
[06:52] bitcycle sustrik: I've updated the paste with the corrected variable names.
[06:53] sustrik uou mean the parent sends requests to children to exit?
[06:53] sustrik exit_req = context.socket(zmq.REQ)
[06:55] bitcycle sustrik: Well, the way that I've originally coded it is to have the child wait to exit until the parent gives the OK. This is the guard against the push socket being closed when the child exits too quickly.
[06:55] sustrik or is it the client code?
[06:55] sustrik the REQ should be on the client side
[06:55] sustrik the REP should be on the server side
[06:56] bitcycle sustrik: the script I pasted includes both the client and server. The clients are child processes, and the parent is the main script process.
[06:57] bitcycle Try running it, and you'll see that: "python"
[06:57] sustrik do you terminate the library correctly
[06:58] sustrik if not, pending outbound messages may be dropped when the process exits
[06:58] bitcycle sustrik: The socket & context are closed & terminated (respectively) upon process exit.
[06:58] bitcycle Its part of the python zmq bindings.
[06:58] sustrik ok
[06:58] sustrik then it's a bug
[06:59] sustrik please, strip down the code to minimal test case
[06:59] sustrik and create a ticket in the bug tracker
[07:00] bitcycle sustrik: Do you mean its a bug that the poller doesn't get the response?
[07:00] bitcycle sustrik: or rather that the poller doesn't get the request.
[07:00] sustrik it's a bug that message send to REQ doesn't arrive at connected REP when the sending peer is closed in correct manner
[07:01] bitcycle Yes, I agree with you there. I'll try a few more things in the morning before I do, though. Are there major problems with REQ/REP sockets, in zmq v2.1?
[07:01] sustrik shouldn't be
[07:01] sustrik it's out there for a long time and pretty stable
[07:06] sustrik mikko: i think i got it!
[07:13] bitcycle sustrik: Do you think it'd be alright for me to attach the test script to a request for help on the mailing list?
[07:14] sustrik sure
[07:14] sustrik but if it's a bug, the minimal test case is crucial
[07:18] bitcycle For sure. That's where the test script will come in handy.
[07:18] bitcycle Anyways, I'm heading to bed. G'night.
[07:33] kfuglsang Hi all. I'm about to start a study project in which I will be using a device running the .NET Micro Framework 4.1. Would anyone know if the ZeroMQ .NET bindings can be used on this platform?
[07:34] kfuglsang I haven't been able to find any information about what requirements ZeroMQ and the .NET bindings have to the platform on which it will run
[07:36] sustrik .net bindings are just a thing wrapper on top on native zmq library
[07:36] sustrik so first try compiling zeromq as such for your platform
[07:37] kfuglsang yhym, okay. Thanks. I will try that when I receive the device.
[07:44] mikko sustrik: cool!
[07:45] mikko sustrik: what is the problem?
[08:10] sustrik mikko: it's shutdown mechanism on the sender side
[08:10] sustrik there's a message queue
[08:10] sustrik and a buffer to batch messages into before sending
[08:11] sustrik when shutting down the system waits for the queue to be empty
[08:11] sustrik but it doesn't check whether the batch buffer was entirely flushed to the network
[09:16] mikko sustrik: cool
[09:16] mikko sustrik: is this just for router type?
[09:16] mikko ermm, dealer
[09:16] mikko or all sockets
[09:17] sustrik all sockets
[09:17] sustrik it's generic
[09:29] mikko sustrik: this is good progress
[09:30] mikko well done for being able to strip down the test case
[09:31] sustrik i'm looking into how to solve it
[09:31] sustrik some kind of handshake between session and engine during the termination phase
[09:58] tuffaha Hello, I was looking at the python binding and it seems they only support zmq 2.1.x can I use them with 2.2 or do I have to use C?
[09:58] mikko tuffaha: why do you want zeromq 2.2?
[09:59] tuffaha mikko, isn't it the latest stable version?
[09:59] tuffaha or should I be using 2.1?
[09:59] mikko 2.1 is the latest stable version
[09:59] mikko
[10:00] tuffaha mikko: thanks :)
[10:06] Odipides I'm getting a bit of a glitch under windows (XP) with multiple servers on the same end point. Can't find anything in the guide or the API about this being a problem, but when the second server is run the bind returns -1, and errno is 100 (which is undefined) in errno.h)
[10:07] mikko Odipides: are you trying to bind to same port multiple times?
[10:07] Odipides Yup
[10:07] mikko that's not supported
[10:08] Odipides The samples in the guide seem to do that so I figured it was allowed
[10:08] mikko Odipides: which samples?
[10:09] Odipides If you look at figure 19 in the guide, the code sample below shows a req/rep server
[10:10] Odipides the diagram shows multiple instances of the server but I didn't notice any machine boundaries
[10:12] mikko Odipides: the REQs and REPs are each different services
[10:13] mikko you can only bind once to a port on same interface
[10:13] mikko just like with normal sockets
[10:13] mikko did you check the returned errno with zmq_strerror ?
[10:13] Odipides Understood. I just thought it might be possible to run all the 'REP' services on the same node
[10:13] mikko Odipides: you can
[10:14] mikko Odipides: but they have to use different ports
[10:14] Odipides No, didn't get the strerror yet, I figured it might be because of using the single port
[10:14] Odipides No prob, though. The sample code confused me because I thought I had missed out some config or setsockopt step for sharing the port
[10:23] Odipides By the way, is the pgm endpoint supported on Windows platforms? I'm writing a portability layer and I'm in the process of generating a matrix of endpoint types * platform * socket types. My test harness returns an error on the pgm type so it's currently going down as 'Not on Win platforms'. I seem to recall reading that it isn't supported but I may have dreamt it
[10:49] mikko Odipides: yes it is
[10:50] mikko Odipides: you need to build zeromq with openpgm support
[12:27] djc I'm hitting Assertion failed: cmd.type == command_t::done (ctx.cpp:148)
[12:27] djc any hint on what might be wrong here?
[12:28] djc it happens in a fairly run-of-the-mill script
[12:42] sustrik djc_: aren't you using single socket from multiple threads?
[13:41] mikko sustrik: can you attach the test cases to LIBZMQ-264 ?
[14:03] sustrik mikko: done
[14:06] mikko sustrik: thanks
[14:24] djc sustrik: we're only using one thread, from pyzmq
[14:27] sustrik djc: then using a handle after it have been closed may be a reason
[14:27] sustrik or memory overwrite
[14:43] djc sustrik: both of those seem unlikely
[14:44] djc and the latter would be a pyzmq bug, I guess?
[14:46] sustrik ack
[14:46] sustrik anyway, create minimal test case and report the bug
[14:53] djc hmm, we don't have a consistent repro scenario yet
[14:53] djc but we'll try
[18:46] Steve-o so the rate limiter disappeared from 3.0, what gives?
[18:50] mikko have you checked history?
[18:54] Steve-o looks like a mistake
[18:55] Steve-o found collateral damage in 2.1.2 and I sent a patch to fix in March, but didn't follow through on 3.
[18:56] Steve-o it's because PGM_TXW_MAX_RTE is used to size the transmit window * and * the rate limiter
[18:56] Steve-o the commit that broke it was to change the definition of the transmit window size
[18:58] Steve-o mmm, github is a bit limited, I cannot appear to navigate through a files history
[19:05] Steve-o ok, I'll add patches for 3 & 4 ...
[19:15] Steve-o mikko: I built OpenPGM 5.1.118 with OSX, I have a hunch from one client that Snow Leopard doesn't support Autotools' silent rules
[19:15] Steve-o need to add patches to update every branch
[19:26] mikko Steve-o: i remember that issue from the past
[19:27] mikko Steve-o: silent rules seem to work fine for me
[19:27] mikko im on 10.6.8
[19:28] mikko autoconf (GNU Autoconf) 2.68
[19:28] Steve-o one client complained about extra output, so I'm not sure
[19:29] Steve-o I need to start testing 5.2 for Lion support though
[19:30] Steve-o I've ported two unit test modules to Google Mock & Google Test and seems to work well, suitable for automation
[19:31] Steve-o need to try building on Windows though, which is the real benefit compared to the previous libcheck
[19:32] Steve-o I think I'm creating more problems with C99 and C++2003 incompatibilities though
[19:32] Steve-o I only just got real, fake, and mock objects working for C++ wrapping C
[19:33] Steve-o namespace hell
[19:33] Steve-o :(
[19:39] Steve-o I think I have to force my C89 patches to also work with C++2003 as I rebuild everything with C++ in order to fit into a Pgm::external namespace
[20:02] mikko Steve-o: are you rebuilding openpgm with C++ ?
[20:03] Steve-o for testing
[20:03] Steve-o because Googletest provides an automation interface for C++ projects
[20:06] Steve-o and some pretty neat mock concepts
[20:06] Steve-o similar to jquery
[20:07] lsousa hello there, I'm new to this zmq stuff. I'm loving it!
[20:07] Steve-o eventually I'll look at adding GoogleTest & GoogleMock to zmq, it definitely needs something inline,
[20:08] lsousa but I couldn't solve something I quess is trivial, even after doing the "Missing Message Problem Solver" tutorial
[20:09] lsousa my question is quite simple, I guess. I'm writting a program using the pub/sub stuff
[20:09] lsousa and I'd like to know what is the best way for making a client answer the publisher server
[20:10] lsousa when the server sends a message to one or more clients, everything works fine, but the contrary is not working
[20:10] lsousa to receive the messages on the server side, I've created another socket and connected to the same address I've binded the server
[20:10] lsousa is this correct?
[20:12] michelp lsousa, yep
[20:12] michelp PUB/SUB is one way
[20:13] michelp if you want the client to talk to the server, you have to have another socket going the other way
[20:13] lsousa michelp: thank you for your answer :)
[20:14] lsousa michelp: could you please suggest me what is the best protocol to use it? once each part of my program is running on a different process
[20:15] michelp lsousa, well, if you want the subscriber to "answer" what the publisher published to it, then REQ/REP would work well
[20:15] michelp the publisher would need to be polling or blocking on those sockets to get the answers, that design is up to you
[20:16] michelp i guess the subscriber could also PUSH to the subscriber that is PULL
[20:16] michelp lsousa, you definitely want to read the entire guide, take it slow and actually try out the examples if you can, then the right patterns will jump out at you
[20:18] michelp but it sounds like waht you want is two socket pairs, on the publisher use 'PUB' and 'PULL' and on the subscriber, use 'SUB' and 'PUSH'
[20:19] michelp when the subscriber gets data on the SUB, do the work, then PUSH the answer
[20:19] michelp then publisher PUBs data to the workers, and periodically PULLs the responses
[20:19] michelp that should at least get you started
[20:20] lsousa humm, this way seems to fit very well on my app
[20:20] michelp the 'ventilator' examples in the guide use these socket types, so definitely read that part of the guide again
[20:20] lsousa michelp: hehe, yep, I'm looking at it right now
[20:21] lsousa thank you michelp! I've followed your suggestion and, at least in my test scripts, things worked like a charm
[20:22] lsousa I'll now integrate it to my main program! :)
[20:22] michelp awesome, good luck
[20:22] lsousa thank you!
[21:07] mikko sustrik: alive?