Thursday December 23, 2010

[Time] NameMessage
[01:26] dos000 how does the network/tcp driver deliver packets for the same flow when a new packet arrives ? does it look up a hash tree to find the correct app listening on that socket ?
[01:26] dos000 i have to do kinda same thing ... and zmq does also
[01:47] Steve-o morning all
[02:15] Skaag morning Steve-o
[04:45] NoobFukaire I'm trying to test out the hello world stuff for C++ and zeromq
[04:45] NoobFukaire the server binds and I can see it's there with netstat (and I can telnet into it)
[04:45] NoobFukaire but when I connect with the client, I get Assertion failed: fetched (rep.cpp:232)
[04:46] NoobFukaire which is:
[04:46] NoobFukaire // Get next part of the message.
[04:46] NoobFukaire bool fetched = in_pipes [current]->read (msg_);
[04:46] NoobFukaire zmq_assert (fetched);
[04:46] NoobFukaire I'm not sure whats up with this lib?
[04:49] NoobFukaire
[04:49] NoobFukaire the stack trace
[04:54] NoobFukaire
[04:54] NoobFukaire
[04:54] NoobFukaire that's the code
[05:07] NoobFukaire okay nm, it had to do with some of my build paths
[06:55] andrewvc cremes: you around?
[07:59] sustrik hi mikko!
[08:16] mikko sustrik: hi
[08:16] sustrik morning
[08:16] sustrik there's a build patch from steve in the ML
[08:16] sustrik can i apply it?
[08:16] mikko steve-o?
[08:16] sustrik yes
[08:17] mikko i'll review today if thats ok
[08:17] sustrik sure, no haste
[08:17] mikko i would like to change the way we build openpgm
[08:17] sustrik what's the idea?
[08:17] mikko it seems that we are duplicating a lot of work at the moment
[08:17] mikko somehow that doesn't feel right
[08:17] sustrik yes, we are
[08:18] sustrik the alternative was to use scons
[08:18] sustrik which create additional dependencies
[08:18] sustrik any better idea?
[08:19] mikko i was thinking about two options:
[08:19] mikko a) invoke scons
[08:19] mikko b) isolate openpgm flags to their own files
[08:19] sustrik a) IMO is just shifting work from 0mq maintainers to 0mq users
[08:19] mikko i think that having large amount of conditionals in the isn't maintainable in the long run. also makes patching harder
[08:20] sustrik yes, agreed
[08:20] sustrik would b) help with that?
[08:20] mikko yes
[08:20] mikko we would have something like foreign/
[08:21] mikko and include from based on OS/compiler
[08:21] sustrik i see
[08:21] mikko not sure if that works yet but ill try today
[08:21] sustrik ok
[08:21] mikko that way we could track openpgm build changes more easily
[08:21] sustrik ack
[08:23] mikko also, we should add support for solaris / freebsd with openpgm and the other steve is working on os x build
[08:23] mikko so that's 5 conditionals in Makefile
[08:24] sustrik sounds messy
[08:25] mikko yeah, that was originally why i wanted to investigate invoking scons
[08:25] mikko as steve has done a lot of work with different platforms / compilers to get the build working
[16:55] NoobFukaire whats the best way to implement bidirectional messaging with zeromq?
[16:56] NoobFukaire two REQ-REP sockets?
[16:56] NoobFukaire I need either client to be able to send and receive messages at any point
[17:10] sustrik what are you trying to do?
[17:11] NoobFukaire I have two processes that need to exchange event information
[17:11] NoobFukaire those events can be created at any time, on either end
[17:12] NoobFukaire I wish there was some non blocking way to process inbound messages
[17:12] NoobFukaire with zeromq
[17:16] sustrik is that some kind of "multicast bus" scenario?
[17:16] sustrik i.e. any message published by anyone is received by anyone else?
[17:17] NoobFukaire no, it's basically client server
[17:17] NoobFukaire right now it's 1:1 but there might be multiple clients in the future
[17:17] NoobFukaire I'm find with 1:1 for now though
[17:17] sustrik so it's 1 server and N clients
[17:17] NoobFukaire *fine
[17:18] NoobFukaire I'll refactor for multiple clients when the time comes and I know zeromq better ;)
[17:18] sustrik what kind of interactions there are between server & client?
[17:18] sustrik server publishing stream of events to be received by each client?
[17:18] NoobFukaire I'm sending google protobuf messages back and forth
[17:19] sustrik back & forth = client initiated handshake?
[17:19] NoobFukaire they're just "events" they can be generated asynchronously by either side
[17:19] sustrik 0mq doesn't work that way, you have to identify patterns in the communications
[17:19] sustrik handshakes
[17:20] sustrik data distribution
[17:20] sustrik load balancing
[17:20] sustrik and similar
[17:20] NoobFukaire okay I don't care about any of that
[17:20] NoobFukaire this is about two process that exist on the same machine
[17:20] NoobFukaire and need to communicate
[17:20] sustrik use TCP then
[17:20] NoobFukaire there will never be a performance issue
[17:20] NoobFukaire gagh
[17:21] sustrik well, 0mq needs to know what you are trying to achieve to help you
[17:21] spht NoobFukaire: Just because you're scenario is 1:1 doesn't mean a 1:N solution doesn't fit
[17:21] spht s/you're/your
[17:21] sustrik just saying "anything" doesn't work
[17:21] NoobFukaire that's true but I'm just trying to keep things as simple as possible until I'm more familiar with zeromq
[17:22] sustrik you can use PAIR socket
[17:22] sustrik it's strictly 1:1
[17:22] sustrik but there are some features missing, like auto-reconnection etc.
[17:23] NoobFukaire spht: is there no way to check read zeromq input in a non blocking fashion?
[17:24] NoobFukaire I have a function that gets called at a regular interval
[17:24] NoobFukaire ideally, I'd like to have that check for input, process it and then process outbound messages
[17:24] sustrik use ZMQ_NOBLOCK flag in recv()
[17:24] spht NoobFukaire: you can setup zeromq non-blocking IIRC,
[17:24] spht what sustrik said ;)
[17:24] NoobFukaire right now it's blocking, so once recv is called, it blocks until input is returned
[17:25] NoobFukaire what did he say
[17:25] spht ZMQ_NOBLOCK
[17:25] NoobFukaire thanks
[17:26] spht but if your app is event driven to start with maybe you want to hook up the zeromq fd's into your runloop/reactor or what you use
[17:26] spht sustrik: IIRC thats possible, right?
[17:27] spht I'm a newbie myself so I hope I'm not spreading lies ;)
[17:29] NoobFukaire in my case, latency is very important and networking is not required at all
[17:29] NoobFukaire because these events may have UI functionality executed as a result
[17:33] sustrik spht: yes, it's possible
[17:34] sustrik VoobFukaire: for best latency you can collacate the sender and receiver in the same process
[17:34] sustrik each running is a different thread
[17:35] spht And for low latency you probably do not want to poll for new messages (as I guess was what you wanted with the non-blocking sockets)
[17:52] NoobFukaire I need to keep the UI responses ideally under ~100ms to eliminate any lag for the end user
[17:52] NoobFukaire granted the UI stuff itself is done in the toolkit's thread, so these events are more of a "trigger" for those
[17:53] NoobFukaire thank you for your help and suggestions
[17:54] NoobFukaire ZMQ_NOBLOCK seems to do what I want with nonblocking io, thank you sustrik
[17:57] sustrik you are welcome
[18:58] mikko sustrik: why doesn't pair auto-reconnect?
[18:59] sustrik nobody implemented it :)
[18:59] mikko is that a hint ?
[18:59] mikko :)
[18:59] sustrik :)
[18:59] mikko drove 800km today
[18:59] sustrik well, the thing is that PAIR is a socket type
[18:59] sustrik that is most prone to misuse
[19:00] sustrik so i quite like it being slightly broken
[19:00] mikko i abuse pair heavily in my program
[19:00] sustrik :)
[19:00] sustrik where have you gone?
[19:00] mikko dark place
[19:00] sustrik finland?
[19:00] mikko yeah
[19:01] sustrik 800km, than must be somewhere behing polar cirscle
[19:01] sustrik circle
[19:01] mikko drove to helsinki and back
[19:01] mikko just to find out that my friend's flight is so late that we drove back
[19:01] sustrik :|
[19:01] mikko now he might even miss the last bus
[19:02] sustrik what's exactly the problem with flights?
[19:02] sustrik too cold or what?
[19:03] mikko he is flying from london heathrow
[19:03] mikko the problem at heathrow is incompetence
[19:03] sustrik i guess so
[19:04] mikko it's not -29.5 celsius where i am
[19:04] mikko and all airports open
[19:04] sustrik actually, it's nice warm here in Bratislava today
[19:05] sustrik temperature went up by 20 degress or so
[19:05] sustrik btw, i'm messing with linux kernel
[19:06] sustrik the experience so far is that in comparison 0MQ codebase is pretty sane and well documented
[19:07] mikko heheh
[19:07] mikko im not suprised
[19:08] sustrik well, it's an old project so some amount of messiness is expected
[19:08] sustrik but the network stack seems to have no comments at all
[19:08] sustrik :(
[19:19] mikko dark arts
[22:06] andrewvc cremes: you around?