Wednesday January 26, 2011

[Time] NameMessage
[00:22] danlarkin hey all. Is there a tool [equivalent | similar] to netcat that I can use to test zmq services?
[03:18] traviscline danlarkin: without a client library it isnt' going to be very useful
[03:18] traviscline i'd just use a flexible language for your tests
[03:20] danlarkin yes, I suppose I will. Thanks
[04:03] stockMQ Hi..
[04:03] stockMQ I read in an old post that pgm multicast was only supported for Linux
[04:04] stockMQ Can anybody confirm if it is now available and working for Windows
[06:02] CIA-23 zeromq2: 03Thijs Terlouw 07master * rf7f1dfc 10/ (8 files in 4 dirs):
[06:02] CIA-23 zeromq2: ZMQ_RECONNECT_IVL_MAX socket option added
[06:02] CIA-23 zeromq2: It allows for exponential back-off strategy when reconnecting.
[06:02] CIA-23 zeromq2: Signed-off-by: Thijs Terlouw <> -
[06:52] jugg is there any guidelines being followed when creating socket options? There seems to be no apparent method to the selection of the socket option value type: int/uint32/int64/uint64...
[07:11] sustrik jugg: it's a mess
[07:12] sustrik as for new options, use int whenever possible
[07:12] sustrik that's in line with BSD socket API (POSIX)
[07:12] jugg ok
[13:26] toni Hi there. I have a XREQ-XREP specific question. I described my issue here: . Would be great if someone could take a look at it. Thanks
[13:29] sustrik in general reply should return via the same route the request was passed through
[13:29] sustrik you can hack in a different behaviour
[13:29] sustrik but it's not really advisable
[13:32] toni okay, but does the XREQ socket accept a reply from a broker, different to the one he sent the request?
[13:34] toni if it works, wouldnt it be simpler do go this way as to route the message back to the original server which send then sends the message back to the client?
[13:43] stockMQ Hi guys
[13:43] stockMQ Google protobuf or Thrift??
[13:43] stockMQ suggestions?
[13:53] sustrik drbobbeaty: hi, are you there?
[13:53] drbobbeaty sustrik: yup
[13:53] sustrik i've had a look at the problem you've had
[13:54] drbobbeaty thanks. I appreciate it.
[13:54] sustrik what you say is "It appears in socket_base.cpp (line 195) that in order to use the epgm protocol I need to NOT have options.requies_in and options.requires_out set to TRUE. Yet, in the constructor in sub.cpp (line 27), the superclass constructor - xsub_t (line 33 in xsub.cpp) is setting them BOTH to be TRUE."
[13:54] sustrik what i see on like 195 is:
[13:54] sustrik if ((protocol_ == "pgm" || protocol_ == "epgm") &&
[13:54] sustrik options.type != ZMQ_PUB && options.type != ZMQ_SUB && options.type !=
[13:54] sustrik ZMQ_XPUB && options.type != ZMQ_XSUB) {
[13:54] sustrik what version are you referring to?
[13:55] drbobbeaty I had pulled it from the git master on github.
[13:55] drbobbeaty I can pull it again, and try again.
[13:55] sustrik how does your line 195 look like?
[13:56] drbobbeaty It reads:
[13:56] drbobbeaty if ((protocol_ == "pgm" || protocol_ == "epgm") &&
[13:57] drbobbeaty options.requires_in && options.requires_out) {
[13:57] sustrik that looks outdated
[13:57] drbobbeaty mention of the exclusion on the XPUB
[13:57] drbobbeaty Let me pull again.
[13:58] sustrik drbobbeary: wait a sec!
[13:58] drbobbeaty OK
[13:58] sustrik i'm looking at the sub-forward branch
[13:58] sustrik let me check the master
[13:59] sustrik drbobbeaty: you are right!
[13:59] sustrik can you try to replace the condition there with the one I've pasted above?
[14:00] drbobbeaty sustrik: thanks... I was getting worried
[14:00] sustrik let me know if it solves the problem
[14:00] drbobbeaty OK, will do.
[14:00] sustrik thanks
[14:17] stockMQ I am dealing with exchange feed and for me Performance and Size are priority
[14:18] stockMQ I am looking at options like Google protobuff,thrift and JSON + zlib ...
[14:18] sustrik stockMQ: you have to test the performance yourself
[14:19] stockMQ ok
[14:19] sustrik there's no such thing as generic performance
[14:19] sustrik it depends on hardware, different layers of software etc.
[14:20] stockMQ yes but looking from the size perspective
[14:20] stockMQ obviously protobuf and thrift will have a serialization wrapper around
[14:21] stockMQ so i just wanted to know say if a struct with two Int was to be serialized using protobuf or thrift will the resulting message size increase very much
[14:22] stockMQ I am assuming that a lot of zeroMQ users would be using any of the above messaging
[14:23] drbobbeaty If you're talking about Exchange Feeds like NASDAQ or SIAC/OPRA, then I use ZeroMQ for that and push a lot of messages a sec through my ticker plants.
[14:24] stockMQ yes.. I am from India and I am talking about NSE (National stock exchange ) feed similar to Nasdaq
[14:25] drbobbeaty Then, yes. I use ZMQ for these Quote, Trade, etc. messages.
[14:25] stockMQ ok..
[14:25] drbobbeaty I do NOT use protobufs, but do use the Google VarInt encoding to pack ints.
[14:26] stockMQ so my understanding is once you receive the feed you might be processing it (aggregating maybe) and then publish
[14:27] stockMQ am i right ?
[14:27] drbobbeaty Actually, I decode the feed, make a message for each 'tick', and then serialize that message into a ZMQ message and send it. On the receiver, I get the payload, deserialize it into a message and use it.
[14:28] drbobbeaty I also add in conflation, use multiple multicast channels to distribute the load, etc.
[14:28] drbobbeaty But that's the basic picture.
[14:28] stockMQ okk.. your use case sounds similar
[14:28] stockMQ once i receive the feed..for each tick i create this struct
[14:28] stockMQ struct ScripRecord { int test; int token; float open,high,low,close; double volume, volumeTradedToday, dateTime, buyVolume, sellVolume; float bid, ask, openInterest; float dayOpen, dayLow, dayHigh, previousClose; ScripRecord() { memset(this, 0, sizeof(ScripRecord)); } };
[14:29] drbobbeaty The serialization scheme is more like boost's serialization scheme - with a version component for changes in the messages. But it's not hard. You can use about anything.
[14:29] stockMQ and i have a PUB-SUB pattern and i want to publish this struct to all my subscribers
[14:30] stockMQ I understand that i cannot just typecast into char* and send
[14:30] stockMQ so i was exploring options like protobufs
[14:30] drbobbeaty You can do a lot. Msgpack is pretty popular... as is protobufs.
[14:31] stockMQ But if I am not wrong you are suggesting that if I am sure that both PUB and SUB are on the same platform (or need not be)..instead of protobufs i can use 0MQ messages
[14:31] stockMQ @seb.. thanks .. i shall have a look
[14:32] drbobbeaty What I'm saying is that ZeroMQ is a *channel*... you have to decide what the "payload" is going to be and how you are going to pack/unpack/encode/decode it.
[14:32] drbobbeaty I need C++/Java compatibility, so I have to pack on C++ and unpack on C++ and Java. That means I can't just do a memcpy() of a struct.
[14:32] drbobbeaty But that's me.
[14:33] drbobbeaty You have to decide what to put in your messages so you can get the data out you need.
[14:33] drbobbeaty protobufs and msgpack are both ways to do this cross-platform.
[14:33] drbobbeaty The ZeroMQ message, itself, is agnostic to this.
[14:34] stockMQ yep.. i need C++ compatibility..but memcpy does not really work
[14:35] stockMQ But yes i would like to design the solution keeping in mind to scale to other platforms in future
[14:37] drbobbeaty It's up to you, but I'd think of a serialization/deserialization scheme as independent of the ZeroMQ choice. Then, if your company picks 29West or Tibco, or RabbitMQ, you're safe in your serialization scheme. For many reasons, I'm using ZeroMQ and am very happy with it, but you mileage may vary.
[14:38] stockMQ I am sorry..but should i be calling you Bob?
[14:39] drbobbeaty That's fine... don't care :)
[14:41] stockMQ anyways.. ZeroMQ does not provide any serialization/deserialization above their ZeroMQ message.. do they..?So if i have to send structured data via zeroMQ..I will have to create a protobuf/thrift/etc message object..then serialize that object and send it across
[14:41] stockMQ Just want to confirm i have got it absolutely right
[14:42] drbobbeaty You have this absolutely right. ZeroMQ is a "delivery system" - a very fancy one, but just a delivery system. What you ask it to deliver is up to you. 100%.
[14:43] drbobbeaty You will find, I think, that when you look at protobufs, or msgpack, it'll be obvious as they won't talk about "delivery" -- only the "package".
[14:43] stockMQ coool.. Now i will look into VarInt and msgpack
[14:43] drbobbeaty Same with boost serialization
[14:44] stockMQ For your solution did u get a chance to compare performance of the various options
[14:46] stockMQ I meant the "package" options available
[14:48] drbobbeaty Yes.
[14:49] drbobbeaty I wasn't thrilled with boost serialization -- too much overhead for me
[14:49] drbobbeaty protobufs was "OK", and I love the VarInt encoding and decoding in that, but the rest wasn't needed and I've had experiences in the past where major version changes in the protobufs libraries causes a big problem to clients.
[14:50] drbobbeaty I also needed cross-platform support.
[14:50] drbobbeaty At the same time, another group at The Shop had a data codec for serialization on sockets, and it was working well. If I used that, then I would not have to write anything for the Java side.
[14:51] drbobbeaty So I picked it and optimized it a bit for speed, and it's been very good for me.
[14:51] drbobbeaty I looked at it this way... the exchange data is very simple: int, float, string - that's about it. Very easy to make a codec fast for these data types.
[14:52] drbobbeaty floats are simple byte copy... strings are packed easily - or not - they aren't all that big.
[14:52] drbobbeaty int were the challenge - hence the VarInt encoding.
[14:52] drbobbeaty (sorry for going on and on... I'll stop now)
[14:53] stockMQ It has been extremely helful
[14:53] stockMQ I would not like you to stop
[14:53] stockMQ :)
[14:55] stockMQ Even my data has just int,double and float
[14:56] drbobbeaty I'm usually here... if you have questions, ask.
[14:59] stockMQ cool.. I was just looking at VarInt and i understand that with varInt encoding that when an int is serialized it will occupy variable bits
[15:00] drbobbeaty Yes, and that's a good thing. Volumes, Size, Qty - all ints but vastly different sizes. They take only what they need with VarInt
[15:01] stockMQ but is that restricted to just int datatype or i can use it for floats and doubles too
[15:01] drbobbeaty Just integers.
[15:01] drbobbeaty If you look at the code, it's clear why. But it's just ints.
[15:08] stockMQ is there a reason you used VarInt but not protocol buffer to serialize
[15:08] stockMQ Is it because it only serialized to a stream
[15:10] drbobbeaty No, my reason for not choosing protobufs is my experience with it has been "less than wonderful", and I had a very good alternative. Had my own serialization not been up to the task, I'd have looked hard at msgpack.
[15:12] stockMQ ok.. so i get that you are not FOR protobuf and BOOST..:)
[15:13] stockMQ
[15:13] stockMQ It does not look very bad
[15:14] stockMQ comparable to VarInt in my opinion
[15:18] stockMQ You there?
[15:18] stockMQ I will tell you what i tried without the "Packages"
[15:19] stockMQ I had the struct like i posted above
[15:19] stockMQ In C++ i used the pragma directive
[15:19] stockMQ #pragma pack(1)
[15:20] stockMQ and i typecast the struct (char *) and send it using zeroMQ
[15:21] stockMQ but if the size of the struct was say 92 at PUB
[15:21] stockMQ at the receiving end it always came as 1Byte
[15:21] drbobbeaty I'm here...
[15:22] stockMQ I figured that ZEROMQ message..which basically is a null terminated String would be the issue and i will have to use a buffer and serializer
[15:22] drbobbeaty My suggestion is to make a serialization/deserialization scheme - based on something. Then write a test app to prove that you can encode and decode the messages.
[15:23] drbobbeaty Once you have that, then you'll know exactly what a message encodes to. Do a hex dump of the data before sending. Do another on the receiver. Look for differences.
[15:23] drbobbeaty I did. It helps a lot in getting this going.
[15:24] stockMQ ohkk
[15:24] drbobbeaty Assume nothing. Check everything. It saves time in the end to be careful in the beginning - especially on the encoding/decoding.
[15:24] stockMQ true
[15:24] drbobbeaty Plus, that way you can see how long it takes to decode 1000 msgs and to what size. You can play with the codec to get it to the optimal size/speed you need.
[15:24] guido_g ømq messages are not 0 terminated
[15:25] drbobbeaty guido_g is totally right... it's a set of bytes. Period.
[15:25] guido_g ack
[15:25] guido_g except for pub/sub :)
[15:25] guido_g there might be an issue w/ the topic
[15:26] stockMQ ok
[15:28] drbobbeaty In my experience with the ZMQ_PUB/ZMQ_SUB sockets and OpenPGM, the data you get from a recv() call is the "payload" -- no topic in that zmq::message_t
[15:28] guido_g the topic is simply the start of the message
[15:29] cremes stockMQ: here's some generic benchmarking results of some of the top serialization packages
[15:29] cremes
[15:29] guido_g so if one _sends_ w/ a topic, it become a problem
[15:29] cremes it doesn't include msgpack... that's probably too new
[15:29] cremes as always, test against *your* data structures to make sure the algo is fast enough
[15:30] stockMQ Thanks cremes
[15:30] stockMQ guido if i show a snippet of the code
[15:30] stockMQ will it help to see if topic could be an issue
[15:30] guido_g you should know what you send
[15:32] guido_g <- even from python msgpack isn't that bad
[15:32] stockMQ Well .. I just bind the publisher to pgm socket and try sending the struct typecasted to char*
[15:33] stockMQ and on receive i expected that on typecasting back to struct it should work..which did not..I am not sure i understand the TOPIC.. i am not explicitly creating a topic
[15:33] cremes stockMQ: i have a suggestion... make sure to abstract out the calls to encode/decode
[15:33] cremes for testing purposes, use a simple and human-readable encoding like json
[15:33] cremes this makes debugging very easy
[15:33] cremes when you need to worry about perf, switch to a tighter encoding
[15:33] cremes which should be simple if you abstracted your code proeprly
[15:34] guido_g jepp, readable serialization format can save the day
[15:34] cremes that is what i have done for the trading system i wrote; using json early on was a great call on my part for making debugging easy
[15:35] cremes and unless you are counting nanoseconds for a HFT, even json is often fast enough
[15:35] stockMQ ok..cremes .. but since json srting will act like just another string i think it will not be an issue. I have tried the sockets with test strings and everything works perfect
[15:35] cremes stockMQ: oh, i see; you are still figuring out how to use 0mq multi-part messages
[15:36] drbobbeaty sustrik: your change got rid of the error in the first call to connect(), but it did NOT allow messages to flow from the ZMQ_PUB server. To be fair, the ZMQ_PUB was running a slightly older version of ZMQ (still post-2.1.0). If this is an issue, let me know, but it looks like there is something else going on that's causing the messages to not be picked up by the ZMQ_SUB.
[15:36] cremes you definitely don't want your "encoding" to include that delimter
[15:37] sustrik drbobbeaty: ok
[15:37] stockMQ actually i have not dabbled with multi part yet.. I figured i should be able to send a C++ struct typecasted to char* in just a single message
[15:38] drbobbeaty sustrik: if you have any other ideas for me to try, please let me know. I'll be glad to try them.
[15:38] stockMQ I could be wrong though..Just started using zeroMQ a couple of days back
[15:38] cremes stockMQ: no, that isn't supported because the struct memcpy could contain the magic byte used as a message delimiter
[15:38] stockMQ ohk
[15:38] cremes and if you are using pub/sub, the topic byte-string is terminated with /0 too
[15:39] cremes please read the guide and the man page on the protocol wire format
[15:40] cremes and definitely read about multi-part messages
[15:40] stockMQ cremes..if it is ok..can i request for an elaboration..maybe an example..if the struct is say struct ScripRecord { int token;
[15:40] stockMQ float open,high,low,close;
[15:40] stockMQ };
[15:40] cremes sure...
[15:40] sustrik drbobbeaty: does it work with release 2.1.0?
[15:40] cremes you can't always control how the compiler will lay that struct out in memory
[15:40] stockMQ okk
[15:41] cremes it may add padding bytes in various places to align structures for fast memory access
[15:41] drbobbeaty sustrik: don't know... let me double-check.
[15:41] cremes it just isn't safe to do
[15:41] stockMQ okk
[15:41] stockMQ you mean a memcpy
[15:41] cremes stockMQ: also, if you are doing any cross-platform messaging (unix to windows to mac) then memory alignments could change
[15:42] cremes and trying to "cast" the message on another platform could result in garbage
[15:42] cremes just don't do it; it isn't portable
[15:42] stockMQ ok
[15:42] cremes and i really doubt you need that much "speed" anyway; don't get too hung up on performance yet
[15:42] stockMQ okk
[15:43] cremes make your program work correctly first, then worry about micro-optimizations like serialization
[15:44] stockMQ okk
[15:45] stockMQ so i should use JSON/protobuf/thrift/msgpack/JSON+zlib/custom encoding
[15:45] cremes for portability, yes
[15:45] stockMQ Is that the gist guys..Isnt it
[15:46] cremes i don't know who the gist guys are
[15:47] stockMQ :) i just meant that 'to summarize the discussion'
[15:48] cremes ah! ok. ;)
[15:50] stockMQ Thank you very much bob,Cremes,seb , guido,sustrik :)
[15:52] stockMQ Another advice i need is on the storage of tick data at SUB end
[15:53] stockMQ i am just writing it as binary to a file.
[15:53] stockMQ But the issue is there could be other Technical Analysis apps reading from it parallely
[15:53] cremes stockMQ: whatever is convenient for you; you are likely going to feed intra-day ticks to consumers via an in-memory db
[15:53] cremes and anything older (historical data) can be sourced from just about any database you like
[15:54] stockMQ the tick data takes the following route basically Exchange->PUB->SUB
[15:54] stockMQ now the SUBs are going to be basic laymen brokers
[15:55] stockMQ So i cannot expect them to maintain/setup a RDBMS
[15:55] stockMQ i was looking at options like binary flat file,HDF5,embedded DBs like BDB,Kyoto cabinet etc
[15:56] stockMQ but the issue is with flat file and HDF5 it will still not handle concurrency
[15:57] cremes stockMQ: i don't like that design so i have no opinion about flat file, hdf5, etc
[15:58] cremes besides, all db access should be abstracted away as another service on your 0mq bus
[15:58] cremes that way each of your "brokers" doesn't need to set up a DB connection
[15:58] cremes it merely needs to know how to ask for what it wants on the 0mq bus and the service responsible for DB work can respond
[15:59] cremes and you can scale that with push/pull or req/rep sockets that do auto load balancing for you
[15:59] cremes stockMQ: you mentioned you were new to 0mq; that's obvious now
[15:59] cremes make sure to read the guide... the concepts it covers will help you figure out a better design for your system
[15:59] stockMQ yes i am new
[16:00] cremes let 0mq help improve the design; right now you are starting out with a design and trying to wedge 0mq sockets into it
[16:00] cremes good luck, bbiab
[16:01] stockMQ bbiab?
[16:05] traviscline be back in a bit
[16:05] stockMQ :)
[16:09] stockMQ Actually..Cremes by broker it did not mean the "ZeroMQ Broker" but the stock broker as clients who will be using the SUB app
[16:19] drbobbeaty sustrik: I've run my tests with three versions: 2.1.0 (ref), my 2.1.0 w/ the RECOVERY_IVL patch (me), and git master (master). All are ZMQ_PUB/ZMQ_SUB socket apps. When sending from 'ref' to 'ref' - OK. When sending from 'me' to 'me' - OK. When sending from 'ref' to 'ref' - nothing received. When sending from 'me' to 'ref' - OK. So the git master (ref) is not receiving - but not generating errors, either.
[16:19] drbobbeaty sustrik: That third test was 'master' to 'master'.
[16:27] francois Hi all! I have multiple producers and multiple consumers for my messages, but each message must be processed only once, on any consumer. What socket types am I supposed to use? PUSH / PULL doesn't work since PUSH will only bind once.
[16:33] mikko francois__: yes, you can either connect all subscribers to all publishers or you can use a device in the middle
[16:36] francois I understand how the device works, but how would "connect all subscribers to all publishers" work? Each consumer / subscribers would need to know about all producers / publishers?
[16:38] mikko francois__: for all consumers you would need to know all publishers
[16:38] francois Makes sense, yes. Thanks for your help!
[16:38] francois Going the device route for now.
[16:38] mikko i think there are examples in zguide regarding routing
[17:15] francois @mikko Thank you again very much: a device that binds to both sides is what I was looking for. I was under the false impression that the side that wanted to receive had to bind - even though the docs are correct and say either side can bind/connect.
[17:17] mikko francois__: cool
[17:21] cremes francois__: you might want to check this out for more ruby examples
[17:21] cremes
[17:22] cremes btw, ruby can do all of the same stuff as all of the other bindings; there are no limitations by choosing ruby
[18:09] borior Hi all. I'm just getting started with 0MQ, coding up a messaging channel between a piece of numerical code and a python user interface, and would appreciate some input on the design.
[18:09] borior There are two main messaging channels: data, and control
[18:10] borior At the moment, I envisage data being PUSH/PULL from the numerical side to the UI, and control being REQ/REP from the UI to the numerical stuff.
[18:11] borior Does that sound sensible? And if so, how do I deal with the fact that I want to be sending many more messages down the PUSH pipe than I expect to receive from the REQ/REP control pipe from the UI? It seems like if I make a blocking call to control_sock.recv() I then can't send any messages down the data socket until a control message is sent...?
[18:26] borior seb`: ah, thank you. I didn't know that was possible.
[18:27] borior silly me. I should have grepped the guide for "poll". thanks for the pointer
[19:40] traviscline seb`: details?
[19:40] codebeaker Hi all, is "no such file or directory" a sane response to zmq_term() /
[19:41] traviscline yes
[19:41] traviscline ah i see
[19:41] traviscline one sec
[19:41] traviscline sorry about that
[19:43] traviscline i'll push in a bit
[19:47] traviscline seb`: pushed
[19:47] traviscline sorry again, thanks for alerting me
[19:48] traviscline i have a testing branch i'm working on so that sort of thing will not happen in the future
[19:51] traviscline seb`: great, doing some profiling and working on some optimizations now
[19:52] traviscline just reducing overhead with some type hints in cython
[20:04] mikko codebeaker: what do you mean?
[20:05] codebeaker mikko: (first time using zmq) I'm using the zmq_term on a context that appears to have been initialized successfully (0 return) - and I get (from zmq_strerror(errno)) "no such file or directory"
[20:05] codebeaker hrm :\
[20:06] codebeaker looks like my mistake… actually, I had a !zmq_term() –– so there must be an errno set already from some other piece of code :\
[20:08] mikko codebeaker: most likely
[20:08] mikko codebeaker: it will return != 0 on error
[20:08] codebeaker yeah, thanks mikko
[20:08] codebeaker I actually didn't look at the code ^ today before asking, it's a problem since a day ago… and I just needed fresh eyes I gues
[20:08] mikko have you read zguide ?
[20:08] codebeaker reading if(zmq_term()){ /* log error */} is still strange on my eyes (new to C)
[20:08] mikko it's a very good resource to kick-start development
[20:09] codebeaker yeah, the long, long, guide with the zmw helpers and so forth is exceptionally good
[20:31] pieterhintjens codebreaker: usually it's clearer to write 'if (zmq_term () != 0) { /* log error */'
[20:31] codebeaker pieterh: good call :)
[20:32] codebeaker because… there's some inconsistency, at least between libs about how that should work, although afaik the "non zero on error" is supposed to be ubuquitos
[20:32] pieterh every library has its own rules
[20:32] pieterh zero = OK is the most common but it's not universal
[20:33] codebeaker ja, good tip about making the if() more explicit though
[20:33] codebeaker do you more experienced guys usually wrap these kind of functions?
[20:33] codebeaker I have done, into a zmq_up() and zmq_down()
[20:33] pieterh yes, very often
[20:33] codebeaker ok, cool - so I'm not acting like a complete noob :)
[20:34] pieterh perhaps not the context initialization and term, those are not used all over the place
[20:34] pieterh certainly other functions, like sending / receiving messages
[20:35] codebeaker sure, actually I'm over engineering slightly here - the code I'm writing is a library, so I don't want the stuff linked against it to care about the implementation, so it's hidden
[20:35] codebeaker "up" and "down"
[20:35] codebeaker but, I recognised from the long (and epic) guide - that the zmq_helpers are very good
[20:36] pieterh when you can systematically replace 10 lines of code with one call, it's profitable
[20:36] pieterh the ZFL library takes this a step further but it's also more complex to understand and use
[20:36] pieterh
[20:37] codebeaker ah, cool
[20:37] codebeaker Ohh, pieterh I see you know a little about ømq :)
[20:37] pieterh a little, as a user
[20:39] codebeaker I'm a little under two weeks into learning C
[20:39] codebeaker the hard way :)
[20:39] codebeaker like setting out to build an assistant for my media synchronisation hobbies at home :) with ZMQ as the transport method to sync hosts
[20:40] codebeaker ^ it's been incredibly helpful, sockets are a disaster :)
[20:42] pieterh codebreaker: you can use any language you like, C can be quite tricky to learn
[20:42] codebeaker still lots and lots to learn
[20:42] codebeaker ah, pieterh there's some reasons I have to use C :)
[20:42] codebeaker but - yeah, I saw zmq has some great bindings for other languages
[20:43] pieterh you might want to read the ZFL docs & code, it's a nice way of writing C as classes
[20:44] codebeaker ^ I saw that, nice stuff - I just had to learn some tricky stuff in C with using structs to store state between function calls… it felt pretty ugly (but it's lightweight, and works very well) - but I like your idea
[20:44] codebeaker Im still figuring out what is sane, and insane with C compared to my background plains :)
[20:46] codebeaker if I might ask - since I don't want to have to set up every machine like a dev server… I should be able to find a way to load the whole of ZMQ into my app, at compile time - something like a static link, right ?
[20:47] pieterh yes, static linking should be possible
[20:47] codebeaker hrm, cool - learning way more than my brain can handle at the moment :)
[20:47] codebeaker and, pieterh I just saw your name… CEO of iMatix… nice, nice work
[20:47] pieterh np :-)
[20:48] codebeaker I can't believe how good zmq is, compared to the alternatives
[20:48] pieterh there are alternatives?
[20:48] pieterh :-)
[20:48] codebeaker ( coming from a company where the business back bone was AMQP :-\ )
[20:48] pieterh AMQP is pretty OK compared to its own alternatives
[20:48] pieterh but the knowledge of "how to do messaging" has moved forwards really quickly
[20:48] mikko hmm
[20:48] codebeaker I remember about a week after the infrastructure was "finished" ømq got some real press, and lots of us figured this might have been a smarter move… but 4 months invested in AMQP - the business wasn't even going to give us a shot at experimenting
[20:49] pieterh mikko: ?
[20:52] mikko humming out loud
[20:52] mikko looking at the dns issue
[21:15] codebeaker pieterh: thanks for your help, and advice :)
[21:15] codebeaker *bow*
[21:15] pieterh codebeaker: np, anytime
[21:15] codebeaker hehe, I might just take you up on that… see you again
[22:32] mikko sustrik: there?
[23:45] traviscline II run into "Assertion failed: !more || pipes [current] != pipe_ (fq.cpp:61)" from python if I don't try several recvs (and get EAGAINS) before switching to polling the file descriptor from a socket
[23:49] traviscline If I don't start waiting until I've seen 10 of them I can't produce the error
[23:50] traviscline at 3 I got the error, at 4 i've yet to, kind of unnerving