ZeroMq IRC Log

Wednesday January 26, 2011

[Time] Name	Message
[00:22] danlarkin	hey all. Is there a tool [equivalent \| similar] to netcat that I can use to test zmq services?
[03:18] traviscline	danlarkin: without a client library it isnt' going to be very useful
[03:18] traviscline	i'd just use a flexible language for your tests
[03:20] danlarkin	yes, I suppose I will. Thanks
[04:03] stockMQ	Hi..
[04:03] stockMQ	I read in an old post that pgm multicast was only supported for Linux
[04:04] stockMQ	Can anybody confirm if it is now available and working for Windows
[06:02] CIA-23	zeromq2: 03Thijs Terlouw 07master * rf7f1dfc 10/ (8 files in 4 dirs):
[06:02] CIA-23	zeromq2: ZMQ_RECONNECT_IVL_MAX socket option added
[06:02] CIA-23	zeromq2: It allows for exponential back-off strategy when reconnecting.
[06:02] CIA-23	zeromq2: Signed-off-by: Thijs Terlouw <thijsterlouw@gmail.com> - http://bit.ly/gCNsRo
[06:52] jugg	is there any guidelines being followed when creating socket options? There seems to be no apparent method to the selection of the socket option value type: int/uint32/int64/uint64...
[07:11] sustrik	jugg: it's a mess
[07:12] sustrik	as for new options, use int whenever possible
[07:12] sustrik	that's in line with BSD socket API (POSIX)
[07:12] jugg	ok
[13:26] toni	Hi there. I have a XREQ-XREP specific question. I described my issue here: https://gist.github.com/796675 . Would be great if someone could take a look at it. Thanks
[13:29] sustrik	in general reply should return via the same route the request was passed through
[13:29] sustrik	you can hack in a different behaviour
[13:29] sustrik	but it's not really advisable
[13:32] toni	okay, but does the XREQ socket accept a reply from a broker, different to the one he sent the request?
[13:34] toni	if it works, wouldnt it be simpler do go this way as to route the message back to the original server which send then sends the message back to the client?
[13:43] stockMQ	Hi guys
[13:43] stockMQ	Google protobuf or Thrift??
[13:43] stockMQ	suggestions?
[13:53] sustrik	drbobbeaty: hi, are you there?
[13:53] drbobbeaty	sustrik: yup
[13:53] sustrik	i've had a look at the problem you've had
[13:54] drbobbeaty	thanks. I appreciate it.
[13:54] sustrik	what you say is "It appears in socket_base.cpp (line 195) that in order to use the epgm protocol I need to NOT have options.requies_in and options.requires_out set to TRUE. Yet, in the constructor in sub.cpp (line 27), the superclass constructor - xsub_t (line 33 in xsub.cpp) is setting them BOTH to be TRUE."
[13:54] sustrik	what i see on like 195 is:
[13:54] sustrik	if ((protocol_ == "pgm" \|\| protocol_ == "epgm") &&
[13:54] sustrik	options.type != ZMQ_PUB && options.type != ZMQ_SUB && options.type !=
[13:54] sustrik	ZMQ_XPUB && options.type != ZMQ_XSUB) {
[13:54] sustrik	what version are you referring to?
[13:55] drbobbeaty	I had pulled it from the git master on github.
[13:55] drbobbeaty	I can pull it again, and try again.
[13:55] sustrik	how does your line 195 look like?
[13:56] drbobbeaty	It reads:
[13:56] drbobbeaty	if ((protocol_ == "pgm" \|\| protocol_ == "epgm") &&
[13:57] drbobbeaty	options.requires_in && options.requires_out) {
[13:57] sustrik	that looks outdated
[13:57] drbobbeaty	...no mention of the exclusion on the XPUB
[13:57] drbobbeaty	Let me pull again.
[13:58] sustrik	drbobbeary: wait a sec!
[13:58] drbobbeaty	OK
[13:58] sustrik	i'm looking at the sub-forward branch
[13:58] sustrik	let me check the master
[13:59] sustrik	drbobbeaty: you are right!
[13:59] sustrik	can you try to replace the condition there with the one I've pasted above?
[14:00] drbobbeaty	sustrik: thanks... I was getting worried
[14:00] sustrik	let me know if it solves the problem
[14:00] drbobbeaty	OK, will do.
[14:00] sustrik	thanks
[14:17] stockMQ	I am dealing with exchange feed and for me Performance and Size are priority
[14:18] stockMQ	I am looking at options like Google protobuff,thrift and JSON + zlib ...
[14:18] sustrik	stockMQ: you have to test the performance yourself
[14:19] stockMQ	ok
[14:19] sustrik	there's no such thing as generic performance
[14:19] sustrik	it depends on hardware, different layers of software etc.
[14:20] stockMQ	yes but looking from the size perspective
[14:20] stockMQ	obviously protobuf and thrift will have a serialization wrapper around
[14:21] stockMQ	so i just wanted to know say if a struct with two Int was to be serialized using protobuf or thrift will the resulting message size increase very much
[14:22] stockMQ	I am assuming that a lot of zeroMQ users would be using any of the above messaging
[14:23] drbobbeaty	If you're talking about Exchange Feeds like NASDAQ or SIAC/OPRA, then I use ZeroMQ for that and push a lot of messages a sec through my ticker plants.
[14:24] stockMQ	yes.. I am from India and I am talking about NSE (National stock exchange ) feed similar to Nasdaq
[14:25] drbobbeaty	Then, yes. I use ZMQ for these Quote, Trade, etc. messages.
[14:25] stockMQ	ok..
[14:25] drbobbeaty	I do NOT use protobufs, but do use the Google VarInt encoding to pack ints.
[14:26] stockMQ	so my understanding is once you receive the feed you might be processing it (aggregating maybe) and then publish
[14:27] stockMQ	am i right ?
[14:27] drbobbeaty	Actually, I decode the feed, make a message for each 'tick', and then serialize that message into a ZMQ message and send it. On the receiver, I get the payload, deserialize it into a message and use it.
[14:28] drbobbeaty	I also add in conflation, use multiple multicast channels to distribute the load, etc.
[14:28] drbobbeaty	But that's the basic picture.
[14:28] stockMQ	okk.. your use case sounds similar
[14:28] stockMQ	once i receive the feed..for each tick i create this struct
[14:28] stockMQ	struct ScripRecord { int test; int token; float open,high,low,close; double volume, volumeTradedToday, dateTime, buyVolume, sellVolume; float bid, ask, openInterest; float dayOpen, dayLow, dayHigh, previousClose; ScripRecord() { memset(this, 0, sizeof(ScripRecord)); } };
[14:29] drbobbeaty	The serialization scheme is more like boost's serialization scheme - with a version component for changes in the messages. But it's not hard. You can use about anything.
[14:29] stockMQ	and i have a PUB-SUB pattern and i want to publish this struct to all my subscribers
[14:30] stockMQ	I understand that i cannot just typecast into char* and send
[14:30] stockMQ	so i was exploring options like protobufs
[14:30] drbobbeaty	You can do a lot. Msgpack is pretty popular... as is protobufs.
[14:31] stockMQ	But if I am not wrong you are suggesting that if I am sure that both PUB and SUB are on the same platform (or need not be)..instead of protobufs i can use 0MQ messages
[14:31] stockMQ	@seb.. thanks .. i shall have a look
[14:32] drbobbeaty	What I'm saying is that ZeroMQ is a channel... you have to decide what the "payload" is going to be and how you are going to pack/unpack/encode/decode it.
[14:32] drbobbeaty	I need C++/Java compatibility, so I have to pack on C++ and unpack on C++ and Java. That means I can't just do a memcpy() of a struct.
[14:32] drbobbeaty	But that's me.
[14:33] drbobbeaty	You have to decide what to put in your messages so you can get the data out you need.
[14:33] drbobbeaty	protobufs and msgpack are both ways to do this cross-platform.
[14:33] drbobbeaty	The ZeroMQ message, itself, is agnostic to this.
[14:34] stockMQ	yep.. i need C++ compatibility..but memcpy does not really work
[14:35] stockMQ	But yes i would like to design the solution keeping in mind to scale to other platforms in future
[14:37] drbobbeaty	It's up to you, but I'd think of a serialization/deserialization scheme as independent of the ZeroMQ choice. Then, if your company picks 29West or Tibco, or RabbitMQ, you're safe in your serialization scheme. For many reasons, I'm using ZeroMQ and am very happy with it, but you mileage may vary.
[14:38] stockMQ	I am sorry..but should i be calling you Bob?
[14:39] drbobbeaty	That's fine... don't care :)
[14:41] stockMQ	anyways.. ZeroMQ does not provide any serialization/deserialization above their ZeroMQ message.. do they..?So if i have to send structured data via zeroMQ..I will have to create a protobuf/thrift/etc message object..then serialize that object and send it across
[14:41] stockMQ	Just want to confirm i have got it absolutely right
[14:42] drbobbeaty	You have this absolutely right. ZeroMQ is a "delivery system" - a very fancy one, but just a delivery system. What you ask it to deliver is up to you. 100%.
[14:43] drbobbeaty	You will find, I think, that when you look at protobufs, or msgpack, it'll be obvious as they won't talk about "delivery" -- only the "package".
[14:43] stockMQ	coool.. Now i will look into VarInt and msgpack
[14:43] drbobbeaty	Same with boost serialization
[14:44] stockMQ	For your solution did u get a chance to compare performance of the various options
[14:46] stockMQ	I meant the "package" options available
[14:48] drbobbeaty	Yes.
[14:49] drbobbeaty	I wasn't thrilled with boost serialization -- too much overhead for me
[14:49] drbobbeaty	protobufs was "OK", and I love the VarInt encoding and decoding in that, but the rest wasn't needed and I've had experiences in the past where major version changes in the protobufs libraries causes a big problem to clients.
[14:50] drbobbeaty	I also needed cross-platform support.
[14:50] drbobbeaty	At the same time, another group at The Shop had a data codec for serialization on sockets, and it was working well. If I used that, then I would not have to write anything for the Java side.
[14:51] drbobbeaty	So I picked it and optimized it a bit for speed, and it's been very good for me.
[14:51] drbobbeaty	I looked at it this way... the exchange data is very simple: int, float, string - that's about it. Very easy to make a codec fast for these data types.
[14:52] drbobbeaty	floats are simple byte copy... strings are packed easily - or not - they aren't all that big.
[14:52] drbobbeaty	int were the challenge - hence the VarInt encoding.
[14:52] drbobbeaty	(sorry for going on and on... I'll stop now)
[14:53] stockMQ	It has been extremely helful
[14:53] stockMQ	I would not like you to stop
[14:53] stockMQ	:)
[14:55] stockMQ	Even my data has just int,double and float
[14:56] drbobbeaty	I'm usually here... if you have questions, ask.
[14:59] stockMQ	cool.. I was just looking at VarInt and i understand that with varInt encoding that when an int is serialized it will occupy variable bits
[15:00] drbobbeaty	Yes, and that's a good thing. Volumes, Size, Qty - all ints but vastly different sizes. They take only what they need with VarInt
[15:01] stockMQ	but is that restricted to just int datatype or i can use it for floats and doubles too
[15:01] drbobbeaty	Just integers.
[15:01] drbobbeaty	If you look at the code, it's clear why. But it's just ints.
[15:08] stockMQ	is there a reason you used VarInt but not protocol buffer to serialize
[15:08] stockMQ	Is it because it only serialized to a stream
[15:10] drbobbeaty	No, my reason for not choosing protobufs is my experience with it has been "less than wonderful", and I had a very good alternative. Had my own serialization not been up to the task, I'd have looked hard at msgpack.
[15:12] stockMQ	ok.. so i get that you are not FOR protobuf and BOOST..:)
[15:13] stockMQ	http://redmine.msgpack.org/projects/msgpack/wiki/FormatDesign
[15:13] stockMQ	It does not look very bad
[15:14] stockMQ	comparable to VarInt in my opinion
[15:18] stockMQ	You there?
[15:18] stockMQ	I will tell you what i tried without the "Packages"
[15:19] stockMQ	I had the struct like i posted above
[15:19] stockMQ	In C++ i used the pragma directive
[15:19] stockMQ	#pragma pack(1)
[15:20] stockMQ	and i typecast the struct (char *) and send it using zeroMQ
[15:21] stockMQ	but if the size of the struct was say 92 at PUB
[15:21] stockMQ	at the receiving end it always came as 1Byte
[15:21] drbobbeaty	I'm here...
[15:22] stockMQ	I figured that ZEROMQ message..which basically is a null terminated String would be the issue and i will have to use a buffer and serializer
[15:22] drbobbeaty	My suggestion is to make a serialization/deserialization scheme - based on something. Then write a test app to prove that you can encode and decode the messages.
[15:23] drbobbeaty	Once you have that, then you'll know exactly what a message encodes to. Do a hex dump of the data before sending. Do another on the receiver. Look for differences.
[15:23] drbobbeaty	I did. It helps a lot in getting this going.
[15:24] stockMQ	ohkk
[15:24] drbobbeaty	Assume nothing. Check everything. It saves time in the end to be careful in the beginning - especially on the encoding/decoding.
[15:24] stockMQ	true
[15:24] drbobbeaty	Plus, that way you can see how long it takes to decode 1000 msgs and to what size. You can play with the codec to get it to the optimal size/speed you need.
[15:24] guido_g	Ã¸mq messages are not 0 terminated
[15:25] drbobbeaty	guido_g is totally right... it's a set of bytes. Period.
[15:25] guido_g	ack
[15:25] guido_g	except for pub/sub :)
[15:25] guido_g	there might be an issue w/ the topic
[15:26] stockMQ	ok
[15:28] drbobbeaty	In my experience with the ZMQ_PUB/ZMQ_SUB sockets and OpenPGM, the data you get from a recv() call is the "payload" -- no topic in that zmq::message_t
[15:28] guido_g	the topic is simply the start of the message
[15:29] cremes	stockMQ: here's some generic benchmarking results of some of the top serialization packages
[15:29] cremes	http://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking
[15:29] guido_g	so if one _sends_ w/ a topic, it become a problem
[15:29] cremes	it doesn't include msgpack... that's probably too new
[15:29] cremes	as always, test against your data structures to make sure the algo is fast enough
[15:30] stockMQ	Thanks cremes
[15:30] stockMQ	guido if i show a snippet of the code
[15:30] stockMQ	will it help to see if topic could be an issue
[15:30] guido_g	you should know what you send
[15:32] guido_g	https://gist.github.com/796852 <- even from python msgpack isn't that bad
[15:32] stockMQ	Well .. I just bind the publisher to pgm socket and try sending the struct typecasted to char*
[15:33] stockMQ	and on receive i expected that on typecasting back to struct it should work..which did not..I am not sure i understand the TOPIC.. i am not explicitly creating a topic
[15:33] cremes	stockMQ: i have a suggestion... make sure to abstract out the calls to encode/decode
[15:33] cremes	for testing purposes, use a simple and human-readable encoding like json
[15:33] cremes	this makes debugging very easy
[15:33] cremes	when you need to worry about perf, switch to a tighter encoding
[15:33] cremes	which should be simple if you abstracted your code proeprly
[15:34] guido_g	jepp, readable serialization format can save the day
[15:34] cremes	that is what i have done for the trading system i wrote; using json early on was a great call on my part for making debugging easy
[15:35] cremes	and unless you are counting nanoseconds for a HFT, even json is often fast enough
[15:35] stockMQ	ok..cremes .. but since json srting will act like just another string i think it will not be an issue. I have tried the sockets with test strings and everything works perfect
[15:35] cremes	stockMQ: oh, i see; you are still figuring out how to use 0mq multi-part messages
[15:36] drbobbeaty	sustrik: your change got rid of the error in the first call to connect(), but it did NOT allow messages to flow from the ZMQ_PUB server. To be fair, the ZMQ_PUB was running a slightly older version of ZMQ (still post-2.1.0). If this is an issue, let me know, but it looks like there is something else going on that's causing the messages to not be picked up by the ZMQ_SUB.
[15:36] cremes	you definitely don't want your "encoding" to include that delimter
[15:37] sustrik	drbobbeaty: ok
[15:37] stockMQ	actually i have not dabbled with multi part yet.. I figured i should be able to send a C++ struct typecasted to char* in just a single message
[15:38] drbobbeaty	sustrik: if you have any other ideas for me to try, please let me know. I'll be glad to try them.
[15:38] stockMQ	I could be wrong though..Just started using zeroMQ a couple of days back
[15:38] cremes	stockMQ: no, that isn't supported because the struct memcpy could contain the magic byte used as a message delimiter
[15:38] stockMQ	ohk
[15:38] cremes	and if you are using pub/sub, the topic byte-string is terminated with /0 too
[15:39] cremes	please read the guide and the man page on the protocol wire format
[15:40] cremes	and definitely read about multi-part messages
[15:40] stockMQ	cremes..if it is ok..can i request for an elaboration..maybe an example..if the struct is say struct ScripRecord { int token;
[15:40] stockMQ	float open,high,low,close;
[15:40] stockMQ	};
[15:40] cremes	sure...
[15:40] sustrik	drbobbeaty: does it work with release 2.1.0?
[15:40] cremes	you can't always control how the compiler will lay that struct out in memory
[15:40] stockMQ	okk
[15:41] cremes	it may add padding bytes in various places to align structures for fast memory access
[15:41] drbobbeaty	sustrik: don't know... let me double-check.
[15:41] cremes	it just isn't safe to do
[15:41] stockMQ	okk
[15:41] stockMQ	you mean a memcpy
[15:41] cremes	stockMQ: also, if you are doing any cross-platform messaging (unix to windows to mac) then memory alignments could change
[15:42] cremes	and trying to "cast" the message on another platform could result in garbage
[15:42] cremes	just don't do it; it isn't portable
[15:42] stockMQ	ok
[15:42] cremes	and i really doubt you need that much "speed" anyway; don't get too hung up on performance yet
[15:42] stockMQ	okk
[15:43] cremes	make your program work correctly first, then worry about micro-optimizations like serialization
[15:44] stockMQ	okk
[15:45] stockMQ	so i should use JSON/protobuf/thrift/msgpack/JSON+zlib/custom encoding
[15:45] cremes	for portability, yes
[15:45] stockMQ	Is that the gist guys..Isnt it
[15:46] cremes	i don't know who the gist guys are
[15:47] stockMQ	:) i just meant that 'to summarize the discussion'
[15:48] cremes	ah! ok. ;)
[15:50] stockMQ	Thank you very much bob,Cremes,seb , guido,sustrik :)
[15:52] stockMQ	Another advice i need is on the storage of tick data at SUB end
[15:53] stockMQ	i am just writing it as binary to a file.
[15:53] stockMQ	But the issue is there could be other Technical Analysis apps reading from it parallely
[15:53] cremes	stockMQ: whatever is convenient for you; you are likely going to feed intra-day ticks to consumers via an in-memory db
[15:53] cremes	and anything older (historical data) can be sourced from just about any database you like
[15:54] stockMQ	the tick data takes the following route basically Exchange->PUB->SUB
[15:54] stockMQ	now the SUBs are going to be basic laymen brokers
[15:55] stockMQ	So i cannot expect them to maintain/setup a RDBMS
[15:55] stockMQ	i was looking at options like binary flat file,HDF5,embedded DBs like BDB,Kyoto cabinet etc
[15:56] stockMQ	but the issue is with flat file and HDF5 it will still not handle concurrency
[15:57] cremes	stockMQ: i don't like that design so i have no opinion about flat file, hdf5, etc
[15:58] cremes	besides, all db access should be abstracted away as another service on your 0mq bus
[15:58] cremes	that way each of your "brokers" doesn't need to set up a DB connection
[15:58] cremes	it merely needs to know how to ask for what it wants on the 0mq bus and the service responsible for DB work can respond
[15:59] cremes	and you can scale that with push/pull or req/rep sockets that do auto load balancing for you
[15:59] cremes	stockMQ: you mentioned you were new to 0mq; that's obvious now
[15:59] cremes	make sure to read the guide... the concepts it covers will help you figure out a better design for your system
[15:59] stockMQ	yes i am new
[16:00] cremes	let 0mq help improve the design; right now you are starting out with a design and trying to wedge 0mq sockets into it
[16:00] cremes	good luck, bbiab
[16:01] stockMQ	bbiab?
[16:05] traviscline	be back in a bit
[16:05] stockMQ	:)
[16:09] stockMQ	Actually..Cremes by broker it did not mean the "ZeroMQ Broker" but the stock broker as clients who will be using the SUB app
[16:19] drbobbeaty	sustrik: I've run my tests with three versions: 2.1.0 (ref), my 2.1.0 w/ the RECOVERY_IVL patch (me), and git master (master). All are ZMQ_PUB/ZMQ_SUB socket apps. When sending from 'ref' to 'ref' - OK. When sending from 'me' to 'me' - OK. When sending from 'ref' to 'ref' - nothing received. When sending from 'me' to 'ref' - OK. So the git master (ref) is not receiving - but not generating errors, either.
[16:19] drbobbeaty	sustrik: That third test was 'master' to 'master'.
[16:27] francois	Hi all! I have multiple producers and multiple consumers for my messages, but each message must be processed only once, on any consumer. What socket types am I supposed to use? PUSH / PULL doesn't work since PUSH will only bind once.
[16:33] mikko	francois__: yes, you can either connect all subscribers to all publishers or you can use a device in the middle
[16:36] francois	I understand how the device works, but how would "connect all subscribers to all publishers" work? Each consumer / subscribers would need to know about all producers / publishers?
[16:38] mikko	francois__: for all consumers you would need to know all publishers
[16:38] francois	Makes sense, yes. Thanks for your help!
[16:38] francois	Going the device route for now.
[16:38] mikko	i think there are examples in zguide regarding routing
[17:15] francois	@mikko Thank you again very much: a device that binds to both sides is what I was looking for. I was under the false impression that the side that wanted to receive had to bind - even though the docs are correct and say either side can bind/connect.
[17:17] mikko	francois__: cool
[17:21] cremes	francois__: you might want to check this out for more ruby examples
[17:21] cremes	https://github.com/andrewvc/learn-ruby-zeromq
[17:22] cremes	btw, ruby can do all of the same stuff as all of the other bindings; there are no limitations by choosing ruby
[18:09] borior	Hi all. I'm just getting started with 0MQ, coding up a messaging channel between a piece of numerical code and a python user interface, and would appreciate some input on the design.
[18:09] borior	There are two main messaging channels: data, and control
[18:10] borior	At the moment, I envisage data being PUSH/PULL from the numerical side to the UI, and control being REQ/REP from the UI to the numerical stuff.
[18:11] borior	Does that sound sensible? And if so, how do I deal with the fact that I want to be sending many more messages down the PUSH pipe than I expect to receive from the REQ/REP control pipe from the UI? It seems like if I make a blocking call to control_sock.recv() I then can't send any messages down the data socket until a control message is sent...?
[18:26] borior	seb`: ah, thank you. I didn't know that was possible.
[18:27] borior	silly me. I should have grepped the guide for "poll". thanks for the pointer
[19:40] traviscline	seb`: details?
[19:40] codebeaker	Hi all, is "no such file or directory" a sane response to zmq_term() /
[19:41] traviscline	yes
[19:41] traviscline	ah i see
[19:41] traviscline	one sec
[19:41] traviscline	sorry about that
[19:43] traviscline	i'll push in a bit
[19:47] traviscline	seb`: pushed
[19:47] traviscline	sorry again, thanks for alerting me
[19:48] traviscline	i have a testing branch i'm working on so that sort of thing will not happen in the future
[19:51] traviscline	seb`: great, doing some profiling and working on some optimizations now
[19:52] traviscline	just reducing overhead with some type hints in cython
[20:04] mikko	codebeaker: what do you mean?
[20:05] codebeaker	mikko: (first time using zmq) I'm using the zmq_term on a context that appears to have been initialized successfully (0 return) - and I get (from zmq_strerror(errno)) "no such file or directory"
[20:05] codebeaker	hrm :\
[20:06] codebeaker	looks like my mistakeâ¦ actually, I had a !zmq_term() ââÂ so there must be an errno set already from some other piece of code :\
[20:08] mikko	codebeaker: most likely
[20:08] mikko	codebeaker: it will return != 0 on error
[20:08] codebeaker	yeah, thanks mikko
[20:08] codebeaker	I actually didn't look at the code ^ today before asking, it's a problem since a day agoâ¦ and I just needed fresh eyes I gues
[20:08] mikko	have you read zguide ?
[20:08] codebeaker	reading if(zmq_term()){ /* log error */} is still strange on my eyes (new to C)
[20:08] mikko	it's a very good resource to kick-start development
[20:09] codebeaker	yeah, the long, long, guide with the zmw helpers and so forth is exceptionally good
[20:31] pieterhintjens	codebreaker: usually it's clearer to write 'if (zmq_term () != 0) { /* log error */'
[20:31] codebeaker	pieterh: good call :)
[20:32] codebeaker	becauseâ¦ there's some inconsistency, at least between libs about how that should work, although afaik the "non zero on error" is supposed to be ubuquitos
[20:32] pieterh	every library has its own rules
[20:32] pieterh	zero = OK is the most common but it's not universal
[20:33] codebeaker	ja, good tip about making the if() more explicit though
[20:33] codebeaker	do you more experienced guys usually wrap these kind of functions?
[20:33] codebeaker	I have done, into a zmq_up() and zmq_down()
[20:33] pieterh	yes, very often
[20:33] codebeaker	ok, cool - so I'm not acting like a complete noob :)
[20:34] pieterh	perhaps not the context initialization and term, those are not used all over the place
[20:34] pieterh	certainly other functions, like sending / receiving messages
[20:35] codebeaker	sure, actually I'm over engineering slightly here - the code I'm writing is a library, so I don't want the stuff linked against it to care about the implementation, so it's hidden
[20:35] codebeaker	"up" and "down"
[20:35] codebeaker	but, I recognised from the long (and epic) guide - that the zmq_helpers are very good
[20:36] pieterh	when you can systematically replace 10 lines of code with one call, it's profitable
[20:36] pieterh	the ZFL library takes this a step further but it's also more complex to understand and use
[20:36] pieterh	https://github.com/zeromq/zfl
[20:37] codebeaker	ah, cool
[20:37] codebeaker	Ohh, pieterh I see you know a little about Ã¸mq :)
[20:37] pieterh	a little, as a user
[20:39] codebeaker	I'm a little under two weeks into learning C
[20:39] codebeaker	the hard way :)
[20:39] codebeaker	like setting out to build an assistant for my media synchronisation hobbies at home :) with ZMQ as the transport method to sync hosts
[20:40] codebeaker	^ it's been incredibly helpful, sockets are a disaster :)
[20:42] pieterh	codebreaker: you can use any language you like, C can be quite tricky to learn
[20:42] codebeaker	still lots and lots to learn
[20:42] codebeaker	ah, pieterh there's some reasons I have to use C :)
[20:42] codebeaker	but - yeah, I saw zmq has some great bindings for other languages
[20:43] pieterh	you might want to read the ZFL docs & code, it's a nice way of writing C as classes
[20:44] codebeaker	^ I saw that, nice stuff - I just had to learn some tricky stuff in C with using structs to store state between function callsâ¦ it felt pretty ugly (but it's lightweight, and works very well) - but I like your idea
[20:44] codebeaker	Im still figuring out what is sane, and insane with C compared to my background plains :)
[20:46] codebeaker	if I might ask - since I don't want to have to set up every machine like a dev serverâ¦ I should be able to find a way to load the whole of ZMQ into my app, at compile timeÂ - something like a static link, right ?
[20:47] pieterh	yes, static linking should be possible
[20:47] codebeaker	hrm, cool - learning way more than my brain can handle at the moment :)
[20:47] codebeaker	and, pieterh I just saw your nameâ¦ CEO of iMatixâ¦ nice, nice work
[20:47] pieterh	np :-)
[20:48] codebeaker	I can't believe how good zmq is, compared to the alternatives
[20:48] pieterh	there are alternatives?
[20:48] pieterh	:-)
[20:48] codebeaker	( coming from a company where the business back bone was AMQP :-\ )
[20:48] pieterh	AMQP is pretty OK compared to its own alternatives
[20:48] pieterh	but the knowledge of "how to do messaging" has moved forwards really quickly
[20:48] mikko	hmm
[20:48] codebeaker	I remember about a week after the infrastructure was "finished" Ã¸mq got some real press, and lots of us figured this might have been a smarter moveâ¦ but 4 months invested in AMQP - the business wasn't even going to give us a shot at experimenting
[20:49] pieterh	mikko: ?
[20:52] mikko	humming out loud
[20:52] mikko	looking at the dns issue
[21:15] codebeaker	pieterh: thanks for your help, and advice :)
[21:15] codebeaker	bow
[21:15] pieterh	codebeaker: np, anytime
[21:15] codebeaker	hehe, I might just take you up on thatâ¦ see you again
[22:32] mikko	sustrik: there?
[23:45] traviscline	II run into "Assertion failed: !more \|\| pipes [current] != pipe_ (fq.cpp:61)" from python if I don't try several recvs (and get EAGAINS) before switching to polling the file descriptor from a socket
[23:49] traviscline	If I don't start waiting until I've seen 10 of them I can't produce the error
[23:50] traviscline	at 3 I got the error, at 4 i've yet to, kind of unnerving