ZeroMq IRC Log

Monday June 20, 2011

[Time] Name	Message
[05:35] CIA-32	libzmq: 03Martin Sustrik 07master * rf4760f0 10/ tests/Makefile.am : Disable timeo test on MinGW ...
[06:12] CIA-32	libzmq: 03Martin Sustrik 07master * r8440de2 10/ src/socket_base.cpp : Fix minor warning in MSVC build ...
[06:18] CIA-32	libzmq: 03Fabien Ninoles 07master * r7572fee 10/ (src/mailbox.cpp src/mailbox.hpp): Optimise block/non-block switching in mailbox_t ...
[06:22] CIA-32	libzmq: 03Martin Sustrik 07master * rc80908c 10/ src/pipe.cpp : Bug fixed in pipe termination ...
[07:33] gleb	hello. there is some problem when compiling latest commit with mingw32 (cross-compiling). Mailbox.cpp won't be compiled.
[07:33] gleb	external/zmq/src/mailbox.cpp:44:24: error: sys/select.h: No such file or directory
[07:35] sustrik	gleb: let me have a look
[07:36] gleb	if change line 44 as "#if defined ZMQ_HAVE_MINGW32 #include <winsock2.h>", i got "mailbox.cpp:464: error: cannot convert 'zmq::command_t' to 'char' for argument '2' to 'int recv(SOCKET, char*, int, int)'"
[07:36] gleb	can anybody help?
[07:36] sustrik	what version are you using?
[07:36] gleb	i'm not familiar with Windows
[07:36] gleb	latest from git
[07:36] sustrik	2.1 or trunk?
[07:36] gleb	trunk
[07:36] sustrik	zeromq/libzmq?
[07:37] gleb	git://github.com/zeromq/libzmq.git
[07:37] sustrik	line 44 looks like this there #if defined ZMQ_HAVE_WINDOWS
[07:37] sustrik	are you sure you have the latest version?
[07:37] gleb	previos commit worked ok
[07:39] gleb	commit 00dc0245e6aacbff247c84ac8480d3ddcabacd5a
[07:39] sustrik	hm
[07:39] sustrik	there's no such commit there
[07:40] sustrik	ah
[07:40] sustrik	found it
[07:40] sustrik	there are 7 new commits since
[07:40] sustrik	they should fix the mingw issue
[07:41] gleb	i'm trying now...
[07:41] gleb	thank you, seems to be compilable
[07:46] gleb	thank you, seems to be working
[07:48] gleb	btw, i'm using "scons" to build it, seems to be a bit more comfortable. if any interested, there are sconstruct/sconscript scripts for (...zmq)
[07:55] sustrik	gleb: send them to the mailing list
[07:55] sustrik	there may be people interested in it
[07:57] gleb	ok
[09:17] CIA-32	libzmq: 03Martin Sustrik 07master * rada5d42 10/ tests/testutil.hpp : Basic tests now test multi-part messages instead of single-part ...
[14:24] private_meta	can I force zmq::poll to return before a timeout occured?
[14:26] cremes	private_meta: no
[14:30] private_meta	not even by terminating the context?
[14:36] Seta00	private_meta, pull the plug xD
[14:37] private_meta	Seta00: Which plug, network or power?
[14:37] Seta00	power
[14:37] mikko	private_meta: terminating the context probably sends eintr
[14:38] private_meta	Seta00: in that case, it wouldn't really return :)
[14:38] Seta00	private_meta, are you using the C++ binding?
[14:46] Seta00	private_meta, do you have to manually call socket->operator void*() when building zmq::pollitems lists?
[14:49] private_meta	Seta00: yes, I am using the C++ bindings, but I don't quite know what you mean with the operator
[14:49] Seta00	let me create a gist
[14:53] Seta00	private_meta, https://gist.github.com/1035756
[14:54] private_meta	Seta00: in short, no
[14:55] private_meta	Seta00: I use it like this: zmq::socket_t m_p_worker; zmq::pollitem_t items[] = { { m_p_worker, 0, ZMQ_POLLIN, 0 } }; zmq::poll(items, 1, m_heartbeat * 1000);
[14:56] Seta00	oh yeah, I used \|->\| but it should be \|.\|, those are socket objects
[14:57] private_meta	hmm
[14:57] Seta00	weird, what compiler are you using?
[14:57] private_meta	gcc44
[14:57] private_meta	Are you using a MS compiler?
[14:58] Seta00	no, gcc too
[14:58] private_meta	version?
[14:58] Seta00	can't remember if it was 4.4 or 4.6 when I got that error
[14:58] Seta00	I'll test when I get home
[14:58] Seta00	in 10 minutes :P
[15:12] private_meta	Seta00: https://gist.github.com/8aa12880a7b7c272a07a this should work perfectly well
[16:16] Skaag	is there some set of tools that use zmq to send pipes over the network, for remote analysis of data?
[17:50] ntelford	is the zmq_poll stuff essentially select() for ZMQ sockets?
[17:58] michelp	ntelford, yes, although the underlying implementation is likely to be more efficient than select()
[17:59] michelp	depending on the arch
[17:59] private_meta	michelp: what's factor binding?
[18:00] michelp	private_meta, factor is a programming language. sort of a more modern version of forth
[18:00] private_meta	ah
[18:01] michelp	was reading about it's FFI interface and was wondering maybe if anyone else had considered making bindings for 0mq to it
[18:01] private_meta	yeah, i think I've seen it on the wikipedia hello world sample page and looked at it's page a bit, was a bit confusing
[18:01] michelp	before i experiment with it myself ;)
[18:01] michelp	yeah forth dialects are definitely a mind-bender
[18:01] michelp	for such a simple language it can be very hard to read due to your ability to extend any part of the syntax at any time
[18:06] private_meta	hmm... factor wiki page is down
[18:06] private_meta	great for learning a language
[18:08] michelp	hmm yeah that sucks
[18:12] private_meta	Seta00: those are long 10 Minutes
[18:12] Seta00	private_meta, got home and found out my HDD broke
[18:13] Seta00	writing testcases lost priority
[18:13] Seta00	:P
[18:13] private_meta	fun
[18:14] private_meta	Had a hard drive with a broken ide adapter once
[18:14] private_meta	Went into constant overdrive, every system I plugged it into grinded to a halt, no matter which OS, no matter which connection (USB, IDE)
[18:14] private_meta	THAT was fun
[18:16] private_meta	I mean, not that the standard Oh-I-Just-Lost-All-Of-My-Data "broken" wouldn't be nice too, but you need some variety once in a while
[18:17] Seta00	private_meta, yeah, mine just goes 100% speed and stays like that
[18:18] Seta00	private_meta, anyway, works on gcc 4.6
[18:18] Seta00	I think I was using MinGW when it happened
[18:18] Seta00	:P
[18:18] Seta00	if it happens to me again I'll save a testcase
[18:23] michelp	private_meta, this looks like a good but slightly academic intro: http://factorcode.org/littledan/dls.pdf
[18:29] private_meta	reminds me, I'd like my Firefox to get pdf working again
[19:36] sustrik	private_meta: zmq_term() will cause zmq_poll() to return immediately with ETERM error
[19:36] sustrik	ntelford: zmq_poll is select for 0mq sockets and standard sockets as well
[19:37] private_meta	sustrik: thanks
[19:37] sustrik	np
[21:33] jonnydee	hi, I'm new to zeromq, but I've read a tutorial and programming guide. But I wonder if it is possible to create a peer-to-peer network between nodes....?
[21:35] private_meta	jonnydee: equal communication between nodes?
[21:35] jonnydee	Do you have any idea how this can be done? AFAIK there is alwas a server to which I need to connect to. But if this server node crashes all clients will fail to communicate with each other
[21:36] private_meta	well, technically, if you want p2p, you have to create dedicated sockets for each node, but that means, if it's tcp, it's limited to the amount of ports you have
[21:36] jonnydee	private_meta: What I need is an infra structure where there is no central server component.
[21:37] private_meta	I don't know how resource intensive this is, but i think you need to create n-1 socket objects if you have n nodes
[21:37] jonnydee	I've read about pgm multicast... is it something that might go in my direction?
[21:37] jonnydee	private_meta: yes, I feared that....
[21:38] private_meta	sec, I'll look up the pgm multicast
[21:38] jonnydee	cool, thank you :)
[21:45] private_meta	jonnydee: still there?
[21:50] private_meta	jonnydee_: back again?
[21:50] jonnydee	sorry, I got kicked out...
[21:50] private_meta	what was the lst line you got from me?
[21:51] jonnydee	private_meta: I guess nothing, the last line was that you will have a look...
[21:51] private_meta	23:40 < private_meta> well, ok, scratch the thing i said about ports, you only need 1, but you still need n-1 objects for my idea
[21:51] private_meta	23:41 < private_meta> I'm not entirely sure, so I might be wrong, but it appears to me that pgm is still meant for a client/worker structure with a server
[21:52] private_meta	23:42 < private_meta> well
[21:52] private_meta	23:42 < private_meta> it depends on what you want to do
[21:52] private_meta	23:42 < private_meta> jonnydee: do you want to send the same message to ALL clients ALL the time?
[21:52] private_meta	As far as I gather, pg multicast is for addressing multiple clients with the same message. I somehow doubt that this is what you want/need
[21:53] jonnydee	private_meta: yes, every client should notify all other clients in the network about their actions/events in order to keep them in sync
[21:53] private_meta	Yes, but there are other problems
[21:53] private_meta	with pgm, as far as I see, you specify the receivers at initialization of the socket_t object
[21:54] private_meta	well, depending on your language binding, the socket
[21:54] jonnydee	ok, so this means I need to know the list of receivers in advance, right?
[21:54] private_meta	yes, or you have to reconnect ALL clients whenever you add one
[21:54] jonnydee	C++ would be my language binding
[21:54] private_meta	ok
[21:55] private_meta	Fixing the addresses of all clients at initializtion makes it quite rigid
[21:55] jonnydee	hmmm....ok...I see
[21:56] private_meta	Also, I don't quite know how pgm many-to-many bidirectional communications work exatly
[21:56] private_meta	*exactl
[21:56] private_meta	+y
[21:56] private_meta	damn this keyboard
[21:56] jonnydee	yes, you're right. I hoped I would able to get a solution where arbitrary many clients can join/leave the network
[21:56] jonnydee	any time
[21:56] private_meta	Well, for that you have to reconnect every time a client joins, which might create some inconsistencies, not sure about that
[21:57] private_meta	if you don't quite care about TRUE p2p, you can make redundant servers, but that's not quite foolproof
[21:58] private_meta	so I guess you can either do pgm or make every client a server/worker and have sockets to each other client
[21:58] jonnydee	ok, I see. I'll have to think a bit more about a solution.
[21:59] jonnydee	yes, I'll try to figure out if haveing a connection to all other clients might be a feasible approach for us...
[21:59] private_meta	Does it have to be openly scalable to, like thousands of clients?
[21:59] private_meta	tens of thsds
[21:59] private_meta	or some such
[22:00] jonnydee	well, maybe not thousands, but up to 1000 maybe yes...
[22:01] jonnydee	do you think this would scale well for about 1000 clients?
[22:01] private_meta	I'm still thinking if there's another way... feel like I'm missing something
[22:02] jonnydee	Another idea I had was to maybe cluster the network into smaller groups of clients where in each group there is a master acting as a server component. And the groups communicate
[22:03] jonnydee	with each other over the "master" clients...
[22:03] private_meta	That would be an option, but you would still have a failure source at those master nodes
[22:03] jonnydee	but, obviously, it would make implementation more complex
[22:04] jonnydee	yes, you're right. that's the thing which I don't like...
[22:04] private_meta	Ã®n this case, you'd arrive at http://zguide.zeromq.org/page:all#Scaling-to-Multiple-Clusters
[22:04] jonnydee	at least, not all groups would loose communication....
[22:04] private_meta	in a way
[22:05] jonnydee	private_meta: thanx, yes, this seems to be exactly my idea...
[22:06] private_meta	Still it would not be p2p
[22:06] jonnydee	Yes, it's not p2p.... it's sort of group-to-group...
[22:07] private_meta	the basic problem is that all guide examples work with brokers. more or less dedicated brokers
[22:07] jonnydee	But if nothing helps I think I will have to go that way.
[22:07] jonnydee	yes, exactly. that's why I tried my luck here
[22:07] private_meta	Well
[22:08] jonnydee	but your link will help me anyway. Now I can read about how this solution would look like using zeromq, which will help me in my decisions
[22:09] private_meta	If you don't want dedicated brokers, no matter in which dimension, I guess you are stuck with the initial options
[22:09] private_meta	pgm with fixed list, pgm with restart on client change (and possible loss of data) or dedicated sockets for each connection
[22:09] jonnydee	yes, I fear you're right
[22:10] private_meta	If you go the p2p way, I'd recommend testing about consistency of messages when restarting connections, and on the other hand you can test scalability
[22:10] private_meta	(with the socket approach)
[22:10] private_meta	create 10 sockets, 20, 30, measure performance
[22:11] jonnydee	that's a good hint. This will actually let me gather experience with zeromq's api, too. I thin I'll implement a p2p prototype an see what I get....
[22:12] private_meta	If you implement it well, maybe you can feed it back to zmq ;)
[22:13] private_meta	then again, I may just be overlooking something, and someone who had more experience might know a solution
[22:13] private_meta	an existing one
[22:13] jonnydee	I don't know if I can tell you guys something new, but I will consider this option if I think it contributes something... ;)
[22:14] jonnydee	private_meta: thank you very very much for your help and your time :)
[22:15] private_meta	jonnydee_: you're welcome. Still, maybe come again tomorrow, midday perhaps, maybe someone is here who might have another solution :)
[22:16] jonnydee	ok, I'll try. So cu, and good night. Bye bye :)
[22:16] private_meta	good night
[22:58] dvitek	is it possible to use 0mq if having it create background threads is out of the question?