Monday June 20, 2011

[Time] NameMessage
[05:35] CIA-32 libzmq: 03Martin Sustrik 07master * rf4760f0 10/ tests/ : Disable timeo test on MinGW ...
[06:12] CIA-32 libzmq: 03Martin Sustrik 07master * r8440de2 10/ src/socket_base.cpp : Fix minor warning in MSVC build ...
[06:18] CIA-32 libzmq: 03Fabien Ninoles 07master * r7572fee 10/ (src/mailbox.cpp src/mailbox.hpp): Optimise block/non-block switching in mailbox_t ...
[06:22] CIA-32 libzmq: 03Martin Sustrik 07master * rc80908c 10/ src/pipe.cpp : Bug fixed in pipe termination ...
[07:33] gleb hello. there is some problem when compiling latest commit with mingw32 (cross-compiling). Mailbox.cpp won't be compiled.
[07:33] gleb external/zmq/src/mailbox.cpp:44:24: error: sys/select.h: No such file or directory
[07:35] sustrik gleb: let me have a look
[07:36] gleb if change line 44 as "#if defined ZMQ_HAVE_MINGW32 #include <winsock2.h>", i got "mailbox.cpp:464: error: cannot convert 'zmq::command_t*' to 'char*' for argument '2' to 'int recv(SOCKET, char*, int, int)'"
[07:36] gleb can anybody help?
[07:36] sustrik what version are you using?
[07:36] gleb i'm not familiar with Windows
[07:36] gleb latest from git
[07:36] sustrik 2.1 or trunk?
[07:36] gleb trunk
[07:36] sustrik zeromq/libzmq?
[07:37] gleb git://
[07:37] sustrik line 44 looks like this there #if defined ZMQ_HAVE_WINDOWS
[07:37] sustrik are you sure you have the latest version?
[07:37] gleb previos commit worked ok
[07:39] gleb commit 00dc0245e6aacbff247c84ac8480d3ddcabacd5a
[07:39] sustrik hm
[07:39] sustrik there's no such commit there
[07:40] sustrik ah
[07:40] sustrik found it
[07:40] sustrik there are 7 new commits since
[07:40] sustrik they should fix the mingw issue
[07:41] gleb i'm trying now...
[07:41] gleb thank you, seems to be compilable
[07:46] gleb thank you, seems to be working
[07:48] gleb btw, i'm using "scons" to build it, seems to be a bit more comfortable. if any interested, there are sconstruct/sconscript scripts for (...zmq)
[07:55] sustrik gleb: send them to the mailing list
[07:55] sustrik there may be people interested in it
[07:57] gleb ok
[09:17] CIA-32 libzmq: 03Martin Sustrik 07master * rada5d42 10/ tests/testutil.hpp : Basic tests now test multi-part messages instead of single-part ...
[14:24] private_meta can I force zmq::poll to return before a timeout occured?
[14:26] cremes private_meta: no
[14:30] private_meta not even by terminating the context?
[14:36] Seta00 private_meta, pull the plug xD
[14:37] private_meta Seta00: Which plug, network or power?
[14:37] Seta00 power
[14:37] mikko private_meta: terminating the context probably sends eintr
[14:38] private_meta Seta00: in that case, it wouldn't really return :)
[14:38] Seta00 private_meta, are you using the C++ binding?
[14:46] Seta00 private_meta, do you have to manually call socket->operator void*() when building zmq::pollitems lists?
[14:49] private_meta Seta00: yes, I am using the C++ bindings, but I don't quite know what you mean with the operator
[14:49] Seta00 let me create a gist
[14:53] Seta00 private_meta,
[14:54] private_meta Seta00: in short, no
[14:55] private_meta Seta00: I use it like this: zmq::socket_t *m_p_worker; zmq::pollitem_t items[] = { { *m_p_worker, 0, ZMQ_POLLIN, 0 } }; zmq::poll(items, 1, m_heartbeat * 1000);
[14:56] Seta00 oh yeah, I used |->| but it should be |.|, those are socket objects
[14:57] private_meta hmm
[14:57] Seta00 weird, what compiler are you using?
[14:57] private_meta gcc44
[14:57] private_meta Are you using a MS compiler?
[14:58] Seta00 no, gcc too
[14:58] private_meta version?
[14:58] Seta00 can't remember if it was 4.4 or 4.6 when I got that error
[14:58] Seta00 I'll test when I get home
[14:58] Seta00 in 10 minutes :P
[15:12] private_meta Seta00: this should work perfectly well
[16:16] Skaag is there some set of tools that use zmq to send pipes over the network, for remote analysis of data?
[17:50] ntelford is the zmq_poll stuff essentially select() for ZMQ sockets?
[17:58] michelp ntelford, yes, although the underlying implementation is likely to be more efficient than select()
[17:59] michelp depending on the arch
[17:59] private_meta michelp: what's factor binding?
[18:00] michelp private_meta, factor is a programming language. sort of a more modern version of forth
[18:00] private_meta ah
[18:01] michelp was reading about it's FFI interface and was wondering maybe if anyone else had considered making bindings for 0mq to it
[18:01] private_meta yeah, i think I've seen it on the wikipedia hello world sample page and looked at it's page a bit, was a bit confusing
[18:01] michelp before i experiment with it myself ;)
[18:01] michelp yeah forth dialects are definitely a mind-bender
[18:01] michelp for such a simple language it can be very hard to read due to your ability to extend any part of the syntax at any time
[18:06] private_meta hmm... factor wiki page is down
[18:06] private_meta great for learning a language
[18:08] michelp hmm yeah that sucks
[18:12] private_meta Seta00: those are long 10 Minutes
[18:12] Seta00 private_meta, got home and found out my HDD broke
[18:13] Seta00 writing testcases lost priority
[18:13] Seta00 :P
[18:13] private_meta fun
[18:14] private_meta Had a hard drive with a broken ide adapter once
[18:14] private_meta Went into constant overdrive, every system I plugged it into grinded to a halt, no matter which OS, no matter which connection (USB, IDE)
[18:14] private_meta THAT was fun
[18:16] private_meta I mean, not that the standard Oh-I-Just-Lost-All-Of-My-Data "broken" wouldn't be nice too, but you need some variety once in a while
[18:17] Seta00 private_meta, yeah, mine just goes 100% speed and stays like that
[18:18] Seta00 private_meta, anyway, works on gcc 4.6
[18:18] Seta00 I think I was using MinGW when it happened
[18:18] Seta00 :P
[18:18] Seta00 if it happens to me again I'll save a testcase
[18:23] michelp private_meta, this looks like a good but slightly academic intro:
[18:29] private_meta reminds me, I'd like my Firefox to get pdf working again
[19:36] sustrik private_meta: zmq_term() will cause zmq_poll() to return immediately with ETERM error
[19:36] sustrik ntelford: zmq_poll is select for 0mq sockets and standard sockets as well
[19:37] private_meta sustrik: thanks
[19:37] sustrik np
[21:33] jonnydee hi, I'm new to zeromq, but I've read a tutorial and programming guide. But I wonder if it is possible to create a peer-to-peer network between nodes....?
[21:35] private_meta jonnydee: equal communication between nodes?
[21:35] jonnydee Do you have any idea how this can be done? AFAIK there is alwas a server to which I need to connect to. But if this server node crashes all clients will fail to communicate with each other
[21:36] private_meta well, technically, if you want p2p, you have to create dedicated sockets for each node, but that means, if it's tcp, it's limited to the amount of ports you have
[21:36] jonnydee private_meta: What I need is an infra structure where there is no central server component.
[21:37] private_meta I don't know how resource intensive this is, but i think you need to create n-1 socket objects if you have n nodes
[21:37] jonnydee I've read about pgm multicast... is it something that might go in my direction?
[21:37] jonnydee private_meta: yes, I feared that....
[21:38] private_meta sec, I'll look up the pgm multicast
[21:38] jonnydee cool, thank you :)
[21:45] private_meta jonnydee: still there?
[21:50] private_meta jonnydee_: back again?
[21:50] jonnydee sorry, I got kicked out...
[21:50] private_meta what was the lst line you got from me?
[21:51] jonnydee private_meta: I guess nothing, the last line was that you will have a look...
[21:51] private_meta 23:40 < private_meta> well, ok, scratch the thing i said about ports, you only need 1, but you still need n-1 objects for my idea
[21:51] private_meta 23:41 < private_meta> I'm not entirely sure, so I might be wrong, but it appears to me that pgm is still meant for a client/worker structure with a server
[21:52] private_meta 23:42 < private_meta> well
[21:52] private_meta 23:42 < private_meta> it depends on what you want to do
[21:52] private_meta 23:42 < private_meta> jonnydee: do you want to send the same message to ALL clients ALL the time?
[21:52] private_meta As far as I gather, pg multicast is for addressing multiple clients with the same message. I somehow doubt that this is what you want/need
[21:53] jonnydee private_meta: yes, every client should notify all other clients in the network about their actions/events in order to keep them in sync
[21:53] private_meta Yes, but there are other problems
[21:53] private_meta with pgm, as far as I see, you specify the receivers at initialization of the socket_t object
[21:54] private_meta well, depending on your language binding, the socket
[21:54] jonnydee ok, so this means I need to know the list of receivers in advance, right?
[21:54] private_meta yes, or you have to reconnect ALL clients whenever you add one
[21:54] jonnydee C++ would be my language binding
[21:54] private_meta ok
[21:55] private_meta Fixing the addresses of all clients at initializtion makes it quite rigid
[21:55] jonnydee hmmm....ok...I see
[21:56] private_meta Also, I don't quite know how pgm many-to-many bidirectional communications work exatly
[21:56] private_meta *exactl
[21:56] private_meta +y
[21:56] private_meta damn this keyboard
[21:56] jonnydee yes, you're right. I hoped I would able to get a solution where arbitrary many clients can join/leave the network
[21:56] jonnydee any time
[21:56] private_meta Well, for that you have to reconnect every time a client joins, which might create some inconsistencies, not sure about that
[21:57] private_meta if you don't quite care about TRUE p2p, you can make redundant servers, but that's not quite foolproof
[21:58] private_meta so I guess you can either do pgm or make every client a server/worker and have sockets to each other client
[21:58] jonnydee ok, I see. I'll have to think a bit more about a solution.
[21:59] jonnydee yes, I'll try to figure out if haveing a connection to all other clients might be a feasible approach for us...
[21:59] private_meta Does it have to be openly scalable to, like thousands of clients?
[21:59] private_meta tens of thsds
[21:59] private_meta or some such
[22:00] jonnydee well, maybe not thousands, but up to 1000 maybe yes...
[22:01] jonnydee do you think this would scale well for about 1000 clients?
[22:01] private_meta I'm still thinking if there's another way... feel like I'm missing something
[22:02] jonnydee Another idea I had was to maybe cluster the network into smaller groups of clients where in each group there is a master acting as a server component. And the groups communicate
[22:03] jonnydee with each other over the "master" clients...
[22:03] private_meta That would be an option, but you would still have a failure source at those master nodes
[22:03] jonnydee but, obviously, it would make implementation more complex
[22:04] jonnydee yes, you're right. that's the thing which I don't like...
[22:04] private_meta în this case, you'd arrive at
[22:04] jonnydee at least, not all groups would loose communication....
[22:04] private_meta in a way
[22:05] jonnydee private_meta: thanx, yes, this seems to be exactly my idea...
[22:06] private_meta Still it would not be p2p
[22:06] jonnydee Yes, it's not p2p.... it's sort of group-to-group...
[22:07] private_meta the basic problem is that all guide examples work with brokers. more or less dedicated brokers
[22:07] jonnydee But if nothing helps I think I will have to go that way.
[22:07] jonnydee yes, exactly. that's why I tried my luck here
[22:07] private_meta Well
[22:08] jonnydee but your link will help me anyway. Now I can read about how this solution would look like using zeromq, which will help me in my decisions
[22:09] private_meta If you don't want dedicated brokers, no matter in which dimension, I guess you are stuck with the initial options
[22:09] private_meta pgm with fixed list, pgm with restart on client change (and possible loss of data) or dedicated sockets for each connection
[22:09] jonnydee yes, I fear you're right
[22:10] private_meta If you go the p2p way, I'd recommend testing about consistency of messages when restarting connections, and on the other hand you can test scalability
[22:10] private_meta (with the socket approach)
[22:10] private_meta create 10 sockets, 20, 30, measure performance
[22:11] jonnydee that's a good hint. This will actually let me gather experience with zeromq's api, too. I thin I'll implement a p2p prototype an see what I get....
[22:12] private_meta If you implement it well, maybe you can feed it back to zmq ;)
[22:13] private_meta then again, I may just be overlooking something, and someone who had more experience might know a solution
[22:13] private_meta an existing one
[22:13] jonnydee I don't know if I can tell you guys something new, but I will consider this option if I think it contributes something... ;)
[22:14] jonnydee private_meta: thank you very very much for your help and your time :)
[22:15] private_meta jonnydee_: you're welcome. Still, maybe come again tomorrow, midday perhaps, maybe someone is here who might have another solution :)
[22:16] jonnydee ok, I'll try. So cu, and good night. Bye bye :)
[22:16] private_meta good night
[22:58] dvitek is it possible to use 0mq if having it create background threads is out of the question?