| [Time] Name | Message |
|
[04:59] Dantman
|
I'm writing something with Java and jzmq. I'm doing a NIO server handling multiple client sockets and using NIO's Selector API. From there, I want to send and receive messages over ZMQ to other things. Is jzmq compatible with NIO's Selector API? Can I register a ZMQ socket with a Selector and incorporate waiting for incoming messages on the ZMQ socket into the .select();, or do I have to create a separate thread just to efficiently send and receive stuff
|
|
[04:59] Dantman
|
over ZMQ.
|
|
[04:59] guido_g
|
yes
|
|
[04:59] guido_g
|
see getsockopt and ZMQ_FD
|
|
[05:00] guido_g
|
you get a fd you can use w/ poll/nio or whatever
|
|
[05:01] Dantman
|
Java NIO... I mean, is it part of the JZMQ java api...
|
|
[05:01] Dantman
|
There's no API docs I can see to check
|
|
[05:02] guido_g
|
http://api.zeromq.org/2-1:zmq-getsockopt
|
|
[05:05] jer
|
only a couple days into 0mq, but it is one of the coolest libraries i've used. to play, i rewrote some code, and managed to quite literally, throw away several hundred lines of code (and the program was already just over a thousand lines of code... 40%, gone.)
|
|
[05:07] Dantman
|
I get that the C api has the ability to return a descriptor that can be polled or selected, so it's possible for jzmq to do it. But I need to know if jzmq has implemented that into a java.nio.channels.SelectableChannel that I can use in conjunction with a java.nio.channels.Selector.
|
|
[05:08] guido_g
|
if you can't figure it out, ask the author of the binding
|
|
[07:00] Steve-o
|
goijfgdgdfg.
|
|
[07:02] guido_g
|
Steve-o: get more coffee :)
|
|
[07:02] Steve-o
|
That's what I"m thinking of MSVC 100
|
|
[07:02] Steve-o
|
They've introduced a tonne of socket option definitions which aren't used anywhere
|
|
[07:03] Steve-o
|
And still a typo on the current page listing IPPROTO socket options
|
|
[07:04] guido_g
|
poor Steve-o
|
|
[07:04] Steve-o
|
oh well, done now, release early, release often.
|
|
[07:04] Steve-o
|
Looks like Martin & Pieter are having similar fun with recent 0mq builds too though :D
|
|
[07:07] Steve-o
|
aren't there a few low latency events in NYC this week?
|
|
[07:08] Steve-o
|
"The Business and Technology of Low-Latency Trading" at Crowne Plaza
|
|
[07:09] Steve-o
|
looks like I miss everything by a week again
|
|
[08:28] mikko
|
Steve-o: hi
|
|
[08:28] Steve-o
|
yo
|
|
[08:28] mikko
|
how was your travelling?
|
|
[08:29] Steve-o
|
not stopped, back to nyc next monday
|
|
[08:29] mikko
|
did i send you an email about mac os x build failure --with-pgm?
|
|
[08:30] mikko
|
i think i was supposed to send but i dont know if i ever did
|
|
[08:30] Steve-o
|
I have something, one moment
|
|
[08:30] mikko
|
i was in lisbon for holidays
|
|
[08:30] mikko
|
lovely city
|
|
[08:30] Steve-o
|
https://github.com/zeromq/libzmq/issues/193 fixed already
|
|
[08:30] Steve-o
|
Lisbon? You like it? It's bit of a dump
|
|
[08:30] mikko
|
ah, nice
|
|
[08:30] mikko
|
yeah, it's not nice and shiny
|
|
[08:31] mikko
|
but a lot of character
|
|
[08:31] Steve-o
|
Zurich is funny, everyone is so old
|
|
[08:31] mikko
|
i shot five rolls of film there
|
|
[08:31] Steve-o
|
we had a nice apartment over looking the lake though, v. nice
|
|
[08:31] mikko
|
i've never been to zurich
|
|
[08:32] Steve-o
|
nice chocolate :-)
|
|
[08:32] mikko
|
i would love to go to south of france at some point
|
|
[08:32] Steve-o
|
st tropez?
|
|
[08:32] Steve-o
|
or just abouts
|
|
[08:32] mikko
|
just anywhere there
|
|
[08:33] Steve-o
|
I see a lot of Brits on HGTV moving to South of France
|
|
[08:33] Steve-o
|
good weather and large residences
|
|
[08:34] Steve-o
|
so I'll be NYC for near 3 months
|
|
[08:34] Steve-o
|
waiting for Green Card interview
|
|
[08:37] Steve-o
|
I've switched my lab off, so that might be entertaining for future updates.
|
|
[08:38] mikko
|
hehe, where is the lab located at?
|
|
[08:38] Steve-o
|
in my office in HK
|
|
[08:38] mikko
|
if we wanted to add PGM testing to daily builds what would that need?
|
|
[08:38] Steve-o
|
although there is a OS X server still running with the source tree on
|
|
[08:38] mikko
|
what kind of gear?
|
|
[08:39] Steve-o
|
bare minimum normally for anything but performance testing
|
|
[08:39] Steve-o
|
you can get away with really crappy gear
|
|
[08:40] Steve-o
|
after all White Barn made PGM in the days of Sparc 1 & 2's
|
|
[08:40] mikko
|
i'm just looking for 'does it work' kind of testing
|
|
[08:40] mikko
|
currently i got two HP DL380s running VMWare ESXi with some amount of virtual machines
|
|
[08:43] Steve-o
|
the unit tests are trivial to run on the command line for any hardware
|
|
[08:43] Steve-o
|
the network tests are not so straight forward
|
|
[08:44] Steve-o
|
I ideally want to rewrite them in 0MQ + Python or something higher level
|
|
[08:44] Steve-o
|
make a great GSoC project :/
|
|
[08:47] Steve-o
|
Ideally you could say 0MQ should be able to mock the PGM socket interface
|
|
[08:57] mikko
|
hmm
|
|
[08:57] mikko
|
do the network tests work with vmware virtual networking?
|
|
[08:59] Steve-o
|
with ESX or VirtualBox would work
|
|
[09:00] Steve-o
|
the older VMware VM's don't support multicast NICs
|
|
[09:01] Steve-o
|
just going to pop out to find an IP-KVM device, later
|
|
[09:02] mikko
|
later
|
|
[11:58] th
|
pieterh: could you give me a pointer where to find the "patch for 191" which applies to 2.1? i'm adapting my testcase to get more information then.
|
|
[11:58] pieterh
|
th: hi
|
|
[11:58] pieterh
|
sustrik posted the patch to the mailing list
|
|
[11:59] pieterh
|
i've also attached it to the issue in github
|
|
[11:59] pieterh
|
in this gist repo: https://gist.github.com/946025
|
|
[12:00] pieterh
|
however... it doesn't apply to 2.1 cleanly...
|
|
[12:00] pieterh
|
give me a second and I'll make a patch for 2.1 and include it in that gist
|
|
[12:00] th
|
pieterh: great thanks!
|
|
[12:01] th
|
pieterh: i did not reduce it to a C testcase (yet) because i could imagine that the issue is within the C++ "bindings"
|
|
[12:01] th
|
some destructing done in incorrect order or something...
|
|
[12:02] pieterh
|
th: could be, esp. since the error always hits the same place afaics
|
|
[12:02] pieterh
|
i've added that patch to the gist repo
|
|
[12:03] th
|
pieterh: the main difference with your patch is that the CLIENT dies while in my testcase the server dies
|
|
[12:04] th
|
2-1/issue191.patch got it
|
|
[12:05] th
|
pieterh: is it a good idea to use 2.1.6 for this?
|
|
[12:05] pieterh
|
th: should not make any difference
|
|
[12:05] th
|
ok
|
|
[12:08] th
|
pieterh: i can not confirm any change with that patch. i dont get the error you get, it's still the server failing because of intermixed message.
|
|
[12:09] pieterh
|
right...
|
|
[12:09] pieterh
|
anyhow the patch didn't fix issue 191, it caused 0MQ to die nastily
|
|
[12:09] th
|
pieterh: which 2.1 revision did you use?
|
|
[12:10] th
|
pieterh: it does not die here
|
|
[12:10] th
|
pieterh: at least not nastily
|
|
[12:10] pieterh
|
indeed, neither with me, then it's probably a different issue than 191
|
|
[12:10] pieterh
|
i'd suggest the best step is to make a C test case that reproduces the problem on 2.1
|
|
[12:10] th
|
ok
|
|
[12:10] pieterh
|
the revision is not going to be significant, anything recent
|
|
[12:10] th
|
and testing 3.0 i assume
|
|
[12:10] pieterh
|
and then, a second test case for 3.0 (it'll be very similar)
|
|
[12:11] pieterh
|
don't use zmq_device, that's gone in 3.0...
|
|
[12:11] th
|
pieterh: i didn't
|
|
[12:11] th
|
pieterh: must be indirectly
|
|
[12:12] pieterh
|
ah, perhaps a side effect of using the C++ binding
|
|
[12:13] pieterh
|
it was a compile error, not a link error
|
|
[12:13] th
|
pieterh: i only use {context,socket,message}_t and bind,connect,recv,send,{get,set}sockopt
|
|
[12:13] th
|
ahh ok
|
|
[12:14] th
|
removing C++ first then, i'd say. to know where it comes from.
|
|
[12:15] pieterh
|
yes
|
|
[12:50] th
|
reference manual says about zmq_msg_init_data: "ÃMQ shall take ownership of the supplied buffer", is that only true when a deallocation function pointer is given?
|
|
[12:51] mato
|
th: you mean "... is not given" ?
|
|
[12:51] pieterh
|
mato: when give, I think...
|
|
[12:51] pieterh
|
*given
|
|
[12:51] th
|
there are examples in the guide passing a constant static string, and NULL,NULL for ffn,hint
|
|
[12:51] th
|
it makes no sense to giveownership to a string like that
|
|
[12:51] mato
|
it does what you think it does :-)
|
|
[12:52] th
|
mato: so it does not call any free() unless i pass a function pointer to a free()?
|
|
[12:52] mato
|
th: actually good question
|
|
[12:52] pieterh
|
th: those examples should absolutely not be using the _data method, I'm changing that
|
|
[12:52] th
|
????????zmq_msg_init_data?(&reply,?"World",?5,?NULL,?NULL);
|
|
[12:52] th
|
talking od that code
|
|
[12:52] pieterh
|
yes, hwserver & hwclient
|
|
[12:52] th
|
sorry for the broken paste
|
|
[12:53] mato
|
th: if no ffn is passed, none is called
|
|
[12:53] pieterh
|
mato: problem is that statement in the man page, "shall take ownership..."
|
|
[12:53] th
|
pieterh: i'm looking fo replacemtn of `zmq::message_t m(len); memcpy((void *) m.data(), buffer, len);`
|
|
[12:53] mato
|
pieterh: yes, it's wrong
|
|
[12:54] th
|
mato: then i think the reference manual should be changed to make that clear. it says "shall take ownership"
|
|
[12:54] pieterh
|
th: in C, zmq_msg_init_size() and then memcpy into zmq_msg_data()
|
|
[12:54] th
|
perhaps it should more hint to the user to keep that buffer valid until message's EOL
|
|
[12:55] mato
|
yes
|
|
[12:55] mato
|
if you don't supply a ffn then the buffer must stay around forever
|
|
[12:55] mato
|
or at least until context deinit anyway
|
|
[12:55] th
|
mato: _context_ deinit? not message deinit?
|
|
[12:56] mato
|
th: technically message deinit, but due to internal refcounting that's probably not a 100% safe approach
|
|
[12:56] th
|
(which should be msg_close, i assume)
|
|
[12:56] mato
|
yes
|
|
[12:56] th
|
pieterh: is it wrong in the examples because it is easily used incorrectly?
|
|
[12:57] th
|
pieterh: or is there another reason?
|
|
[12:57] pieterh
|
th: the init_data method is rather poorly named
|
|
[12:57] pieterh
|
it should be called zmq_msg_zero_copy_do_not_use_unless_you_know_what_its_for()
|
|
[12:58] mato
|
:-)
|
|
[12:58] th
|
pieterh: should it be named something like zmq_msg_set_data_reference?
|
|
[12:58] pieterh
|
and my error in using that in hello world methods
|
|
[12:58] th
|
ahh ok
|
|
[12:58] pieterh
|
highly misleading, but I also got caught by the name originally
|
|
[13:10] Steve-o
|
pieterh: updated PGM github issues in libzmq/zeromq2-1, mainly appear to be cases resolved in newer versions. One wishlist issue for improved logging.
|
|
[13:11] pieterh
|
neat, thanks
|
|
[13:12] pieterh
|
Steve-o: btw I put Miru on http://www.zeromq.org/intro:commercial-support
|
|
[13:12] pieterh
|
can you take a look and see if the description is OK for you?
|
|
[13:14] Steve-o
|
Cool, I have a Chinese site too: http://è¦.é¦æ¸¯/
|
|
[13:15] Steve-o
|
Well, Traditional Chinese anyway.
|
|
[13:16] pieterh
|
Steve-o: afaik zeromq.org isn't available in mainland China... :-/
|
|
[13:18] Steve-o
|
Works in Hong Kong & Taiwan though
|
|
[13:20] pieterh
|
It could be neat to make a Chinese zeromq.org site... but not using Wikidot then
|
|
[13:21] Steve-o
|
is it just .org blocked?
|
|
[13:21] pieterh
|
Steve-o: it's Wikidot that's blocked
|
|
[13:21] pieterh
|
the IP address, afaik
|
|
[13:21] Steve-o
|
oh, as a wiki, igi
|
|
[13:21] ezl
|
having trouble installing zeromq on ubuntu 10.10
|
|
[13:21] ezl
|
uuid-dev : Depends: libuuid1 (= 2.17.2-0ubuntu1) but 2.17.2-0ubuntu1.10.10.2 is to be installed
|
|
[13:21] ezl
|
E: Broken packages
|
|
[13:21] ezl
|
But installing uuid-dev shows I have the most recent version
|
|
[13:22] Steve-o
|
pieterh: all wikis are blocked, including similar bits of Google
|
|
[13:22] ezl
|
any ideas?
|
|
[13:23] pieterh
|
Steve-o: I assume the Chinese govt hates the Internet
|
|
[13:23] pieterh
|
ezl: no ideas immediately, haven't hit this on Ubuntu 10.10 myself
|
|
[13:23] pieterh
|
ezl: when do you get this error message?
|
|
[13:24] ezl
|
.configure
|
|
[13:24] pieterh
|
what 0MQ package did you download?
|
|
[13:24] ezl
|
2.1.6 off the website this morning
|
|
[13:25] pieterh
|
let me just try that for sanity's sake...
|
|
[13:25] ezl
|
thanks pieterh
|
|
[13:25] ezl
|
checking for uuid_generate in -luuid... no
|
|
[13:25] ezl
|
configure: error: cannot link with -luuid, install uuid-dev.
|
|
[13:25] ezl
|
was the 2 lines of output before configure quit
|
|
[13:26] pieterh
|
ezl: so I have uuid-dev installed
|
|
[13:26] pieterh
|
it sounds like you don't have it installed (or properly)
|
|
[13:27] ezl
|
ah. right, i don't. when trying to install uuiddev, i get the error message above about Depending on libuuid1
|
|
[13:27] ezl
|
"Depends: libuuid1 (= 2.17.2-0ubuntu1) but 2.17.2-0ubuntu1.10.10.2 is to be installed"
|
|
[13:27] ezl
|
which when i try to install linuuid1, it says that i have the latest version
|
|
[13:28] ezl
|
libuuid1 is already the newest version.
|
|
[13:28] pieterh
|
ezl: I know there was a Ubuntu update today
|
|
[13:28] pieterh
|
maybe you're halfway through an update?
|
|
[13:28] ezl
|
(sorry, I did google it too, just didn't get very far)
|
|
[13:28] ezl
|
(also pretty new to ubuntu, so it could be just a newb problem)
|
|
[13:29] pieterh
|
try sudo apt-get dist-upgrade
|
|
[13:29] pieterh
|
all the way through, including restart if needed
|
|
[13:29] ezl
|
ok thanks trying
|
|
[13:29] pieterh
|
it's something aberrant because this normally /always/ works on Ubuntu
|
|
[13:30] ezl
|
dist upgrade: 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
|
|
[13:30] ezl
|
yeah my problem seems to be libuuid1
|
|
[13:30] mikko
|
ezl: can you put config.log to gist.github.com?
|
|
[13:30] pieterh
|
hmm, try uninstalling that, and then installing uuid-dev
|
|
[13:30] ezl
|
says its installed, but then when i try to install uuid-dev, it complains that i don't have libuuid1
|
|
[13:30] ezl
|
ok
|
|
[13:30] ezl
|
mikko ok also
|
|
[13:32] ezl
|
mikko: pieterh: config.log >> https://gist.github.com/946344
|
|
[13:32] mikko
|
ezl: can you do ldconfig -p | grep uuid
|
|
[13:34] ezl
|
mikko: ok
|
|
[13:34] ezl
|
mikko: libuuid.so.1 (libc6) => /lib/libuuid.so.1
|
|
[13:35] ezl
|
piterh: as per your recommendation also tried to uninstall, but didn't go smoothly
|
|
[13:35] ezl
|
https://gist.github.com/946352
|
|
[13:38] mikko
|
ezl: looks like you have it
|
|
[13:39] mikko
|
interesting
|
|
[13:39] ezl
|
mikko: yes, ubuntu thinks i have it, but then when i try to install uuid-dev, it says it can't because it needs libuuid1
|
|
[13:39] mikko
|
you also need /usr/lib/libuuid.so
|
|
[13:40] ezl
|
"Depends: libuuid1 (= 2.17.2-0ubuntu1) but 2.17.2-0ubuntu1.10.10.2 is to be installed"
|
|
[13:40] ezl
|
sounds sort of like a version problem
|
|
[13:40] ezl
|
but i can't decipher what that error means
|
|
[13:42] ezl
|
what are ".so" and ".so.1" files?
|
|
[13:43] ezl
|
(linux newb)
|
|
[13:52] mikko
|
ezl: i think thats the so version
|
|
[13:52] mikko
|
it sounds like ubuntu has messed up the dependencies
|
|
[13:54] ezl
|
how can i force ubuntu to reinstall it?
|
|
[13:55] macson_g
|
hi, I have problem with app crashing on SIGPIPE on exit in 2.1.6. Can anyone help?
|
|
[13:56] macson_g
|
The backtrace is: zmq::mailbox_t::send, zmq::object_t::send_stop, zmq::ctx_t::terminate
|
|
[13:59] mikko
|
ezl: i don't know
|
|
[13:59] pieterh
|
ezl: remove it, reinstall it
|
|
[13:59] ezl
|
https://gist.github.com/946352 -- not letting me remove it
|
|
[13:59] pieterh
|
oh...
|
|
[14:00] ezl
|
I'm getting that E: broken packages thing on anyhting surrounding libuuid1
|
|
[14:00] ezl
|
not sure what that means, but "broken" sounds suboptimal
|
|
[14:00] ezl
|
=P
|
|
[14:01] pieterh
|
ezl: try 'apt-get clean', 'purge', 'update'
|
|
[14:02] pieterh
|
ezl: actually, looks like you have quite a mess
|
|
[14:02] ezl
|
haha awesome.
|
|
[14:03] ezl
|
i really haven't done much on my machine, not sure how i got into this
|
|
[14:03] ezl
|
(i did clean, purge, update: libuuid1 is still installed, tried to uninstall, wouldn't allow again.)
|
|
[14:06] headzone
|
this is why it pays to use debian
|
|
[14:12] pieterh
|
ezl: if you haven't done much on the system, reinstall it...
|
|
[14:13] pieterh
|
headzone: I thought Ubuntu was basically debian, especially its package management layers...
|
|
[16:32] ParadoxG
|
#gitready
|
|
[16:32] ParadoxG
|
sorry, wrong command
|
|
[16:40] mabes
|
does anyone know if zdevice (https://github.com/imatix/zdevices/tree/master/c/generic) is available as binaries/in a deb package?
|
|
[17:08] pieterh
|
mabes: nope, it's not
|
|
[17:08] pieterh
|
it's a single C source file...
|
|
[17:10] mabes
|
pieterh: I know, but my sysadmins like packages :) I'll have to look into creating our own I suppose. thakns
|
|
[17:10] pieterh
|
mabes: you're using zdevice?
|
|
[17:13] mabes
|
pieterh: well, no, not yet... I'm investigating zeromq and I need to use ROUTER/DEALER (XREQ/XRES). One process is in ruby and the other Java. Anyways, I need to use the zmq_queue and it seemed like using zdevice was the easiest way to do that from a deployment standpoint. I am new to zeromq so I may be wrong though- if so please clue me in...
|
|
[17:14] pieterh
|
mabes: the simplest for you would be to rewrite zdevice in Java or Ruby IMO
|
|
[17:14] pieterh
|
it's a very simple app, mostly concerned with reading configuration data
|
|
[17:14] pieterh
|
that should be trivial in Ruby
|
|
[17:15] pieterh
|
all the hard work is done by the zmq_device() call at the end
|
|
[17:15] pieterh
|
before you do this you need to read the Guide and understand how devices work
|
|
[17:16] pieterh
|
there is a detailed explanation of what zmq_device() actually does, and how to roll your own
|
|
[17:16] mabes
|
pieterh: Yeah, I read the guide and I actually already translated the c msgqueu example that uses zmq_queue into ruby already.
|
|
[17:16] pieterh
|
that's what you want to use then
|
|
[17:17] mabes
|
pieterh: ok, thanks for the advice. I'll go ahead with that approach then.
|
|
[17:17] pieterh
|
np
|
|
[18:52] DanielFriesen
|
I'm trying to figure out what type of ZMQ sockets I should use; Say I have 1 A and a number of B's. There are multiple requests A can send messages to B for. Individual requests can be sent to any B, but all subsequent messages related to that same request have to be sent to the same B as the first one, so push/pull is out. That sounds like req/rep but A needs to be able to send multiple messages to B and B needs to be able to make multiple replies,
|
|
[18:52] DanielFriesen
|
they can't wait for a message from the other before sending the next, so req/rep seams to be out.
|
|
[18:57] guido_g
|
sounds like a job for xreq/xrep sockets
|
|
[18:57] mikko
|
XREP/XREQ with multipart messages
|
|
[18:57] guido_g
|
why multipart?
|
|
[18:58] guido_g
|
multipart is just one message send in one go
|
|
[18:58] mikko
|
depends on how far apart the messages are i guess
|
|
[18:58] guido_g
|
see
|
|
[18:59] guido_g
|
DanielFriesen: The Guide has a chapter on lru routing, just replace the lru algo with one that fits your need
|
|
[18:59] DanielFriesen
|
Ok, I thought about XREP/XREQ but didn't know if it was usable as the primary pair
|
|
[19:00] guido_g
|
first of all, check if you really need stateful routing
|
|
[19:00] guido_g
|
most of the time it's not worth the trouble
|
|
[19:02] DanielFriesen
|
I'm writing a NIO http server, sending requests to workers doing the request handling... request initiation/headers, and body are separate messages... naturally the body of a http request needs to go to the same worker as the headers went to...
|
|
[19:02] guido_g
|
a request should be a single message
|
|
[19:02] mikko
|
DanielFriesen: have you checked mongrel2 as well?
|
|
[19:03] DanielFriesen
|
That would destroy streaming capability
|
|
[19:03] guido_g
|
as mikko pointed out, you might use multipart messages
|
|
[19:03] mikko
|
guido_g: not necessarily assuming request has large body
|
|
[19:03] DanielFriesen
|
When I saw how mongrel2 was explained in that video I shuddered...
|
|
[19:04] guido_g
|
some of us have the same feeling when seeing the average java software...
|
|
[19:05] guido_g
|
which is the only way to use java :)
|
|
[19:06] guido_g
|
whatever that might be
|
|
[19:06] DanielFriesen
|
JavaScript
|
|
[19:06] DanielFriesen
|
;) cept much cleaner
|
|
[19:07] guido_g
|
back to streaming
|
|
[19:08] guido_g
|
if you have content to stream, why partiition it into discrete messages?
|
|
[19:09] DanielFriesen
|
Open up a new socket for each connection?
|
|
[19:10] guido_g
|
no, thinking about where to get the streams content and why
|
|
[19:11] guido_g
|
streaming and request/reply are not different enough to treat them differently
|
|
[19:11] guido_g
|
ups
|
|
[19:11] guido_g
|
i ment: streaming and request/reply are different enough to treat them differently
|
|
[21:23] chuck
|
hello
|
|
[21:24] chuck
|
I'm reading through the guide, and I'm sort of confused
|
|
[21:24] chuck
|
if I want to be able to start multiple "worker" processes for a backend service and I want to contact those processes from my web application server, which socket types should I be looking at?
|
|
[21:31] mikko
|
chuck: what kind of communication are you looking at?
|
|
[21:31] mikko
|
fire and forget ?
|
|
[21:32] chuck
|
mikko, yeah
|
|
[21:32] mikko
|
then you want PUSH on the web application server
|
|
[21:32] mikko
|
and pull on backend
|
|
[21:33] chuck
|
mikko, what would it be for a synchronous response?
|
|
[21:36] mikko
|
XREP/XREQ
|
|
[21:36] chuck
|
hrm
|
|
[21:37] chuck
|
mikko, i'm just trying to think how I would set this up
|
|
[21:38] chuck
|
mikko, so I would create a server that binds, and then multiple worker clients that would connect to that
|
|
[21:38] chuck
|
and then somehow I would contact the server from my web application?
|
|
[21:39] mikko
|
yes
|
|
[21:42] chuck
|
mikko, is that the most straightforward way? it seems like the server middleman should be cut out somehow but I can't see how it would be
|
|
[21:45] mikko
|
chuck: sorry, i don't follow
|
|
[21:45] mikko
|
chuck: what server middleman?
|
|
[21:46] mikko
|
you mean "Architecture of a Single Cluster" in guide?
|
|
[21:48] chuck
|
mikko, well I mean, what I'm describing is this: web application -> backend server -> backend clients
|
|
[21:49] chuck
|
is there any way to talk directly to the clients while retaining the ability to add and remove clients from the "cluster" easily?
|
|
[21:49] mikko
|
the middleman is usually there to do routing
|
|
[21:49] chuck
|
right, and if that's necessary that's fine, i'm just not familiar with the ins and outs of every socket type so i'm not sure if it's necessary or not
|
|
[21:49] mikko
|
if there are multiple web application servers they have no idea of amount of work each backend server has
|
|
[21:50] mikko
|
the backend server in your scenario would probably do the distribution of work
|
|
[21:50] mikko
|
you can connect to backend clients directly from the web application as well
|
|
[21:50] chuck
|
can't zeromq balance the messages it sends to each backend client?
|
|
[21:51] mikko
|
it can, but if you have 10 web application servers they all balance differently
|
|
[21:51] mikko
|
you can end end up into situation where 10 servers all send work to 1 worker
|
|
[21:52] mikko
|
unless you syncronise the state across all web application servers
|
|
[21:52] chuck
|
hmm
|
|
[21:52] mikko
|
or implement a protocol where you ask each backend client how much work they have
|
|
[21:53] mikko
|
the middleman in your case would maintain LRU list of work distribution
|
|
[21:53] mikko
|
or any other algo suitable for the scenario
|
|
[21:58] mikko
|
let's say in the simplest scenario you would connect the web application server directly to backend client
|
|
[21:58] mikko
|
now when you add another backend client and connect to that as well zeromq will do round robin balancing between those backend clients
|
|
[21:59] mikko
|
now when you add another web application server and connect to those backend workers it will do round-robin as well
|
|
[21:59] mikko
|
so work goes to 1 then 2, 1, 2, 1, 2 etc
|
|
[21:59] chuck
|
but the downside is that I can't just start backend clients and call it a day, i have to add their IP addresses to the code my web app server is running?
|
|
[21:59] mikko
|
if you got two web application servers your work might go 1, 1, 2 ,2, 1,1, 2,2,
|
|
[22:00] mikko
|
due to naive round-robin and no state between the two web application servers
|
|
[22:00] mikko
|
so let's say you add third web application server
|
|
[22:00] mikko
|
worst case is: 1,1,1,2,2,2,1,1,1,2,2,2
|
|
[22:00] chuck
|
oh okay, i gotcha
|
|
[22:01] chuck
|
mikko, so this is what i should be doing? https://github.com/imatix/zguide/raw/master/images/fig49.png
|
|
[22:01] chuck
|
the single cluster architecture
|
|
[22:01] mikko
|
the middleman allows you to do a bit more intellegient routing
|
|
[22:01] chuck
|
right
|
|
[22:01] mikko
|
chuck: it all depends on the application as well
|
|
[22:03] chuck
|
mikko, so in that single cluster architecture, on the worker side, the broker would be PUSHing and the workers would be PULLing, right? how would the "clients" (in that diagram) contact the broker?
|
|
[22:03] mikko
|
the broker would be XREP and clients would be XREQ
|
|
[22:04] mikko
|
XREP = ROUTER
|
|
[22:04] mikko
|
in that diagram
|
|
[22:05] chuck
|
oh so that's not a PUSH/PULL thing?
|
|
[22:07] mikko
|
nope
|
|
[22:07] mikko
|
XREP/XREQ allow to do custom routing
|
|
[22:07] mikko
|
take a look at "A Request-Reply Message Broker"
|
|
[22:15] chuck
|
mikko, will do
|
|
[22:15] chuck
|
i've got to go, thanks for the help though, I understand it much better now. not all the way, but I'm getting there ;)
|
|
[22:17] mikko
|
cool
|
|
[22:52] Bartzy
|
Hi
|
|
[22:52] Bartzy
|
I currently have a PHP app that uses system sometimes to call executables that create images on-the-fly... I now need to separate the app to scale: one or more machines for PHP (FastCGI) only, and one ore more machines for running the executables..
|
|
[22:52] Bartzy
|
I also want to be able to use multiple "executing" servers.
|
|
[23:10] DanielFriesen
|
'Assertion failed: rc == 0 (zmq_connecter.cpp:48)' caused by accidentally using "tcp://localhost:localhost", should I report that as a bug somewhere?
|
|
[23:51] mikko
|
Bartzy: what is the actual question?
|
|
[23:51] mikko
|
DanielFriesen: yes, if you don't mind opening an issue
|
|
[23:52] Bartzy
|
mikko: I was wondering if zeromq is suitable for that... I'll need to script some custom code for the 'server' part, right?
|
|
[23:53] mikko
|
Bartzy: yes, that is correct
|
|
[23:53] Bartzy
|
Also, if I do a script in PHP or python for example.. and it does a while (true) { ... } and receives data from ZMQ inside the while... What happens if this server-script dies for some reason ?
|
|
[23:53] Bartzy
|
I'm responsible to re-spawn it, right ?
|
|
[23:53] mikko
|
Bartzy: yes
|
|
[23:54] Bartzy
|
mikko: OK, what's considered a good way to do that ?
|
|
[23:54] mikko
|
for respawning the scripts?
|
|
[23:54] Bartzy
|
yes
|
|
[23:54] Bartzy
|
inittab? monit?
|
|
[23:54] mikko
|
if it's calling an external executable i would create a simple C or C++ daemon
|
|
[23:55] Bartzy
|
and what will spawn that C daemon if it dies ? :)
|
|
[23:55] mikko
|
http://dustin.github.com/2010/02/28/running-processes.html
|
|
[23:55] mikko
|
that is a very good read on running processes
|
|
[23:55] Bartzy
|
Thank you very much.
|
|
[23:56] Bartzy
|
I'm viewing "ZeroMQ is the Answer" talk... I'm quite amazed how easy it is
|
|
[23:56] mikko
|
it's a very good talk
|
|
[23:56] Bartzy
|
But I'm wondering if Gearman would be more suitable because I wouldn't have to code the 'server'
|
|
[23:56] mikko
|
i saw it in london
|
|
[23:56] mikko
|
with gearman you still need the script running
|
|
[23:56] Bartzy
|
yes it's very straight-forward...
|
|
[23:57] Bartzy
|
So what's different between gearman and zmq ?
|
|
[23:57] mikko
|
zeromq is more oriented towards messaging
|
|
[23:57] mikko
|
where as gearman is more strictly for distributing work
|
|
[23:59] Bartzy
|
messaging, how ? What do you mean by messaging ?
|
|
[23:59] Bartzy
|
Because most of the talk I'm viewing is about distributing work in zmq
|
|
[23:59] mikko
|
with zeromq the core concept is sending a message using different patterns
|
