IRC Log

Tuesday March 1, 2011

[Time] NameMessage
[00:45] cremes pieterh: https://github.com/zeromq/zeromq2/issues/174
[00:46] cremes ha! i finally reproduced the condition i am seeing in my system
[00:46] cremes there's a bug with ZMQ_LINGER and zmq_connect()
[00:46] cremes when ZMQ_LINGER = 0, memory allocated for zmq_connect() will *eventually* get released (it's slow)
[00:46] pieterh cremes: ok, I'll look at it tomorrow
[00:46] cremes when ZMQ_LINGER > 0, the memory alloc'ed by zmq_connect() is *never* released
[00:47] cremes oh, you're awake... i was just leaving a note for when you got up tomorrow
[00:47] pieterh hah, IRC is now or never
[00:47] cremes it's all documented in the issue
[00:47] pieterh otherwise, email
[00:47] cremes heh
[00:47] pieterh i don't usually read back the IRC logs
[00:47] cremes get some reset already!
[00:47] cremes oh, i think sustrik does
[00:47] pieterh yeah, there's an Orval waiting for me
[00:48] cremes ok, take it easy... no worries about this issue... it has an easy work around
[00:48] pieterh I almost have the client and worker ready for MDP
[00:48] pieterh heh :-)
[00:48] pieterh the broker will take a full day
[00:48] cremes oh, very cool!
[00:48] pieterh after that, the Titanic pattern
[00:49] cremes using xreq/xrep for the server or just the client?
[00:49] cremes i assume it is...
[00:49] pieterh broker is xrep<==>xrep
[00:49] pieterh client is req
[00:49] pieterh worker is xreq
[00:49] cremes oh, that's right, one job at a time for the worker
[00:49] cremes er, client
[00:49] pieterh I fixed the protocol so the broker can handle clients and workers on a single socket
[00:50] pieterh not quite sure how I make this though, it'll need some of the containers from ZFL
[00:50] pieterh i'll probably do as for zmsg, create a zlist and zhash clone of the ZFL classes
[00:51] cremes yes, that makes sense
[00:51] pieterh yeah
[00:51] cremes i'm sure you'll have plenty of stuff to keep track of, e.g. routing envelopes
[00:51] pieterh nah, that's all handled by zmsg
[00:51] pieterh pretty much
[00:51] pieterh the difficulty in C is no data structures to reuse
[00:51] cremes really? i'll have to look at that C a little closer... all of this bug repro work has blown some of the rust off my C skills
[00:51] pieterh it turns a 100-line example into a 500 line one
[00:52] cremes yuck
[00:52] pieterh one really needs to create abstractions
[00:53] cremes i agree; my ruby code has many layers of abstractions
[00:53] cremes i wouldn't be able to create some of the complex apps that i do without them
[00:53] cremes my brain is too tiny
[00:53] pieterh mine is so small it regularly falls out of my ears
[00:54] cremes heh
[00:55] cremes ok, time for me to get back to work...
[01:37] ctarmor have anyone seen this error: Exception in MSVCR100D - c0000005
[01:37] ctarmor It occurs during heave volume
[05:28] andrewvc cremes: around?
[05:29] cremes andrewvc: for a little bit; what's up?
[05:29] andrewvc hey, some good news, wanted your thoughts on something
[05:29] andrewvc so, one issue with ffi-rzmq is depending on ffi
[05:29] andrewvc which is good in MRI, but bad in RBX and jRuby
[05:30] cremes right
[05:30] andrewvc https://github.com/andrewvc/em-zeromq/blob/master/em-zeromq.gemspec
[05:30] andrewvc did you know you can do that in a gemspec?
[05:30] cremes the jruby & rbx guys are working on a solution so the dependencies are tracked properly
[05:30] andrewvc I https://github.com/andrewvc/em-zeromq/blob/master/em-zeromq.gemspec#L20
[05:30] andrewvc err that line
[05:30] andrewvc I think that should work
[05:30] cremes true!
[05:30] cremes it's just ruby after all
[05:31] andrewvc yep
[05:31] andrewvc yeah, oruen contributed that bit
[05:31] cremes i'll commit something similar; thanks for the idea
[05:31] andrewvc but it should really be in ffi-rzmq not em-zeromq
[05:31] andrewvc cool, actually, one other thing
[05:31] andrewvc he switched me off bones and onto bundler for gem releases
[05:32] cremes yeah, i saw that; honeslty, bones has been pissing me off
[05:32] andrewvc yeah, it doesn't work well under rbx either
[05:32] cremes there is no compatibility between the 2.x and 3.x versions
[05:32] andrewvc oh? yeah it's a bit of overkill, the skeleton is nice, the rake tasks, not so much
[05:32] cremes it's kind of a big change for so little benefit... i'm open to suggestions and/or patches
[05:33] andrewvc yeah, I'll throw one together this week if you like the idea
[05:33] cremes sure
[05:33] andrewvc I'm excited with these changes it should finally be possible to just gem install and get everything you need
[05:34] cremes that is a nice little benefit; the 'ffi' gem situation has been poor for a while now
[05:34] andrewvc right now dripdrop needs a custom eventmachine and libzeromq2 and maybe ffi.
[05:34] andrewvc such a pain
[05:34] cremes true
[05:34] cremes every little bit to make it easier is worthwhile
[05:35] andrewvc yep, the only missing piece is probably an apt package for a current zeromq
[05:35] andrewvc but that's prolly about 2 years off :)
[05:35] cremes well, 0mq seems to be moving a little faster again... packages are always out of date
[05:36] cremes we live in a git world now
[05:36] andrewvc yeah, someone should right an apt adapter to github
[05:36] andrewvc or git in general I guess
[05:37] andrewvc hmmm, that's definitely going on my spare time ideas list
[05:41] cremes i think something like that exists already... but i can't remember where i saw it :(
[05:41] cremes time for bed... i've been burning the candle at both ends lately
[08:15] private_meta jstrom`: Your first message was cut off after "reproduce in C but canno"
[08:37] sustrik jstrom`: afaics what's happening there is 0mq filling up the socketpair buffer, so it doubles it's size, checks whether size was doubled and tries to write to the socketpair once again -- it fails but it should not
[08:37] sustrik some strange freebsd quirk i assume
[08:40] sustrik you don't have to reproduce it
[08:40] sustrik the question is why freebsd behaves in such a way
[08:40] sustrik and whether it can be worked around
[08:48] private_meta The term "workaround" makes me shudder every time I read or hear it...
[08:52] sustrik that's often exaclty what you have to do in the case with OS bug :)
[09:07] private_meta Hmm... if heartbeats are asynchroneous in both directions and heartbeat sending and receiving are made via zmq_poll on the server side, how can you make independent sending on the client side if you don't have zmq_poll's timeouts?
[09:38] private_meta When using socket_t::getsockopt, the equivalent of C's zmq_getsockopt in C++, it works if I call getsockopt in the same function as I create the socket, but fails if I give the socket pointer to a different function and request it there. It gives me an invalid argument error in the function.
[09:38] private_meta According to the man pages, that's "The requested option option_name is unknown, or the requested option_len or option_value is invalid, or the size of the buffer pointed to by option_value, as specified by option_len, is insufficient for storing the option value."
[09:38] private_meta But I call it exactly the same, I even call the same pointer, but in two different places, yet the same thread. Any idea what might be the problem there?
[09:45] private_meta hmm now it suddenly works... I hate demonstrative effects to the core
[09:46] private_meta Somewhat unreliable behavior
[10:48] Guthur sustrik: will there be minutes are a webcast of the meetup at san fran
[11:41] Guest19497 can any one help me about 0mq
[11:41] skm 0mq for sending files: stupid idea, yes?
[11:42] Guest19497 what the 0mq architecture?
[11:42] drbobbeaty skim: It's not what I'd use, no.
[11:42] Guest19497 hi
[11:42] skm sweet - ill stick to 0mq for messaging and regular sockets for file sending
[11:43] drbobbeaty Guest19497: There's a very nice into to ZeroMQ at http://zguide.zeromq.org/page:all
[11:43] Guest19497 thanks
[11:44] drbobbeaty Guest19497: it contains lots of examples, explanations, even different language bindings.
[11:44] pieterh I've made a URL shortener at zero.mq
[11:45] pieterh e.g. http://zero.mq/ib -> https://github.com/ianbarber/ZeroMQ-Talk
[11:45] pieterh if anyone wants access to this, let me know
[11:53] private_meta pieterh: hmm... when trying to translate the lazy pirate pattern to C++ I get assertions
[11:54] pieterh private_meta: such as...?
[11:54] private_meta I have the lazy pirate client, I try to start it, and I would have the setsockopt call for ZMQ_LINGER. When calling it, I get the error "Assertion failed: rc == 0 (zmq_connecter.cpp:47)"
[11:55] pieterh please post your code to a gist
[11:55] private_meta gist?
[11:55] pieterh gist.github.com or another pastebin
[11:55] pieterh presumably the connection failed because you have no server running?
[11:55] pieterh I don't know, can't tell without code to look at...
[11:56] pieterh you've run the C code, and it works?
[11:56] private_meta The server is running
[11:56] private_meta hmm... No, I didn't try the C version
[11:56] private_meta I don't really code in C, I try to avoid it
[11:56] private_meta (don't like it)
[11:57] pieterh at least run it, as a sanity check
[11:57] private_meta meh... I'll try
[11:58] pieterh it's a basic tool for problem solving, replace pieces and see when it starts/stops breaking
[12:00] private_meta pieterh: btw, you wrote in the docs that you use the lazy pirate client for the paranoid pirate server. You made heartbeats in the server, but apparently not on the client. am I misunderstanding something there?
[12:00] pieterh nope, that's how it works
[12:01] pieterh there are good reasons to do heartbeating from queue to worker, but not from client to queue, afaics
[12:01] pieterh queues don't hold any resources per client
[12:01] pieterh clients don't care if the queue is present/absent except when they are doing a request
[12:01] private_meta You send a heartbeat out, but the client doesn't even seem to receive one, I don't quite get how that works to assure that there actually IS a working connection
[12:02] pieterh the client-to-queue dialog has no heartbeating
[12:02] private_meta ok, you say, queue, do you mean the server?
[12:03] pieterh I'm avoiding the word 'server' because it's ambiguous
[12:03] pieterh we have client, queue, and worker
[12:03] pieterh when I say 'queue', I mean 'queue'... :-)
[12:05] private_meta ok, then can you define those three, what they do exactly, because I didn't seem to find any good definition in the docs
[12:05] pieterh it is always local to the pattern
[12:06] pieterh that's why I provide a schema for each pattern with the boxes clearly labelled
[12:06] pieterh client is easy
[12:06] pieterh queue is easy
[12:06] pieterh worker is also unambiguous
[12:06] pieterh but "server" is anything that accepts a connection, or does work, or ...
[12:07] private_meta So going after figure 21 in the docs, "Client" is what I and everyone I work with always understood as "Server", and "worker" is what I understand as "Client", and the "Server in the figure is something I didn't even name yet in that regard
[12:08] private_meta I have never met anyone who defined it the way it's defined here
[12:10] pieterh private_meta: figure 21 is really clear
[12:11] pieterh the client is the application that makes a request
[12:11] pieterh the server is the host that does the work
[12:11] private_meta pieterh: Yeah, according to that, as I said, I never met ANYONE who defined it like that, not even remotely
[12:11] pieterh hmm, e.g. web browser and web server
[12:11] pieterh so your browser is a "server"?
[12:11] pieterh and google.com is the "client" of your browser?
[12:11] private_meta No, my browser is a client, google is a server
[12:12] pieterh and that's figure 21
[12:12] pieterh what's confusing about it?
[12:12] pieterh maybe you want the server at the top of the page, and the client below?
[12:12] pieterh all the diagrams show the logical flow of requests / tasks from top to bottom
[12:13] private_meta Wait, in Figure 21 Google would be the worker?
[12:13] pieterh figure 21 shows an application server, in fact
[12:13] pieterh workers are applications, if you like
[12:13] pieterh server plugins
[12:14] pieterh this is all classic stuff, can I please go back to my Majordomo coding now ? :-)
[12:14] private_meta Well, you didn't really help me understand it
[12:14] private_meta :/
[12:14] pieterh since you don't tell me what your confusion is...
[12:15] pieterh I'd suggest reading the text like 10 times until it clicks
[12:15] pieterh these concepts are slippery, I know
[12:15] pieterh but I can't explain them over IRC better than the text and pictures already do
[12:15] private_meta Well, just give me a real world example what CLIENT, SERVER and WORKER would be in an example I can understand
[12:15] private_meta you're giving me confusing information
[12:16] pieterh I've just done that, you're making me repeat myself
[12:16] pieterh web browser = client
[12:16] pieterh web server = server
[12:16] pieterh PHP application producing web content = worker
[12:16] private_meta ok, that was clear
[12:16] pieterh there are hundreds of such examples
[12:16] guido_g rofl
[12:16] private_meta it wasn't clear before
[12:17] private_meta hmm... sanity check for lazy pirate pattern worked in C
[12:17] guido_g <private_meta> I have never met anyone who defined it the way it's defined here <- hard to believe
[12:17] pieterh private_meta: like I said, re-read the text until it clicks
[12:18] private_meta int linger = 0; client->setsockopt(ZMQ_LINGER, &linger, sizeof(linger)); <-- that line fails in c++
[12:19] skm i have 3 servers that dish out work (in the form of mp3 files) and 5 clients that need to receive the files - i want to add servers and clients with little effort so im going to have a manager inbetween them all..
[12:19] jugg eh, api.zeromq.org is broken...
[12:19] skm im using push/pull for the servers - my problem is what if the manager fails then comes back up - what's the best way to resend the files
[12:19] pieterh jugg: where?
[12:19] jugg http://api.zeromq.org/zmq-setsockopt-html
[12:19] jugg but any of the links
[12:20] skm (the files exist on the server until a client downloads it from the server - the manager just has a list in a queue)
[12:20] pieterh skm: there's no documented patterns for reliable push/pull, I'm working on those
[12:20] pieterh jugg: bleh
[12:20] private_meta pieterh: yeah, all links from the guide to the api are broken
[12:20] private_meta or seem to be
[12:20] jugg hmm, looks like api.zeromq.org was cached, forced refresh pulls the new page.
[12:20] pieterh private_meta: nope, fixed those about 10 minutes ago, reload
[12:20] jugg which has different links
[12:20] pieterh jugg: I'll need to make forwards from all the old API pages... bleh
[12:21] jugg yah
[12:21] private_meta ah ok
[12:21] private_meta just reloaded 30 minutes ago XD
[12:22] private_meta pieterh: maybe I made a mistake, but I thought it should be a working translation https://gist.github.com/45ae7e2a36ad5c1ca50f https://gist.github.com/a4df2891f9c8edc1ac0c
[12:23] jugg v2.1.1 is not tagged in Git
[12:23] pieterh private_meta: yeah, it looks OK, I've no idea why the C++ setsockopt would be failing
[12:24] pieterh jugg: it's in http://github.com/zeromq/zeromq2-1
[12:24] pieterh we forked the repository to make life easier for everyone
[12:24] jugg ugh
[12:25] pieterh jugg: there are pros and cons but it's simplified our work a lot
[12:25] private_meta The setsockopt fails with the error stated 30 minutes ago, and if continued I get the error "Bad file descriptor, nbytes != -1 (mailbox.cpp:241)"
[12:25] jugg well, thankfully Git supports multiple remotes...
[12:30] private_meta pieterh: Now that I got the terminology described, my worker needs to know if the client is there, and my client needs to know if my worker is there, so i need heartbeats from client to server and vice versa, that's why I was asking before about the heartbeats
[12:31] Guthur umm this talk of linger has just made be notice a minor type bug in clrzmq2
[12:31] Guthur at least it was good for something, hehe
[12:32] Guthur be/me
[12:34] pieterh private_meta: what pattern are you talking about?
[12:34] pieterh figure 21 is asynchronous server
[12:35] pieterh clients don't see workers, workers don't see clients
[12:35] private_meta pieterh: I was talking about figure 21 because it made it understandable somewhat more easily
[12:36] pieterh jugg: all the old API links should properly redirect now
[12:36] pieterh I sent them to 2.1.1
[12:39] private_meta pieterh: I need custom routing, something like router-to-dealer, but with heartbeating, but in both directions, as workers and clients should be aware of each other
[12:39] private_meta I don't know if there is a pattern more fitting to that
[12:40] pieterh take a look at http://zero.mq/md, specifically the server protocol, it may what you need
[12:40] pieterh your client could implement the broker part of the protocol
[12:40] jugg pieterh, looks like it.
[12:41] pieterh if you implement precisely the protocol as specified it'll interoperate with other code using the same spec
[12:41] pieterh jugg: great!
[12:46] private_meta pieterh: ok, so I would have n Clients and 1 Worker I assume
[13:03] private_meta So if I adhere to the protocol specifications best as possible, a guide chapter is in the works to help me with the problem specificied
[13:03] private_meta ?
[13:15] skm if i have 1 client pushing and 3 clients pulling - if the client pushes 100msgs will they be sent to the first client that connects
[13:15] skm or in order of each client recving
[13:19] private_meta pieterh: should I create an issue for the ZMQ_LINGER problem in the c++ translation?
[13:21] cremes private_meta: put up a pastie with the compilation error; that c++ you posted earlier looked correct
[13:21] private_meta cremes: there's a runtime error, not a compilation error
[13:21] cremes ok, then post that error (all of it)
[13:22] private_meta Yeah, just wanted to say that it doesn't change that I should post the error :)
[13:22] cremes heh
[13:26] private_meta cremes: theoretically, if it's translated correctly, the C server should work with the C++ client, shouldn't it?
[13:27] cremes yes; i have 0mq from ruby interacting with 0mq from C on a regular basis
[13:27] cremes and it's not even theoretical!
[13:27] cremes :)
[13:27] stimpie Iam having some problems using an xrep socket on en inproc transport, any way of sniffing or logging the inproc transport?
[13:28] cremes stimpie: i am not sure.... there is a new "sys" logging facility that may be of use
[13:29] cremes check the mail archives for discussion or the git log for patches
[13:29] private_meta ok, c server works with c client, c++ server works with c client, but both servers won't work with c++ client
[13:29] cremes private_meta: ok; please post the error *and* the code for all of the components
[13:29] cremes you might have something wired wrong
[13:30] cremes (i'm expecting a gist with five files: error, c client, c server, c++ client, c++ server)
[13:31] private_meta Well, I write 4 gists with the 4 c/c++ files and I write the error in the issue
[13:32] private_meta cremes: then again the c client and server are exactly those from the docs
[13:32] private_meta so i won't need to "gist" them
[13:34] cremes private_meta: i'm reading "cremes, i don't really want your help"
[13:34] private_meta lol
[13:34] cremes at least send me a link to the components that are from the docs
[13:34] private_meta cremes: I intended to put them into the issue
[13:34] cremes don't make me do a lot of extra work to help you out; it isn't fair
[13:34] cremes i don't think we have established yet that this warrants an issue
[13:35] cremes these things interoperate all of the time
[13:35] cremes let's dig just a little before opening a ticket
[13:35] private_meta If you want to
[13:37] private_meta cremes: https://gist.github.com/dadca0c3b776aa5713da sufficient?
[13:38] cremes good enough; give me a few to look through it
[13:39] private_meta Maybe it's just a problem with me using setsockopt, but I hope not
[13:40] guido_g client->connect("tcp://localhost.5555"); <- wrong
[13:41] skm uf a pusher has pushed 100msgd, the first client to call recv then gets all 100 msgs (and has to recv 99 more times to see them) - correct?
[13:42] private_meta guido_g: I translated it with that url from the C version
[13:42] private_meta guido_g: why is it wrong?
[13:42] guido_g read it
[13:43] H-nes hint: ":"
[13:43] private_meta oh
[13:43] guido_g right
[13:43] private_meta damn me
[13:43] guido_g all thar trouble for basically nothing
[13:43] guido_g *taht
[13:43] guido_g *that
[13:44] guido_g how effective
[13:44] cremes guido_g: good eye
[13:45] guido_g cremes: we in germany are now known for our keen eyes on copied work ,)
[13:45] Guthur guido_g: I did realise that I need to implement these options in the clrzmq2 binding due to this discussion, if that is any consolation, hehe
[13:45] cremes guido_g: ;)
[13:46] cremes private_meta: after you make that change, let us know if all is well
[13:46] private_meta hmm apparently not
[13:47] guido_g more typos?
[13:47] private_meta I hope not
[13:47] private_meta But it's a different error now
[13:53] private_meta cremes: update
[13:53] private_meta cremes: https://gist.github.com/dadca0c3b776aa5713da
[13:54] cremes private_meta: that is oftentimes a buffer sizing problem; what OS are you on?
[13:55] private_meta Ubuntu 10.4 x64
[13:56] cremes private_meta: give me a gist with: net.core.rmem_default, net.core.rmem_max, net.core.wmem_default, net.core.wmem_max
[13:57] cremes these control the size of the buffers for "socketpairs" which are used for all internal signaling in 0mq
[13:57] private_meta How do I get those?
[13:58] cremes sysctl -a | egrep 'wmem|rmem'
[13:58] private_meta kk
[13:58] cremes how long before you saw that error? 10s, 1m, longer?
[13:58] private_meta Let me test
[13:59] cremes and does it happen on *every* run
[13:59] stimpie took me 4 hours to find out I encountered an known issue #issue/82 grrrr
[13:59] cremes private_meta: the reason i ask is because the c client should produce the same error in roughly the same timeframe
[13:59] private_meta cremes: Immediately, at every run
[14:00] private_meta But the C Client works
[14:00] cremes now that is odd
[14:01] private_meta cremes: https://gist.github.com/6c43ecadae82d425773b
[14:01] cremes you should run it run gdb and pastie the backtrace too
[14:02] cremes those sizes should be ok
[14:02] private_meta cremes: the entire gdb trace?
[14:02] cremes i use 5242880 for all of those settings, fyi
[14:03] cremes private_meta: yes, all of it
[14:03] cremes do: thread apply all bt
[14:03] private_meta cremes: https://gist.github.com/8c059893dbec50829e84
[14:06] cremes private_meta: i can't make heads or tails of that output
[14:06] cremes it looks like nonsense
[14:07] private_meta cremes: I never worked closly with gdb or traces, so I gave you what eclipse tagged as "traces"
[14:07] cremes oh...
[14:07] private_meta gdb traces to be exact
[14:07] cremes from a command line, run this: gdb --args <name of lpclient>
[14:07] private_meta sec
[14:07] cremes (gdb) run
[14:08] cremes (gdb) thread apply all bt
[14:08] cremes pastie the output
[14:11] private_meta cremes: better? https://gist.github.com/0eba301579e05243a715
[14:12] cremes better
[14:14] cremes private_meta: change: zmq::poll (items, 1, REQUEST_TIMEOUT * 1000);
[14:14] cremes to: zmq::poll (&items, 1, REQUEST_TIMEOUT * 1000);
[14:14] cremes (just added & in front of items)
[14:17] private_meta cremes: cannot convert 'zmq::pollitem_t (*)[1]' to 'zmq_pollitem_t*' for argument '1' to 'int zmq::poll(zmq_pollitem_t*, int, long int)'
[14:21] cremes private_meta: oops, my mistake; try this:
[14:21] cremes zmq::poll (&items[0], 1, REQUEST_TIMEOUT * 1000);
[14:21] guido_g yeah, that's c++ :)
[14:21] cremes btw, i'm getting all of this from looking at the c++ examples in the zguide
[14:21] guido_g hrhrhr
[14:21] cremes your translation needs work
[14:22] private_meta same
[14:22] cremes won't compile?
[14:22] private_meta no, same as items
[14:23] private_meta so, same runtime error
[14:23] cremes can you get any of the c++ examples in the zguide to run?
[14:24] private_meta sure
[14:24] guido_g just to make sure, try a delay (sleep) after the send and before the poll, might be that the connection is not yet established
[14:24] private_meta guido_g: nope
[14:25] guido_g why?
[14:25] private_meta "nope" as in "doesn't help"
[14:25] Guthur guido_g: would polling before the connection establishes cause issues?
[14:26] guido_g i'm not sure
[14:26] private_meta cremes: I haven't tried ALL of them, but they normally DO work
[14:26] guido_g that's why i asked to test it
[14:26] Guthur guido_g: ok, the reason I asked was I had an issue like that last night on linux
[14:26] private_meta guido_g: A sleep of 1 second didn't help
[14:26] Guthur using the clrzmq2 binding
[14:27] guido_g private_meta: where was the sleep?
[14:27] private_meta after s_send before zmq::pollitem_t, but not after every poll.
[14:28] guido_g ok
[14:28] cremes when i run it under gdb, it shows that it is spawning 2 client threads and connecting to the server twice
[14:29] cremes wtf?
[14:30] cremes no, something else is up...
[14:34] private_meta pieterh: Just out of premature curiosity, how would I then be able to use the Majordomo Protocol?
[14:37] cremes private_meta: fixed it
[14:37] cremes change: zmq::pollitem_t items [] = { { client, 0, ZMQ_POLLIN, 0 } };
[14:37] cremes to: zmq::pollitem_t items [] = { { *client, 0, ZMQ_POLLIN, 0 } };
[14:38] cremes (and keep the other changes)
[14:39] cremes wow, i haven't done any c++ in about 10 years and this experience hasn't made me miss it :)
[14:39] guido_g hehe
[14:39] private_meta cremes: wtf?
[14:39] cremes private_meta: do i get a coauthor credit on this translation? :P
[14:40] cremes private_meta: look at the other c++ examples; yours is the only one that uses 'new'
[14:41] cremes i think you had a pointer to a socket pointer
[14:41] cremes or something... i'm out of my depth :)
[14:41] guido_g cremes: oh nice, didn't spot that one
[14:42] private_meta cremes: It's not that I wanted to use new
[14:42] cremes guido_g: better lucky that good <--- my motto
[14:42] cremes s/that/than/
[14:43] private_meta cremes: As far as I can see it's the only example where the socket is created in a different method, and for one the first example returned a pointer, so I wanted as well, and I thought it made the same problems as message_t
[14:43] cremes private_meta: what i remember about c++ could fit in a thimble; i just started matching up working code against non-working
[14:43] cremes and went from there
[14:44] cremes seems like the library should do some more sanity checking on passed arguments though
[14:44] private_meta cremes: the copy constructor of socket_t is private, so I can't return it as a non-pointer, and I didn't want to push socket_t into the parameters
[14:44] cremes it doesn't seem like we should be able to compile and run something with so many errors
[14:45] cremes (easy to make errors... i'm not poitning any fingers)
[14:45] Guthur sanity and c++?
[14:45] cremes Guthur: i know!
[14:45] private_meta Well, C takes a pointer, C++ doesn't, it's nasty
[14:46] private_meta ok
[14:47] private_meta cremes: the problem is that zmq_pollitem_t is in the zmq.h, there is no zmq.hpp version where (void *) is correctly translated to socket_t apparently
[14:47] cremes i don't know what that means ;)
[14:48] cremes heh
[14:48] cremes if that's a genuine hole in the lib, open a ticket with a patch
[14:48] cremes sounds like it would be pretty easy to make
[14:49] private_meta hmm
[14:50] private_meta either a genuine type for c++ or a sanity check in zmq::poll
[14:50] private_meta but the latter won't fix the problem at compile time
[14:51] private_meta typedef struct { void *socket; int fd; short events; short revents; } zmq_pollitem_t; <-- the void* there is the unfriendly thing
[14:56] private_meta hmm
[15:02] private_meta cremes: doesn't seem THAT easy
[15:13] CIA-21 zeromq2: 03Martin Sustrik 07master * r4c74462 10/ src/socket_base.cpp :
[15:13] CIA-21 zeromq2: Explicit identities bug in inproc transport fixed.
[15:13] CIA-21 zeromq2: The identity of bound socket was not used. Instead, transient
[15:13] CIA-21 zeromq2: identity was generated.
[15:13] CIA-21 zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/heLOOL
[15:14] private_meta Been led to the following problem, http://stackoverflow.com/questions/231760/what-does-a-type-followed-by-t-underscore-t-represent/231807#231807, don't you concern yourself with the problem because you prefix every type in C with "zmq_"?
[15:40] stimpie sustrik, great I was just running in to it.
[16:02] sustrik stimpie :)
[16:52] private_meta https://gist.github.com/5beb18da2cdb0ed04e02 https://gist.github.com/427a77b7837b5010483a <-- I thought that would be an easy class based example for communicated through zmq_poll, shouldn't I get the "START" message at the worker? What am I doing wrong?
[16:53] ianbarber pieterh: are the lazy pirate examples translatable now then?
[17:12] private_meta k, got it to work >_>
[17:12] cremes private_meta: you're going through some of the same growing pains most of us did when we first
[17:13] cremes started playing with 0mq
[17:13] cremes though i'll say maybe the c++ aspect is throwing an additional monkey-wrench into the process
[17:13] cremes :)
[17:15] private_meta cremes: my problem is that we had to switch from a different comm library because it threw it's own monkey wrench into the project, and now everybody tells me "quick, we need it, immediately, if you don't, we have problems"
[17:16] private_meta cremes: and I'm completely new with zmq
[17:16] cremes uh oh
[17:16] cremes not to say you won't have success with 0mq, but it comes with time & experience
[17:17] private_meta uhm... do durable sockets send multipart messages with the id as the first part?
[17:19] ianbarber xrep does
[17:19] private_meta hmm
[17:19] cremes the ID is only sent upon connect/bind for each socket type
[17:19] cremes this is invisible to the application
[17:19] private_meta cremes: quote from my project manager: "If you take too long with zmq I assume it won't help us, and you have to implement communication from scratch"
[17:19] ianbarber ouch!
[17:20] ianbarber xrep wraps the message with the ID, so it knows who to route back to
[17:20] cremes xrep sends the identity as a visible message part as part of a routing envelope on a reply
[17:20] ianbarber it'll unwrap when you send back
[17:20] cremes private_meta: your manager must have pointy hair
[17:21] private_meta cremes: actually, he's a programmer, he even programmed in projects
[17:21] private_meta cremes: and apparently the projects didn't fail
[17:21] private_meta hmm
[17:21] cremes well, making that statement undermines past performance
[17:22] cremes he may be a good programmer but he doesn't seem to be too hot of a manager (IMHO)
[17:22] private_meta That's the paradox
[17:22] cremes we'll help where we can; have you read the guide from start to finish twice yet? ;)
[17:22] ianbarber and tried the examples!
[17:23] cremes ianbarber: indeed!
[17:23] private_meta I read much of it and tried some examples, tried modifying them and stuff
[17:23] ianbarber it does help to thing of XREP and XREQ as router and dealer
[17:23] ianbarber makes it clearer what they do
[17:23] ianbarber s/thing/think
[17:24] private_meta Well, I did try to build a router/dealer
[17:24] private_meta sec
[17:24] private_meta https://gist.github.com/5beb18da2cdb0ed04e02 https://gist.github.com/427a77b7837b5010483a
[17:25] private_meta I was just confused that I got something like <id>\n<message>\n<id>\n<message2>\n<id>\n<message3>
[17:26] sarikan greetings
[17:26] ianbarber yeah, that's just xrep doing its thing, you can use req and rep for straightforward client server
[17:26] private_meta Well, I already established I need something in the range of router/dealer
[17:26] ianbarber if you went req -> xrep -> xreq -> rep the 'req' and 'rep' would never see the extra field
[17:26] private_meta pieterh said something that would fit my requirements would be the majordomo protocol he is currently working on
[17:28] private_meta ianbarber: I don't think the extra layer is all that neccessary
[17:28] private_meta damn.... still wanted to buy food :/
[17:28] private_meta damn this town and it's stores closing at 18.30
[17:29] ianbarber takeaway! friend to the hungry coder :)
[17:30] private_meta hmm
[17:30] private_meta I don't have a car :D
[17:31] private_meta And the only pizza delivery service here requires you to order 2-3 pizzas for them to deliver *sigh*
[17:31] ianbarber maybe pieterh has a fast food distribution system planned for chapter 5 of the guide
[17:31] private_meta hahaha
[17:31] cremes private_meta: it might be best if you described your use-case in detail
[17:31] cremes then we can make suggestions
[17:31] private_meta hmm
[17:31] cremes this might also be a good thing to take to the mailing list
[17:32] cremes i'm going to have to go "async" on irc in a few... lots of work to do
[17:32] ianbarber yeah, probably better if it's more than a one-line job to describe it
[17:32] cremes so following a conversation will be hard for me
[17:32] ianbarber also helpful for future people searching
[17:32] cremes private_meta: feel free to leave out the super-secret-squirrel-intellectual-property parts
[17:32] private_meta I'll start drawing a pic
[17:32] cremes that would make your boss mad if they were public :)
[17:40] cremes sustrik: ping
[17:40] sustrik pong
[17:40] cremes anything i can do to make https://github.com/zeromq/zeromq2/issues/174 clearer for you and pieterh?
[17:40] cremes he can't seem to repro the problem as stated
[17:41] cremes maybe file descriptors need to be bumped for the process to some larger value?
[17:41] cremes or src/config.hpp needs to be modified so max_sockets is > 512 (default)
[17:42] cremes i usually build the lib with 51200 as my max_sockets :)
[17:42] private_meta I have 1 "manager", the worker. I have n clients, nodes. The manager needs to send routed messages to single specified nodes which logon beforehand. Communication needs to be asynchroneous, bidirectional, so both have to initiate communication if neccessary. I also need awareness of connection termination, so a heartbeat on both sides.
[17:43] private_meta cremes, ianbarber, that would be an overall description
[17:43] cremes private_meta: yep, sounds like pieterh's new mdp project
[17:43] cremes he's hard at work on it right now; perhaps there could be collaboration?
[17:43] private_meta As soon as these asynchroneous messages arrive at their respective endpoints, a logic is already in place to handle these messages, if needed even to create synchronity out of async messages
[17:44] private_meta Well, I would like to help, it would make it faster for me in some respect, I just don't know how I can helpo
[17:44] private_meta -o
[17:50] private_meta cremes: Well... I can ask pieterh if I could do anything to help, but if not I guess that the best thing to do right now would be to get to understand zmq better until he's done
[17:51] cremes good plan... you are kind of diving into the deep end
[17:51] private_meta cremes: well, I guess it wouldn't be good for me if he finished it and I still don't know that much about it >_>
[17:52] private_meta pieterh: so anything I can do to help, just tell :D
[17:52] private_meta k, bbl
[17:57] sarikan if I use the java bindings for zeromq in a web application, would I run into threading issues?
[17:58] sarikan multiple java threads will be using jni to access zeromq
[17:59] cremes sarikan: only if you try to access the same socket from multiple threads; allocate a 0mq socket per thread and you'll be fine
[18:02] sarikan cremes: so If I create, use and close a socket for each request, I should be fine
[18:03] cremes sarikan: sure, but that's an anti-pattern
[18:03] cremes what's your use-case?
[18:03] sarikan java web services, talking to some native code using 0mq as the middleware
[18:03] sarikan each request to java web service ends up being a thread
[18:04] cremes sarikan: a new thread or a thread from a pool?
[18:04] sarikan so I can't think of anything other than what I've written above
[18:04] sarikan more likely to be a thread from a pool
[18:05] cremes ok, so allocate a socket for each thread in the pool and reuse the socket just like you're reusing the thread
[18:05] sarikan cremes: that is an interesting approach, if I can monitor and control thread creation in the java web server, it should work fine
[18:07] sarikan cremes: do you think I need to watch for this if I'm using c++ instead of java? I guess it would be the same
[18:08] sarikan cremes: thanks anyway, helpful starting point
[19:03] Guthur I just noticed ZMQ_FD, is that not going to be a little problematic if we manage to get named pipes on Windows?
[20:20] Guthur is there some overriding technical reason why it is not possible to query for assigned subscription patterns
[21:51] mikko man, i love london
[21:51] mikko it's full of random things and events
[22:02] Guthur mikko, you a londoner then?
[22:02] mikko yep
[22:03] Guthur I was there last week actually, it's ok.
[22:03] Guthur I was only there for 2 days though, and it was mostly business
[22:03] mikko you should've told!
[22:03] mikko we could've had a mini-meetup
[22:03] Guthur never thought actually
[22:04] mikko we went out for beers last time sustrik was here
[22:04] mikko one other guy from london came out as well
[22:04] Guthur i'm sure i'll be there again sometime, i'm only across the irish sea, hehe
[22:05] ianbarber definitely up for london mini-meetups if everyone is about
[22:05] Guthur actually does anyone leave near Hamburg
[22:05] Guthur leave/live
[22:05] mikko we could organise one
[22:05] Guthur i'm in hamburg for the European Lisp Symposium
[22:06] Guthur it's a shame I never thought of setting up some demo with ZeroMQ and Common Lisp, the theme for this year is concurrency
[22:07] mikko ianbarber: why don't we organise one at some point?
[22:07] ianbarber mikko: that's a good idea!
[22:07] ianbarber we can setup a meetup event or something, and email the list
[22:07] ianbarber mid-late march?
[22:09] ianbarber i'm at confoo next week, but maybe the 17th or something like that
[22:09] mikko groupspaces
[22:09] mikko support local industries
[22:10] ianbarber good point :)
[22:18] ianbarber done: http://groupspaces.com/zeromq-london/ :)

Web site design and content is copyright © 2010-2011 Peter Alexander, ProDataLab. Site content licensed under the Creative Commons Attribution-Share Alike 3.0 License. ØMQ is copyright © Copyright © 2007-2011 iMatix Corporation and Contributors. Contact us for professional support. ØMQ is free software licensed under the LGPL. ØMQ, ZeroMQ, and 0MQ are trademarks of iMatix Corporation. Terms of Use ? Privacy Policy