[Time] Name | Message |
[01:24] strich
|
WOndering if anyone could help me - I'm programming zmq in C++. Had it working fine all in a single class with methods. I refactored it into a more OO and classful system. But I'm getting crashes now that I've tried to define socket and pollitems as pointers and initialize them in the constructor. Whenever poll() is called it crashes
|
[01:25] strich
|
Its got to be that I'm passing a bad pointer around or not defining and initializing properly. I've been on this for half a day now
|
[01:26] strich
|
Is there are documentation or code examples that show it being done in a class?
|
[01:32] strich
|
My code: http://pastebin.com/avpqM2JG
|
[01:33] strich
|
Can anyone point out whats wrong?
|
[02:17] strich
|
Can anyone help?
|
[03:03] strich
|
WOndering if anyone could help me - I'm programming zmq in C++. Had it working fine all in a single class with methods. I refactored it into a more OO and classful system. But I'm getting crashes now that I've tried to define socket and pollitems as pointers and initialize them in the constructor. Whenever poll() is called it crashes
|
[03:03] strich
|
Its got to be that I'm passing a bad pointer around or not defining and initializing properly. I've been on this for half a day now
|
[03:03] strich
|
Is there are documentation or code examples that show it being done in a class?
|
[03:03] strich
|
My code: http://pastebin.com/avpqM2JG
|
[03:03] strich
|
Can anyone point out whats wrong?
|
[03:49] neopallium
|
strich: line 63: pollitems = pts; /* taking the pointer to a stack variable and assigning it to a member variable. Dont' do that. */
|
[03:54] neopallium
|
strich: also the way the code handles the zmq::context_t and zmq::socket_t values looks wrong too. You create a context object but don't store it anywhere except in a stack variable. When creating the socket object you should be storing it directly in a member variable.
|
[03:56] strich
|
neopallium - Thanks for responding. How would you suggest I keep a pointer on the pollitems array then?
|
[03:57] neopallium
|
strich: you need to allocate the array using "new"
|
[03:57] neopallium
|
so that it will be in the heap instead of on the stack.
|
[03:58] strich
|
pollitems = new zmq::pollitem_t[1]; ?
|
[04:00] neopallium
|
Yeah that should work for the pollitems. It has been a few years since I last worked on C++ code, I mainly write code in C now, so I am a little rusty with C++.
|
[04:00] strich
|
Doing this with socket now: socket = new zmq::socket_t(context, ZMQ_ROUTER);
|
[04:00] neopallium
|
also I think line 64 is wrong, you are taking a pointer to another stack variable.
|
[04:01] neopallium
|
that looks right too.
|
[04:01] strich
|
Yeah thats commented out now
|
[04:01] strich
|
Lastly, for context, I'm not assigning a global member variable because none of the class methods use it directly.
|
[04:02] neopallium
|
are you creating one context per socket?
|
[04:02] strich
|
Yes
|
[04:02] neopallium
|
you shouldn't do that.
|
[04:02] strich
|
Only the one socket too, really. The class is only instantiated once
|
[04:02] strich
|
Oh?
|
[04:03] neopallium
|
should only use one context for the whole process/program and create sockets from that one instance.
|
[04:03] neopallium
|
also when your program close it should cleanup that context.
|
[04:04] strich
|
Pretty sure zmq does that as part of its C++ codebase when the zmq destructor is called
|
[04:04] neopallium
|
sending messages is async. in zmq. You can't be sure that all messages are sent until you terminate the zmq context.
|
[04:05] neopallium
|
I haven't worked with the C++ bindings for zmq, so maybe that will not be an issue.
|
[04:06] strich
|
Either way, it still seems to be doing weird things
|
[04:06] neopallium
|
is it still crashing?
|
[04:08] strich
|
No, but something is blocking. I basically have al oop that runs, and it appears something it blocking during instantiation of Game
|
[04:09] strich
|
mmm, its that try catch I think. Weird
|
[04:09] neopallium
|
ah, I know why
|
[04:09] neopallium
|
the try catch is the scope for the zmq context
|
[04:09] strich
|
ohhhh
|
[04:09] neopallium
|
it is trying to destroy the context
|
[04:09] strich
|
and its being deleted
|
[04:09] neopallium
|
yup
|
[04:09] strich
|
hah
|
[04:10] neopallium
|
another reason to make it a member variable and not a stack variable.
|
[04:10] strich
|
Yep.
|
[04:10] strich
|
I'll try that now
|
[04:11] strich
|
Yep thats fixed it
|
[04:11] strich
|
You're a champion Neo
|
[04:11] neopallium
|
np
|
[04:12] strich
|
I think I would have been pulling my hair out :P
|
[04:12] strich
|
Scope always gets me
|
[06:16] strich
|
neopallium - Hrm, doesn't look like its fully fixed. Doing this causes a crash: zmq::poll(pollitems, 1, 0);
|
[06:17] strich
|
Its got to be that pollitems pointer is bad
|
[06:26] strich
|
Fixed it
|
[06:26] strich
|
pollitems[0].socket = *socket; // Must derefence pointer. Still compiles but will crash
|
[10:12] loxs
|
folks, when I use the REQ/RES pattern, does it imply that I must always wait for the previous RES to arrive before I fire the next REQ?
|
[10:15] jcase
|
Yes
|
[10:16] loxs
|
can't I emulate "multiple clients" from a single application?
|
[10:18] loxs
|
oh, just saw it says "too many open files"
|
[10:18] jcase
|
You can. Have a look here: http://zguide.zeromq.org/page:all#A-Request-Reply-Broker. It describes the zmq_router and zmq_dealer sockets.
|
[10:19] jcase
|
Ah, I see :)
|
[10:21] loxs
|
I'm trying something like http://friendpaste.com/1z1hl45e5lbBqRzPu2bKWw
|
[10:21] loxs
|
(which spawns 1000 erlang processes, each of them tring to open a connection to the server)
|
[10:21] loxs
|
so why do I get too many files?
|
[10:43] jcase
|
iirc you want to create one context
|
[10:45] jcase
|
Also, what OS are you on?
|
[10:46] loxs
|
linux
|
[10:58] guido_g
|
see ulimnit
|
[10:58] guido_g
|
ulimit even
|
[10:59] ntelford
|
anyone here use the jzmq bindings?
|
[11:00] ntelford
|
I've been seeing some strange behviour when manually calling socket.close and context.destroy
|
[11:00] ntelford
|
socket.close *ocassionally* causes an "Assertion: ok (mailbox.cpp:78)"
|
[11:01] ntelford
|
and context.destroy() seems to block and never return :-\
|
[11:01] ntelford
|
both of these are bound to the object destructors, so (in theory) are called by the JVM when the object's go out of scope (on shutdown, in the case of my application) - and when this happens they both seem fine
|
[13:21] CIA-121
|
jzmq: 03Nicholas Telford 07master * rf84eb8d 10/ src/Socket.cpp : Fixed setLongSockOpt() not permitting LINGER for ZMQ 3.0. - http://git.io/Mup03A
|
[13:21] CIA-121
|
jzmq: 03Gonzalo Diethelm 07master * r8658b84 10/ src/Socket.cpp : Merge pull request #69 from nicktelford/upstream-fixes ...
|
[13:30] ntelford
|
using jzmq, we're seeing occasional problems on a SUB socket - sometimes the Thread segfaults, sometimes we get this: "*** glibc detected *** /usr/java/jdk1.6.0_23/bin/java: *** glibc detected *** /usr/java/jdk1.6.0_23/bin/java: double free or corruption (!prev): 0x00000000570dc5f0 ***"
|
[13:33] ntelford
|
basically, I'm looking for any help as to how to debug this
|
[13:42] mikko
|
valgrind / gdb
|
[13:42] mikko
|
compile with --enable-debug
|
[13:42] mikko
|
and jzmq with CPPFLAGS='-g -O0'
|
[13:43] mikko
|
that way you get symbols
|
[13:44] mikko
|
you can also set /proc/sys/kernel/core_pattern to something sensible and ulimit -c to get core dump
|
[13:53] pouete
|
Hi all 0/ : question : i have a program that launch a thread to listen to a TCP port and a main program that read all the connected clients with a select(). it is working great. I add a ZMQ socket to send the data i receive from the TCP port to other modules, but the program "crash". Here is the code : http://pastebin.com/0fsCeC6R it is just a PoC , so, there maybe is some ugly hacks
|
[13:54] pouete
|
( the code as it is on the pastebin does not actually works.
|
[13:57] ntelford
|
mikko, the error above, is that indicative of an issue in the JVM itself, or could it also have been caused by a problem in zmq/jzmq?
|
[13:57] ntelford
|
(given that the JVM loads jzmq.so which in turn is linked to zmq.so)
|
[14:01] mikko
|
ntelford: it could be in the jni
|
[14:01] ninjas
|
Hi, I'm a n00b and trying to understand ZMQ_DEALER sockets a little bit better. I have 2 ZMQ_DEALER sockets which work pretty much one-way : A sends to B. In what circumstances could A's internal message queue build up? My assumption is that the queue would be building up on the incoming side, but the behavior I am seeing suggests the buildup is on the outgoing side.
|
[14:02] ninjas
|
no HWM is set, btw.
|
[14:02] ntelford
|
mikko, that's my thought, seems to be the area with the least amount of testing certainly
|
[14:23] ntelford
|
zmq_poll is interrupted by delivery of signals, right?
|
[14:38] ntelford
|
scratch that question
|
[14:38] ntelford
|
turns out in Java, there's no way to interrupt a syscall (or other blocking IO call like zmq_poll)
|
[18:06] mickem
|
Is there a "c++" version czmq?
|
[18:09] roadrunner
|
is this the correct channel to as czmq questions on?
|
[18:16] cremes
|
yes, it's the correct channel
|
[18:23] roadrunner
|
cool - is czmq intended for production? It's not just a reference implementation, is it?
|
[18:41] Fade
|
hum.. i'm getting this error starting a just checked out mongrel2 : http://librelist.com/browser//mongrel2/2011/4/25/mongrel2-trap-divide-error-linux-kernel-reports-no-time-stamp-counter-tsc/#33fceb84d325363ae7e1ebd9ca8203f6
|
[18:42] Fade
|
the system is running the ubuntu oneric beta, with sqlite and omq installed from the ubuntu pool.
|
[18:45] Fade
|
this is with just the test config as documented in section 3.2 of the manual
|
[18:45] Fade
|
so everything should be pretty vanilla.
|
[19:09] roadrunner
|
Do most people run czmq using the 'safe_malloc()' version of the allocation macro? Seems like that could be painful on systems that actually return NULL if too much memory is asked for...
|
[19:25] cremes
|
roadrunner: czmq is intended for production
|
[19:26] cremes
|
Fade: i don't know how much mongrel2 help you'll be able to get in here... it's not often
|
[19:26] cremes
|
i see people discussing it
|
[19:29] Fade
|
cremes: yeah. this is indirectly related to mongrel. the problem is in zeromq.
|
[19:30] Fade
|
I got it past the problem by commenting out the --with-pgm call in the zeromq.spec file.
|
[19:31] cremes
|
pgm support was poor in the earlier 2.1.x releases
|
[19:31] cremes
|
however, i don't think you can easily upgrade to later ones without breaking mongrel2
|
[19:32] cremes
|
i have heard/seen that it is pretty tied to specific 0mq releases
|
[19:33] Fade
|
i guess the zmq packages in the ubuntu pool (which are in the 2.1.x line) use the standard spec file.
|
[20:17] Matt
|
Hi, I am stuck on a problem and maybe someone can help me come up with a solution.
|
[20:17] Matt
|
I have web services facade that is receiving requests and I want to perform a REQ/REP within the service call.
|
[20:18] Matt
|
My initial attempt was to just open a new REQ socket for each request, but under any load the whole thing crashed
|
[20:18] Matt
|
I assume this is either a bug, or my strategy is flawed
|
[20:18] Matt
|
Anyone care to comment?
|
[20:23] michelp
|
Matt__, are you cleaning up the sockets when you're done with them?
|
[20:23] Matt
|
Not sure, it seems like the CLRZMQ library takes care of that, how would i explicitly clean them up?
|
[20:24] michelp
|
not sure about clr, but in python you do socket.close()
|
[20:24] cremes
|
Matt__: that's the wrong pattern anyway
|
[20:24] cremes
|
it doesn't make sense to create a socket for each request
|
[20:24] Matt
|
ok
|
[20:24] Matt
|
What would be the best strategy?
|
[20:24] cremes
|
Matt__: can the web service process multiple simultaneous requests or is it one at a time?
|
[20:25] cremes
|
is the processing synch or async?
|
[20:25] Matt
|
Multi requests - each request is handled on a new thread
|
[20:25] Matt
|
sync
|
[20:25] cremes
|
ok
|
[20:25] cremes
|
are you creating the threads when the request arrives or are they from a pool?
|
[20:26] Matt
|
they are from a pool
|
[20:26] cremes
|
great
|
[20:26] Matt
|
i dont have control over the threads
|
[20:26] cremes
|
there is a whole discussion on how to handle this in the guide but i'll give you the short version
|
[20:26] cremes
|
1. 1 context
|
[20:26] cremes
|
2. when you create the thread, pass it the context and create a socket inside the thread
|
[20:27] cremes
|
3. when your request comes in, the next guy in the pool should be passed the request
|
[20:27] cremes
|
make sense?
|
[20:27] cremes
|
the 0mq restriction is that sockets are not thread-safe so you can't read/write to them from many
|
[20:27] Matt
|
problem is, i dont have control over the incoming threads
|
[20:27] cremes
|
threads simultaneously unless you surround them with a mutex
|
[20:28] Matt
|
the services are sitting in IIS and a new thread is created automatically
|
[20:28] cremes
|
you don't control the thread pool?
|
[20:28] cremes
|
hmmm, ok
|
[20:28] cremes
|
do you have a max number of threads that can be created?
|
[20:29] cremes
|
regardless, here's the new short version
|
[20:29] cremes
|
1. create 1 context
|
[20:29] cremes
|
2. populate a socket pool with REQ sockets
|
[20:29] cremes
|
3. from each request thread, so a thread-safe checkout of a socket from the pool
|
[20:30] cremes
|
as long as the socket is only ever read/write in one thread at a time, you're safe
|
[20:30] Matt
|
I see
|
[20:30] cremes
|
but check the guide for other patterns... i know this is covered in one of the chapters
|
[20:30] Matt
|
is it bad practice to create/destroy the sockets?
|
[20:30] cremes
|
yes
|
[20:30] roadrunner
|
cremes: my initial scan of some of the czmq code made me nervous about what will happen in OOM situations on systems that aren't quite as lenient as linux about allocations
|
[20:30] cremes
|
they are cheap
|
[20:31] cremes
|
roadrunner: you should post your concerns to the ML... i'm a ruby guy so i don't use that lib :)
|
[20:31] roadrunner
|
cremes: okie dokie :) thanks!
|
[20:31] Matt
|
why is that bad practice to create/destroy?
|
[20:31] Matt
|
if you say they are "cheap"...
|
[20:32] cremes
|
cheap in terms of memory
|
[20:32] cremes
|
so keeping them around doesn't hurt (a few K)
|
[20:32] Matt
|
right, so expensive to create/destroy?
|
[20:32] cremes
|
but each time you create one and you connect/bind to a socket, that process takes upwards of 100ms
|
[20:32] Matt
|
I see
|
[20:33] cremes
|
ok, everyone have a nice weekend!
|
[20:33] Matt
|
Would there be anyway to have some type of asynchronous proxy that would receive the requests and respond to the requester?
|
[20:33] Matt
|
Instead of having a pool of sockets?
|
[20:33] cremes
|
Matt__: read the entire guide, please. i'm taking off now.
|
[20:33] Matt
|
ok, thank you
|
[20:44] michelp
|
Matt__, can you create thread local variables?
|
[20:45] michelp
|
you could try that approach if you can. if the variable exists, then use it, if not, create a new thread local initialized with a socket
|
[20:47] replicator
|
1. Is that a good idea to just use ÃMQ as an in-memory queue (inproc://)? Rather than for example to use various Queue data structures in many languages..
|
[20:50] replicator
|
I guess.. is there any immediate advantage in this case?
|
[20:59] zedas
|
so, i get this Assertion failed: zmq_msg_size (msg_) == 0 (req.cpp:88)
|
[20:59] zedas
|
yet another assert, and i'm just pointing a plain REQ at an XREP
|
[20:59] zedas
|
and this is in pyzmq, so, since this is a hard abort and not a real exception, it can't get caught and tell me what line of python is doing it
|
[21:00] zedas
|
this is why your asserts suck. they should be exceptions or error codes, not aborts, there should be less of them, and they should clearly state what went wrong and how to fix it.
|
[21:01] zedas
|
for example, in this assert, i have no idea what could possibly be sending a 0 length message or even what that means.
|
[21:54] bluesmoon
|
hi, I was wondering if there was any way to tell zeromq where it should create its swap files
|
[21:55] bluesmoon
|
at the moment it writes to the current directory, but I have my app set up such that its working directory is not writable, but an alternate "running state" directory is
|
[22:00] mikko
|
i think the swap is a bit bad
|
[22:01] mikko
|
you can chdir at the beginning of the app
|
[22:01] mikko
|
usually the swap setting shouldn't be needed imho
|
[22:02] mikko
|
zedas: i think i raised this issue earlier
|
[22:02] mikko
|
zedas: i was trying to make a reproducable test case but couldn't reproduce it with a simple case
|
[22:06] bluesmoon
|
Ideally I could do that and then chdir to wherever I need to be, but swap files are created on demand, when a client with an identity connects
|
[22:06] bluesmoon
|
and this might happen at any time in the process's lifecycle
|
[22:39] ninjas
|
hi I have a question. Given a one-way ZMQ TCP socket between two points A and B, where B is not consuming the messages fast enough to keep up with A (such that there is a ZMQ queue buildup occurring somewhere), what determines where the messages build up? Will it build up on A or B?
|
[23:02] mikko
|
zedas: i got a test case done and hopefully a valid fix
|
[23:14] mikko
|
zedas: https://zeromq.jira.com/browse/LIBZMQ-252
|
[23:14] mikko
|
ninjas: you can set HWM on each side
|
[23:21] ninjas
|
mikko, thank you, but I'm interested in knowing what the behavior is, assuming HWM isn't set on either end
|
[23:22] ninjas
|
granted, I may have missed it in the documentation
|
[23:49] mikko
|
ninjas: iirc on receivers side
|
[23:49] mikko
|
assuming they can be sent over the network
|