ZeroMq IRC Log

Friday January 7, 2011

[Time] Name	Message
[01:19] codebeaker	Good evening all, quick question for an inexperienced C-developer, I'm following the Introduction, and I'm getting an odd error - https://gist.github.com/4f82911f324c8fae8f9e - trying to compile one of the exampels. What have I missed, zmq installed (maint branch) from Git, built and make-installed to the standard places, and I'm compiling my sample project with a vanilla cmake project
[01:20] codebeaker	is there a CMake find_zmq I'm missing?
[01:32] codebeaker	I found this https://github.com/PatrickCheng/zeromq2/blob/master/FindZMQ.cmake - P.Chen - thanks if you're idling here somewhere!
[03:55] jugg	Can zmq ever error with EAGAIN if ZMQ_SNDMORE is set?
[03:57] jugg	(of course ZMQ_NOBLOCK is also set)
[04:26] rqmedes	anyone use zeromq with c#?
[07:36] sustrik	jugg: yes it can
[07:36] sustrik	for the first message part
[08:01] parik	Hi Sustrik and Steve
[08:02] parik	I am trying to build zeromq src but running ./configure script before that to check the dependencies and I am getting error "configure: error: cannot link with -luuid, install uuid-dev."
[08:03] parik	I tried looking on web-also for downloading uuid-dev but not of much success.I didn't get the package
[08:04] parik	Does anybody help me out
[08:04] parik	in this regard
[09:00] mikko	rqmedes: there are a couple of people who do use it in C#
[09:16] jugg	sustrik, So, after send(...,SNDMORE\|NOBLOCK) the first message part successfully, I could subsequently tight loop send(..., SNDMORE\|NOBLOCK) an arbitrarily large amount of messages and never get EAGAIN? If so, could you explain what causes EAGAIN to be generated?
[09:39] sustrik	jugg: EAGAIN means there are already HWM messages in 0mq's buffer
[09:39] sustrik	so you get EAGAIN when you try to insert new message
[09:39] sustrik	which happens with the first message parts
[09:39] sustrik	subsequent message parts belong to the same message that was already validated
[09:39] sustrik	so you never get EAGAIN there
[09:41] jugg	so nothing to do whether data can be put on the wire?
[09:44] parik	Hi
[09:46] jugg	anyway, what seems to me a bug - zmq::xsub_t::xhas_out () is implemented returning true. Thus getsockopt(events) shows the a sub socket can be written to.
[09:46] parik	Where I can find the definiton for zmq_init()
[09:47] jugg	http://api.zeromq.org/
[09:48] parik	That's the manual page.I have referred it but I want to know What inside zmq_init is happening.How its initalizing and returning context and creating a thread poool
[09:48] parik	I am interesting in code
[09:49] jugg	http://github.com/zeromq/zeromq2
[09:52] parik	Thanks jugg But I want to have C source
[09:55] guido_g	funny
[09:56] guido_g	Ã¸mq is written in c++
[09:58] parik	But In 0MQ Guide its showing C example using zeroMQ facililites
[09:58] parik	Also other Languages
[09:58] guido_g	sure
[09:59] guido_g	the Ã¸mq core is still written in c++
[09:59] guido_g	"<parik> That's the manual page.I have referred it but I want to know What inside zmq_init is happening.How its initalizing and returning context and creating a thread poool" <- this part is in the core
[10:01] parik	ok.
[10:02] yrashk	I am new to zeromq2, and have a simple question â for inproc, should endpoint be unique per process if each thread has its own pair of push/pull sockets?
[10:02] yrashk	s/unique per process/unique process-wide/
[10:03] mikko	yrashk: it should be unique to context
[10:03] yrashk	mikko: alright, make sense, thanks :) is it okay to have a context per thread?
[10:03] mikko	yrashk: no
[10:04] yrashk	ok, I'll generate unique endpoints then :)
[10:04] mikko	yrashk: you can only have inproc communications between sockets allocated from same context
[10:04] mikko	yrashk: i usually use something like "inproc://thread-%d" where %d is thread id or similar
[10:04] yrashk	yeah this is exactly what I am going to do
[10:04] yrashk	sprintf thread id
[10:05] parik	I guesss Context is represeneting application.I mean to say Context shall be think w.r.t. application
[10:06] parik	In application context
[10:09] yrashk	fine, done :) another question is when exactly msg's ffn will be called? I didn't really get this from the man (3)
[10:09] yrashk	should I look deeper into the manual?
[10:09] mikko	parik: not quite i would say
[10:09] mikko	parik: context is more of a "operating context" for 0mq
[10:10] mikko	let's say you have two libraries, libxy and libzz and both of them use 0mq internally
[10:10] mikko	most likely each one of these libraries would have their own context
[10:10] mikko	and you could happily use both libraries in a project without having to worry about collisions in inproc names etc
[10:10] mikko	that's at least how i see it
[10:11] yrashk	I suspect I have occasional test failures because I don't quite understand when msg data is deallocated :S
[10:12] mikko	yrashk: does this lead to a specific question?
[10:12] yrashk	yep, the one above :)
[10:12] yrashk	"another question is when exactly msg's ffn will be called? I didn't really get this from the man (3)"
[10:12] yrashk	"should I look deeper into the manual?"
[10:14] mikko	yrashk: the ffn is called when the message refcount is 0 and the message is not shared
[10:15] mikko	yrashk: https://github.com/zeromq/zeromq2/blob/master/src/zmq.cpp#L126
[10:15] yrashk	thanks
[10:15] yrashk	mm I think I know what causes those occasional failures
[10:23] yrashk	yup, nothing to do with zeromq (almost, rather with its awesomeness :)
[10:32] yrashk	although I am surprised it didn't gain me any extra performance comparing to pthread_cond message delivery method. Well, may be I am doing something wrong
[10:32] parik	Hi Mikko, I agree with you.Thanks for expalining thorugh the scenarios.Sorry for replying late as I was off from the discussion for sometime
[10:48] yrashk	mikko: if I am using ZMQ_NOBLOCK in zmq_send, should I initialize context with at least one thread? or I am misunderstanding how NOBLOCK works for inproc?
[10:48] yrashk	am
[10:49] yrashk	*am I
[10:57] jsimmons	with inproc, it should be non-blocking anyway yrashk, as far as I can tell
[10:57] mikko	yrashk: sorry, i dont understand the question
[10:58] mikko	jsimmons: zero lock yes but not necessarily non-blocking
[10:59] jsimmons	oh noblock returns failure when it's going to block I see
[11:01] rqmedes	sorry miko was away from keyboard
[11:03] rqmedes	have installed and am playing through c# examples, any pointers on how you run your seperate programs, wcf, services?
[11:18] mikko	rqmedes: i guess you could do a custom wcf binding
[11:18] mikko	as far as i understand
[11:18] mikko	rqmedes: i don't really know anything about windows world
[11:19] mikko	rqmedes: i think you should mail to mailing-list. there are several windows people there
[11:19] mikko	i gotta hop into a meeting ->
[11:23] parik	Thanks alot mikko and guido_g
[11:46] sustrik	jugg: good point
[11:46] sustrik	let me fix it
[11:50] jugg	sustrik, is the idea for subscriptions to be sent to the publisher eventually? Or why is zmq::sub_t::xsetsockopt sending subscriptions as a payload?
[11:51] sustrik	jugg: yes, it's the first step towards subscription forwarding
[11:51] jugg	cool
[11:59] CIA-21	zeromq2: 03Martin Sustrik 07master * r8e0049e 10/ (src/sub.cpp src/sub.hpp):
[11:59] CIA-21	zeromq2: Disable sending messages on SUB socket
[11:59] CIA-21	zeromq2: The ability was inherited from XSUB socket.
[11:59] CIA-21	zeromq2: Now it's properly disabled.
[11:59] CIA-21	zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/ih6erI
[11:59] sustrik	jugg: done
[12:01] rqmedes	thanks mikko
[12:01] jugg	thanks
[12:10] yrashk	hmm, caught a rare segfault in recv, https://gist.github.com/c985ee3760af3aadaf96
[12:10] yrashk	(2.1.0)
[12:20] sustrik	yrashk: is it reproducible
[12:20] sustrik	?
[12:20] yrashk	sustrik: rarely â running some indirect stress tests
[12:21] sustrik	can you possible compile 0mq with debug enabled?
[12:21] sustrik	so that backtrace is more useful?
[12:21] sustrik	./configure --enable-debug
[12:21] yrashk	sure, may be not tonight, but will do
[12:21] sustrik	sure, take your time
[12:22] yrashk	hopefully its nothing serious
[12:22] yrashk	or should I switch to master?
[12:26] sustrik	wait a sec, i'm going to apply one pending patch
[12:29] CIA-21	zeromq2: 03Dhammika Pathirana 07master * rbabdf48 10/ src/pipe.cpp :
[12:29] CIA-21	zeromq2: Fix pipe writer termination
[12:29] CIA-21	zeromq2: Signed-off-by: Dhammika Pathirana <dhammika@gmail.com> - http://bit.ly/gNG05Z
[12:29] sustrik	yrashk: ok, done
[12:30] sustrik	you can use the trunk, there are couple of patches there
[12:30] sustrik	that may (or may not) fix your problem
[12:30] yrashk	hmmm this patchh is for PUSH/PULL sockets, am I right?
[12:30] sustrik	it's generic
[12:30] sustrik	all socket types
[12:30] yrashk	oh ok
[12:31] yrashk	I am new to actual zeromq usage (knew about it for quite some time, though) âÂ just trying to switch my event loop from pthread_cond kind of loop
[12:31] yrashk	(although I naively expected to get some performance gains from this transition, but it didn't happen :)
[12:32] sustrik	you should get the performance gain once the batching kicks in
[12:32] yrashk	at least it simplified my code
[12:32] sustrik	i.e. sending many messages using a single operation
[12:32] yrashk	when it is going to kick in?
[12:32] yrashk	to be kicked in*
[12:33] sustrik	with high message load
[12:33] yrashk	hmm I do have a high message load in my stress test
[12:33] yrashk	I guess it depends on what we understand by high essage load
[12:34] sustrik	dunno, million a second or somesuch
[12:34] yrashk	mm I have less, I guess
[12:35] yrashk	most likely my performance issues aren't in event loop algorithm (so either cond or zeromq get the same bottleneck down the road)
[12:35] sustrik	that's quite likely
[12:35] sustrik	as both of them are pretty much optimised
[12:35] yrashk	I would assume so
[12:35] yrashk	still with zeromq my code is much cleaner
[12:36] yrashk	helped me to drop message queueing/pooling thing of my own
[12:38] yrashk	and hopefully will help more down the road
[12:38] codebeaker	what's the practical difference on Github between the @zeqomq user, and the repository of @PatrickCheng? They both seem pretty much the same, on the surface - but Cheng's includes useful snips for using ÃMQ in a CMake project
[12:40] sustrik	i think he just made the cmake build
[12:40] sustrik	not 100% sure though
[12:40] sustrik	check the project history
[12:44] codebeaker	thanks sustrik - maynbe I'll fork both (running) and diff the whole trees, but his cmake work looks good
[12:45] codebeaker	appreciate teh answer to a dumb question!
[12:45] sustrik	:)
[14:17] codebeaker	hey everyone
[14:26] keffo	hey ho
[14:44] codebeaker	was wondering if anyone could lend me a hand, I'm trying to include Ã¸mq in my project - I build it with CMake, and I would like a portable "FindÃMQ.cmake" â I found it in Patrick Chen's github repo, but ZMQ_DIR is always unsetâ¦ I wondered if I'm barking up the wrong tree - and should simply hard-code the paths?
[14:45] codebeaker	right now, I have the maint branch from the @zeromq repository built and installed the defailt location, and a as a submodule in my project
[14:49] zchrish	Hi.
[14:49] codebeaker	â¦ easy problem really, the linker isn't finding the library - not all together surprising, but I'd like a portable solution
[14:51] zchrish	I have downloaded zeromq to my OpenSuse box and compiled the examples. I modified the mtserver and hwclient to suppress the printf and increased the message length to 80 bytes. When I run the program and pass 1000000 messages into it, it seems like the software works at around 7,500 messages per second. This seems low; any pointers?
[14:55] zchrish	AAA
[14:55] keffo	?
[14:55] zchrish	I have downloaded zeromq to my OpenSuse box and compiled the examples. I modified the mtserver and hwclient to suppress the printf and increased the message length to 80 bytes. When I run the program and pass 1000000 messages into it, it seems like the software works at around 7,500 messages per second. This seems low; any pointers?
[15:13] sustrik	zchrish: are you using local_thr & remote_thr to measure the throughput?
[16:43] zchrish	I am just using the code out-of-the-box. Not sure how to answer the question; I am new to zeromq
[16:47] zchrish	sustrik: I am measuring on the same machine.
[17:04] sustrik	zchrish: how have you get the number?
[17:04] sustrik	7500?
[17:13] zchrish	sustrik: (1) started mtserver, (2) time hwclient
[17:14] zchrish	sustrik: both mtserver and hwclient were stripped of printf statements and the sleep(1) was removed from mtserver and the number of iterations was increased to 1 million
[17:14] zchrish	runtime / # iterations was about 7,500.
[17:15] zchrish	sorry; other way around.
[17:15] sustrik	that's something from the guide?
[17:15] sustrik	anyway, try the offical perf tests:
[17:15] sustrik	http://www.zeromq.org/results:perf-howto
[17:16] zchrish	yes, in the c++ section. I will look at perf-howto; thank you.
[18:27] zchrish	sustrik: OK, I ran these programs and the throughput is much higher. I will look more closely into how mtserver and hwclient are constructed to see how I can modify them to improve the throughput of these programs.