ZeroMq IRC Log

Monday November 29, 2010

[Time] Name	Message
[03:09] Steve-o	master is broken in msvc2010
[04:26] mgc	Steve-o: ping
[04:55] swills	hi, i'm trying to port the perl bindings for zeromq to FreeBSD, but make test is failing:
[04:55] swills	# Error: Can't load '/usr/local/tinderbox/portstrees/FreeBSD/ports/net/p5-ZeroMQ/work/ZeroMQ-0.02_02/blib/arch/auto/ZeroMQ/ZeroMQ.so' for module ZeroMQ: /usr/local/lib/libzmq.so.0: Undefined symbol "pthread_create" at /usr/local/lib/perl5/5.10.1/mach/DynaLoader.pm line 200.
[04:55] swills	pardon the aweful paths...
[04:55] swills	so i'm guessing my problem is zmq is built threaded but my perl isn't. that sound right?
[05:00] Steve-o	you need -pthread on the ZeroMQ.so build?
[05:01] Steve-o	mgc: pong
[05:01] mgc	Steve-o: what do the symptoms of https://github.com/zeromq/zeromq2/issues#issue/128 and https://github.com/zeromq/zeromq2/issues#issue/104 look like?
[05:01] mgc	am seeing epgm subscribers dying under load
[05:02] mgc	and/or losing my mind
[05:02] Steve-o	128 is pretty bad if you have a low rate limit
[05:02] Steve-o	I saw performance drop from say 80mb/s to 800kb/s
[05:03] swills	ah, yes -lpthread makes the test work much better, thanks.
[05:03] Steve-o	104 is to cover a special case if you have multiple senders on one host, it is impossible for them both to receive unicast NAKs
[05:04] Steve-o	so you end up with unrecoverable data loss
[05:04] Steve-o	if you actually see any data loss that is
[05:04] mgc	do things continue working after unrecoverable loss?
[05:05] mgc	I see data loss, but then it is like the subscriber has been blacklisted
[05:05] mgc	and it never gets anything ever again
[05:05] mgc	even if the load subsides
[05:11] Steve-o	zeromq will close the socket and try reconnecting later similar to TCP
[05:13] Steve-o	if the receiver is too slow you are pretty much never going to see anything
[05:30] swills	and the make test just hangs... great...
[05:31] swills	and the version from github needs a bunch of additional modules just to build... oh well..
[09:39] mikko	howdy
[09:39] mikko	mato: there?
[09:41] sustrik	mikko: btw, the patch you've sent some time ago
[09:41] sustrik	there was some discussion about it
[09:41] mikko	sustrik__: yes, still going on
[09:41] sustrik	ah, good
[09:41] mikko	sustrik__: i've been refactoring some stuff lately
[09:42] sustrik	i was not sure
[09:42] mikko	sustrik__: waiting for mato to look if he is ok
[09:42] mikko	sustrik__: the patch for checking if compiler supports visibility breaks in my local env
[09:42] sustrik	ok, i won't apply it
[09:42] mikko	i just need to figure out a way to break this refactoring into patches
[09:43] mikko	https://github.com/mkoppanen/zeromq2/commit/b797da17f788534298e7cb6101c9c7dccb006ee5
[09:43] mikko	currently it's one large blob of misc stuff
[09:43] mato	mikko: yes?
[09:43] mikko	mato: the LT_COMPILER_FLAG breaks here for some reason
[09:44] mikko	maybe autoconf version differences
[09:44] mato	mikko: autoconf or libtool, it's possible
[09:44] mikko	i had two days off last week and i wrote a couple of macros
[09:44] mato	mikko: but the autobuilds work ...
[09:44] mato	mikko: so what system does it break for you on?
[09:44] mikko	mato: debian stable
[09:44] mikko	daily builds are on debian testing
[09:44] mato	mikko: oh, interesting
[09:45] mikko	mato: take a look at this https://github.com/mkoppanen/zeromq2/commit/b797da17f788534298e7cb6101c9c7dccb006ee5#diff-1
[09:45] mato	mikko: i'm also on debian stable, but have a backported libtool 2.x
[09:45] mikko	i added a couple of maybe useful macros
[09:45] mikko	AC_ZMQ_CHECK_LANG_FLAG checks if compiler supports given flag
[09:45] mikko	AC_ZMQ_CHECK_LANG_PRAGMA checks if compiler supports given pragma
[09:45] mikko	AC_ZMQ_CHECK_LANG_FLAG_PREPEND is same as AC_ZMQ_CHECK_LANG_FLAG but it prepends to CFLAGS or CPPFLAGS
[09:46] mikko	also added macros for WERROR and WALL for different compilers
[09:49] mato	mikko: wow, that looks good
[09:49] mato	mikko: one problem
[09:49] mikko	mato: it moves things more into direction that it checks if the flag is supported
[09:50] mikko	rather than just adding
[09:50] mato	mikko: your changes to zmq.h/zmq_utils.h don't make sense
[09:50] mato	mikko: because a) ZMQ_HAVE_<anything> is not present and can't be tested against in zmq.h/zmq_utils.h
[09:50] mikko	mato: i can revert that
[09:50] mikko	mato: that breaks clang on daily builds
[09:50] mikko	the visibility
[09:50] mato	i know
[09:50] mato	the problem is...
[09:50] mikko	it poses as gcc
[09:51] mikko	without supporting gcc things
[09:51] mato	yes, and does a bad job :-)
[09:51] mikko	clang 2.2+ supports visibility ok
[09:51] mato	well, the problem is, AFAIK the visibility bits should be in the function declarations also
[09:51] mikko	i wonder if this is a non-issue
[09:52] mikko	the daily builds run with clang 1.1 (age old debian shipped version)
[09:52] mato	I'm not 100% sure about it, but Drepper's piece on shared libraries claims so and so does MSFT for __declspec
[09:52] mato	I reckon it's a non-issue
[09:52] mikko	i could upgrade clang on daily build box
[09:52] mato	If clang is posing as GCC, then it better do that properly
[09:52] mikko	to a version from this century :)
[09:52] mato	yeah
[09:53] CIA-20	zeromq2: 03Steven McCoy 07master * r28db150 10/ src/thread.cpp :
[09:53] CIA-20	zeromq2: Fix thread thunker scope for MSVC.
[09:53] CIA-20	zeromq2: Signed-off-by: Steven McCoy <steven.mccoy@miru.hk> - http://bit.ly/hIM3js
[09:53] mikko	so, only check for -fvisibility compiler flag is needed
[09:53] mato	anyhow, I like the overall idea (test for XXXX being supported, rather than enumerating all the combinations where it might/might not work)
[09:53] mikko	i wrote a bashrc for myself to switch compilers / build zeromq for testing :)
[09:53] mikko	built probably 100+ times during the weekend
[09:53] mato	reading further thru the patch ...
[09:54] mato	it's not obvious what the default outcome (and others) of AC_ZMQ_CHECK_ENABLE_DEBUG
[09:54] mato	is
[09:54] mato	i.e. the comment block should probably say what the intended result is with the various autoconf settings that may affect that
[09:55] mikko	it should also support [action-if-true], [action-if-false]
[09:55] mikko	just for reusability
[09:55] mikko	so that these macros can be used for example in ZFL build as well
[09:55] mato	sure
[09:56] mato	there's some bogus indentation in AC_ZMQ_CHECK_COMPILERS
[09:57] mato	why the different CHECK_LANG_FLAG and CHECK_LANG_FLAG_PREPEND?
[09:57] mikko	mato: becuase the former takes a copy of CFLAGS
[09:57] mato	and there's a typo in the comment on line 370
[09:58] mikko	if you modify them in action-if-found its hard to know what flags have been modified
[09:58] mato	well, you say in the comment blocks to both not to do that anyway :-)
[09:58] mikko	yeah, thats why there is the prepend on
[09:58] mikko	allows you to prepend to flags easily
[09:59] mato	um, so the non-prepend just checks for support but leaves the resulting flags alone?
[09:59] mikko	yes
[09:59] mato	vs. the prepend that actually changes the resulting flags?
[09:59] mikko	yes, it prepends to CFLAGS or CPPFLAGS
[09:59] mato	ok, that was not obvious from reading the comment block or the function name
[09:59] mikko	whatever is the given language
[10:00] mikko	i think i need to comment them all more clearly
[10:00] mato	I haven't had enough coffee to attempt reading the actual Autoconf code :-)
[10:01] mato	anyway, thanks for this, your patience with autoconf is impressive
[10:02] mato	mikko: i'd say just submit the whole lot as a single patch once you're done reviewing
[10:02] mato	mikko: just list all the separate bits in the commit message
[10:02] mikko	mato: ok
[10:02] mato	it's all build-related anyway, so no point in splitting it up
[10:03] mikko	cool, i'll form a patch during today
[10:04] mikko	im beginning to like sun studio as a compiler
[10:04] mikko	it seems to be the strictest of them all
[10:04] mikko	when you enable -compat=5 +w -errwarn=%all (equals to -pedantic -Wall -Werror)
[10:10] sustrik	mato: what about the DSO visibility patch?
[10:10] sustrik	should i apply it?
[10:11] mato	sustrik__: mikko's work in progress supersedes that
[10:11] sustrik	ok
[10:12] mato	mikko: although... regarding the visibility stuff, we still have no way of testing ZMQ_HAVE_VISIBILITY in zmq.h
[10:13] mikko	mato: i think we can remove that part
[10:13] mikko	and just check for -fvisibility and rely on gcc version
[10:13] mikko	clang 2.2+ handles that properly
[10:13] mato	mikko: yup, but keep the AC_ZMQ_CHECK_LANG_PRAGMA macro around anyway, it'll come in handy
[10:14] mikko	mato: yeah, i thought that too
[10:14] mato	mikko: incidentally, if you look in the openpgm source it also does symbol visibility for sun studio using their syntax
[10:14] mato	mikko: so you can try and make that work if you feel like it
[10:14] mikko	mato: sure, i am getting into this autoconf stuff
[10:14] mato	mikko: the objective is that the DSO only exports the actual zmq_* API functions and nothing else.
[10:15] mato	mikko: there are also various other ways to do it, but they're all more annoying (linker map files, etc.) so I picked -fvisibility as the least invasive approach for now
[10:17] mikko	mato: http://developers.sun.com/solaris/articles/symbol_scope.html
[10:18] mikko	i think i need to define ZMQ_EXPORT to __global using sun studio
[10:18] mato	mikko: hmm, right
[10:19] mato	mikko: in that case, we could skip the pragma and use the __attribute__ ((visibility("default")))
[10:19] mato	mikko: (for GCC)
[10:19] mato	mikko: see e.g. http://gcc.gnu.org/wiki/Visibility
[10:20] mato	mikko: anyhow, you can verify the result both by running nm src/.libs/libzmq.so \| grep ' T '
[10:20] mato	mikko: which should show only zmq_* (plus the _init , _fini and internal bits)
[10:23] mato	mikko: and/or readelf --use-dynamic --syms src/.libs/libzmq.so
[10:23] mikko	i think -fvisibility=hidden works with ICC as well
[10:24] mikko	http://software.intel.com/sites/products/documentation/studio/composer/en-us/2011/compiler_c/optaps/common/optaps_cmp_visib.htm
[10:24] mikko	it also supports __attribute__ ((visibility ("default")));
[11:21] kobus	Hi, I would appreciate some pointers on problem troubleshooting. I'm running the weather update server and client (C++ code) and the server is sending, but the client is not receiving any messages.
[11:21] kobus	I'm building the code on Windows with MinGW
[11:35] mikko	kobus_: sure
[11:35] mikko	kobus_: is this PUB/SUB?
[11:36] kobus	mikko: Yes. I couldn't get my code to work, so I resorted to getting samples to work.
[11:38] mikko	kobus_: did you remember to subscribe to topics on the SUB side?
[11:42] kobus	The client conneccts directly to the server. https://github.com/imatix/zguide/blob/master/examples/C%2B%2B/wuclient.cpp
[11:42] kobus	mikko: I think...
[11:43] mikko	kobus_: are you publishing in the topic ?
[11:48] kobus	mikko: it is a client-server example
[11:57] kobus	mikko: I'm not sure how you publish in a topic. This example doesn't make use of a broker; looks like a tcp client & server to me
[12:10] kobus	mikko: as I understand this is the subscription: subscriber.setsockopt(ZMQ_SUBSCRIBE, filter, strlen (filter));
[12:14] mikko	kobus_: yes
[12:15] mikko	kobus_: the message you publish needs to start with that prefix
[12:19] kobus	mikko: I changed the server to prefix the message with the same "zipcode" as the client subscribes to and now it works. Thanks!
[12:24] mikko	kobus_: no problem
[22:33] magicblaze0071	I'm trying to get a file transferred between two machines behind nat (without loading up a central server too much). Can this be done using zeromq?
[22:34] magicblaze0071	is anyone here using zeromq on windows with python?
[23:08] mikko	magicblaze0071: your question is one layer below zeromq
[23:09] mikko	magicblaze0071: it sounds more like a routing/networking questions
[23:09] mikko	magicblaze0071: if you use tcp:// transport for example the semantics for the transfer are the same as with tcp socket
[23:09] mikko	magicblaze0071: not sure about windows python zmq users
[23:09] mikko	do you have a specific problem?
[23:38] magicblaze0071	mikko: My first problem is installation on a windows 7 machine
[23:38] magicblaze0071	I tried to copy the dll file in src\zmq directory and then point setup.cfg into that directory...but couldnt compile it.
[23:39] magicblaze0071	I would prefer to get a copy that I dont have to compile myself, because i've to ship code to my users who cant compile things
[23:41] Guthur	magicblaze0071, you get the dll from the zmq/lib directory
[23:41] Guthur	follow the instructions for building libzmq
[23:42] Guthur	then grab the dll, and follow the instructions for building pyzmq
[23:42] Guthur	I haven't used the pyzmq binding so beyond that I can't say
[23:43] Guthur	There should be no reason for you to require your users to install libzmq, the dll is all you need
[23:47] magicblaze0071	Guthur: what I dont know is, once i get the .dll file, how to import it in python...can i just say "import zmq"?
[23:47] Guthur	I'm not sure of pythons exact process for foreign func calls
[23:48] Guthur	but pyzmq 'should' take care of it
[23:48] magicblaze0071	it seems that pyzmq needs a C/cpp compiler to link the dll to something else perhaps?
[23:48] magicblaze0071	I've the pyzmq and the dll in the pyzmq src/zmq directory.
[23:48] magicblaze0071	It seems pyzmq install needs the compilers as well...
[23:48] Guthur	have you built pyzmq?
[23:49] magicblaze0071	no
[23:49] magicblaze0071	but if i build it, wont i have to build it on my users machines as well? or can i copy some files ?
[23:49] Guthur	well then you aren't going to get very far, hehe
[23:49] Guthur	you need to build it
[23:49] Guthur	nope you should get some sort of python lib I would imagine
[23:50] magicblaze0071	k, say i am able to build it, how do i ship the built copy?
[23:50] Guthur	it should be just a python lib, and you would most likely need to include the libzmq.dll as well
[23:50] magicblaze0071	Hasnt anyone here done that already? built pyzmq on windows? I was hoping to get the prebuilt files so that i cud ship the same thing to my users.
[23:51] Guthur	to be honest I'm surprised you haven't built it already
[23:52] magicblaze0071	k, lemme build it
[23:52] Guthur	that's the spirit, hehe