Monday November 29, 2010

[Time] NameMessage
[03:09] Steve-o master is broken in msvc2010
[04:26] mgc Steve-o: ping
[04:55] swills hi, i'm trying to port the perl bindings for zeromq to FreeBSD, but make test is failing:
[04:55] swills # Error: Can't load '/usr/local/tinderbox/portstrees/FreeBSD/ports/net/p5-ZeroMQ/work/ZeroMQ-0.02_02/blib/arch/auto/ZeroMQ/' for module ZeroMQ: /usr/local/lib/ Undefined symbol "pthread_create" at /usr/local/lib/perl5/5.10.1/mach/ line 200.
[04:55] swills pardon the aweful paths...
[04:55] swills so i'm guessing my problem is zmq is built threaded but my perl isn't. that sound right?
[05:00] Steve-o you need -pthread on the build?
[05:01] Steve-o mgc: pong
[05:01] mgc Steve-o: what do the symptoms of and look like?
[05:01] mgc am seeing epgm subscribers dying under load
[05:02] mgc and/or losing my mind
[05:02] Steve-o 128 is pretty bad if you have a low rate limit
[05:02] Steve-o I saw performance drop from say 80mb/s to 800kb/s
[05:03] swills ah, yes -lpthread makes the test work much better, thanks.
[05:03] Steve-o 104 is to cover a special case if you have multiple senders on one host, it is impossible for them both to receive unicast NAKs
[05:04] Steve-o so you end up with unrecoverable data loss
[05:04] Steve-o if you actually see any data loss that is
[05:04] mgc do things continue working after unrecoverable loss?
[05:05] mgc I see data loss, but then it is like the subscriber has been blacklisted
[05:05] mgc and it never gets anything ever again
[05:05] mgc even if the load subsides
[05:11] Steve-o zeromq will close the socket and try reconnecting later similar to TCP
[05:13] Steve-o if the receiver is too slow you are pretty much never going to see anything
[05:30] swills and the make test just hangs... great...
[05:31] swills and the version from github needs a bunch of additional modules just to build... oh well..
[09:39] mikko howdy
[09:39] mikko mato: there?
[09:41] sustrik mikko: btw, the patch you've sent some time ago
[09:41] sustrik there was some discussion about it
[09:41] mikko sustrik__: yes, still going on
[09:41] sustrik ah, good
[09:41] mikko sustrik__: i've been refactoring some stuff lately
[09:42] sustrik i was not sure
[09:42] mikko sustrik__: waiting for mato to look if he is ok
[09:42] mikko sustrik__: the patch for checking if compiler supports visibility breaks in my local env
[09:42] sustrik ok, i won't apply it
[09:42] mikko i just need to figure out a way to break this refactoring into patches
[09:43] mikko
[09:43] mikko currently it's one large blob of misc stuff
[09:43] mato mikko: yes?
[09:43] mikko mato: the LT_COMPILER_FLAG breaks here for some reason
[09:44] mikko maybe autoconf version differences
[09:44] mato mikko: autoconf or libtool, it's possible
[09:44] mikko i had two days off last week and i wrote a couple of macros
[09:44] mato mikko: but the autobuilds work ...
[09:44] mato mikko: so what system does it break for you on?
[09:44] mikko mato: debian stable
[09:44] mikko daily builds are on debian testing
[09:44] mato mikko: oh, interesting
[09:45] mikko mato: take a look at this
[09:45] mato mikko: i'm also on debian stable, but have a backported libtool 2.x
[09:45] mikko i added a couple of maybe useful macros
[09:45] mikko AC_ZMQ_CHECK_LANG_FLAG checks if compiler supports given flag
[09:45] mikko AC_ZMQ_CHECK_LANG_PRAGMA checks if compiler supports given pragma
[09:45] mikko AC_ZMQ_CHECK_LANG_FLAG_PREPEND is same as AC_ZMQ_CHECK_LANG_FLAG but it prepends to CFLAGS or CPPFLAGS
[09:46] mikko also added macros for WERROR and WALL for different compilers
[09:49] mato mikko: wow, that looks good
[09:49] mato mikko: one problem
[09:49] mikko mato: it moves things more into direction that it checks if the flag is supported
[09:50] mikko rather than just adding
[09:50] mato mikko: your changes to zmq.h/zmq_utils.h don't make sense
[09:50] mato mikko: because a) ZMQ_HAVE_<anything> is not present and can't be tested against in zmq.h/zmq_utils.h
[09:50] mikko mato: i can revert that
[09:50] mikko mato: that breaks clang on daily builds
[09:50] mikko the visibility
[09:50] mato i know
[09:50] mato the problem is...
[09:50] mikko it poses as gcc
[09:51] mikko without supporting gcc things
[09:51] mato yes, and does a bad job :-)
[09:51] mikko clang 2.2+ supports visibility ok
[09:51] mato well, the problem is, AFAIK the visibility bits should be in the function declarations also
[09:51] mikko i wonder if this is a non-issue
[09:52] mikko the daily builds run with clang 1.1 (age old debian shipped version)
[09:52] mato I'm not 100% sure about it, but Drepper's piece on shared libraries claims so and so does MSFT for __declspec
[09:52] mato I reckon it's a non-issue
[09:52] mikko i could upgrade clang on daily build box
[09:52] mato If clang is posing as GCC, then it better do that properly
[09:52] mikko to a version from this century :)
[09:52] mato yeah
[09:53] CIA-20 zeromq2: 03Steven McCoy 07master * r28db150 10/ src/thread.cpp :
[09:53] CIA-20 zeromq2: Fix thread thunker scope for MSVC.
[09:53] CIA-20 zeromq2: Signed-off-by: Steven McCoy <> -
[09:53] mikko so, only check for -fvisibility compiler flag is needed
[09:53] mato anyhow, I like the overall idea (test for XXXX being supported, rather than enumerating all the combinations where it might/might not work)
[09:53] mikko i wrote a bashrc for myself to switch compilers / build zeromq for testing :)
[09:53] mikko built probably 100+ times during the weekend
[09:53] mato reading further thru the patch ...
[09:54] mato it's not obvious what the default outcome (and others) of AC_ZMQ_CHECK_ENABLE_DEBUG
[09:54] mato is
[09:54] mato i.e. the comment block should probably say what the intended result is with the various autoconf settings that may affect that
[09:55] mikko it should also support [action-if-true], [action-if-false]
[09:55] mikko just for reusability
[09:55] mikko so that these macros can be used for example in ZFL build as well
[09:55] mato sure
[09:56] mato there's some bogus indentation in AC_ZMQ_CHECK_COMPILERS
[09:57] mato why the different CHECK_LANG_FLAG and CHECK_LANG_FLAG_PREPEND?
[09:57] mikko mato: becuase the former takes a copy of CFLAGS
[09:57] mato and there's a typo in the comment on line 370
[09:58] mikko if you modify them in action-if-found its hard to know what flags have been modified
[09:58] mato well, you say in the comment blocks to both not to do that anyway :-)
[09:58] mikko yeah, thats why there is the prepend on
[09:58] mikko allows you to prepend to flags easily
[09:59] mato um, so the non-prepend just checks for support but leaves the resulting flags alone?
[09:59] mikko yes
[09:59] mato vs. the prepend that actually changes the resulting flags?
[09:59] mikko yes, it prepends to CFLAGS or CPPFLAGS
[09:59] mato ok, that was not obvious from reading the comment block or the function name
[09:59] mikko whatever is the given language
[10:00] mikko i think i need to comment them all more clearly
[10:00] mato I haven't had enough coffee to attempt reading the actual Autoconf code :-)
[10:01] mato anyway, thanks for this, your patience with autoconf is impressive
[10:02] mato mikko: i'd say just submit the whole lot as a single patch once you're done reviewing
[10:02] mato mikko: just list all the separate bits in the commit message
[10:02] mikko mato: ok
[10:02] mato it's all build-related anyway, so no point in splitting it up
[10:03] mikko cool, i'll form a patch during today
[10:04] mikko im beginning to like sun studio as a compiler
[10:04] mikko it seems to be the strictest of them all
[10:04] mikko when you enable -compat=5 +w -errwarn=%all (equals to -pedantic -Wall -Werror)
[10:10] sustrik mato: what about the DSO visibility patch?
[10:10] sustrik should i apply it?
[10:11] mato sustrik__: mikko's work in progress supersedes that
[10:11] sustrik ok
[10:12] mato mikko: although... regarding the visibility stuff, we still have no way of testing ZMQ_HAVE_VISIBILITY in zmq.h
[10:13] mikko mato: i think we can remove that part
[10:13] mikko and just check for -fvisibility and rely on gcc version
[10:13] mikko clang 2.2+ handles that properly
[10:13] mato mikko: yup, but keep the AC_ZMQ_CHECK_LANG_PRAGMA macro around anyway, it'll come in handy
[10:14] mikko mato: yeah, i thought that too
[10:14] mato mikko: incidentally, if you look in the openpgm source it also does symbol visibility for sun studio using their syntax
[10:14] mato mikko: so you can try and make that work if you feel like it
[10:14] mikko mato: sure, i am getting into this autoconf stuff
[10:14] mato mikko: the objective is that the DSO only exports the actual zmq_* API functions and nothing else.
[10:15] mato mikko: there are also various other ways to do it, but they're all more annoying (linker map files, etc.) so I picked -fvisibility as the least invasive approach for now
[10:17] mikko mato:
[10:18] mikko i think i need to define ZMQ_EXPORT to __global using sun studio
[10:18] mato mikko: hmm, right
[10:19] mato mikko: in that case, we could skip the pragma and use the __attribute__ ((visibility("default")))
[10:19] mato mikko: (for GCC)
[10:19] mato mikko: see e.g.
[10:20] mato mikko: anyhow, you can verify the result both by running nm src/.libs/ | grep ' T '
[10:20] mato mikko: which should show only zmq_* (plus the _init , _fini and internal bits)
[10:23] mato mikko: and/or readelf --use-dynamic --syms src/.libs/
[10:23] mikko i think -fvisibility=hidden works with ICC as well
[10:24] mikko
[10:24] mikko it also supports __attribute__ ((visibility ("default")));
[11:21] kobus Hi, I would appreciate some pointers on problem troubleshooting. I'm running the weather update server and client (C++ code) and the server is sending, but the client is not receiving any messages.
[11:21] kobus I'm building the code on Windows with MinGW
[11:35] mikko kobus_: sure
[11:35] mikko kobus_: is this PUB/SUB?
[11:36] kobus mikko: Yes. I couldn't get my code to work, so I resorted to getting samples to work.
[11:38] mikko kobus_: did you remember to subscribe to topics on the SUB side?
[11:42] kobus The client conneccts directly to the server.
[11:42] kobus mikko: I think...
[11:43] mikko kobus_: are you publishing in the topic ?
[11:48] kobus mikko: it is a client-server example
[11:57] kobus mikko: I'm not sure how you publish in a topic. This example doesn't make use of a broker; looks like a tcp client & server to me
[12:10] kobus mikko: as I understand this is the subscription: subscriber.setsockopt(ZMQ_SUBSCRIBE, filter, strlen (filter));
[12:14] mikko kobus_: yes
[12:15] mikko kobus_: the message you publish needs to start with that prefix
[12:19] kobus mikko: I changed the server to prefix the message with the same "zipcode" as the client subscribes to and now it works. Thanks!
[12:24] mikko kobus_: no problem
[22:33] magicblaze0071 I'm trying to get a file transferred between two machines behind nat (without loading up a central server too much). Can this be done using zeromq?
[22:34] magicblaze0071 is anyone here using zeromq on windows with python?
[23:08] mikko magicblaze0071: your question is one layer below zeromq
[23:09] mikko magicblaze0071: it sounds more like a routing/networking questions
[23:09] mikko magicblaze0071: if you use tcp:// transport for example the semantics for the transfer are the same as with tcp socket
[23:09] mikko magicblaze0071: not sure about windows python zmq users
[23:09] mikko do you have a specific problem?
[23:38] magicblaze0071 mikko: My first problem is installation on a windows 7 machine
[23:38] magicblaze0071 I tried to copy the dll file in src\zmq directory and then point setup.cfg into that directory...but couldnt compile it.
[23:39] magicblaze0071 I would prefer to get a copy that I dont have to compile myself, because i've to ship code to my users who cant compile things
[23:41] Guthur magicblaze0071, you get the dll from the zmq/lib directory
[23:41] Guthur follow the instructions for building libzmq
[23:42] Guthur then grab the dll, and follow the instructions for building pyzmq
[23:42] Guthur I haven't used the pyzmq binding so beyond that I can't say
[23:43] Guthur There should be no reason for you to require your users to install libzmq, the dll is all you need
[23:47] magicblaze0071 Guthur: what I dont know is, once i get the .dll file, how to import it in python...can i just say "import zmq"?
[23:47] Guthur I'm not sure of pythons exact process for foreign func calls
[23:48] Guthur but pyzmq 'should' take care of it
[23:48] magicblaze0071 it seems that pyzmq needs a C/cpp compiler to link the dll to something else perhaps?
[23:48] magicblaze0071 I've the pyzmq and the dll in the pyzmq src/zmq directory.
[23:48] magicblaze0071 It seems pyzmq install needs the compilers as well...
[23:48] Guthur have you built pyzmq?
[23:49] magicblaze0071 no
[23:49] magicblaze0071 but if i build it, wont i have to build it on my users machines as well? or can i copy some files ?
[23:49] Guthur well then you aren't going to get very far, hehe
[23:49] Guthur you need to build it
[23:49] Guthur nope you should get some sort of python lib I would imagine
[23:50] magicblaze0071 k, say i am able to build it, how do i ship the built copy?
[23:50] Guthur it should be just a python lib, and you would most likely need to include the libzmq.dll as well
[23:50] magicblaze0071 Hasnt anyone here done that already? built pyzmq on windows? I was hoping to get the prebuilt files so that i cud ship the same thing to my users.
[23:51] Guthur to be honest I'm surprised you haven't built it already
[23:52] magicblaze0071 k, lemme build it
[23:52] Guthur that's the spirit, hehe