IRC Log


Wednesday December 1, 2010

[Time] NameMessage
[00:03] Nate75Sanders I'm using python (I don't think that would matter, but thought I'd mention) and when I use a PUSH/PULL pair _without_ HWM, the send() calls do not block when there is no PULLer. Is this correct behavior?
[00:04] Nate75Sanders manpage for zmq_socket reads to me like it should block when there is no downstream node
[00:31] kisielk hm I don't have a ZMQ install here where I can test that, but according to the docs you are correct
[00:34] Nate75Sanders the messages queue up and are delivered later....but there's no blocking behavior
[00:35] Nate75Sanders to me block means "i'm going to sleep until there's a receiver"
[00:45] kisielk agreed
[00:46] kisielk it's either a bug in the code or documentation
[00:46] kisielk you should report it
[00:48] Nate75Sanders I should probably take a look at the mailing list....or bug tracker if that's public
[00:48] Nate75Sanders I haven't followed their dev process/mailing list/etc very closely....mostly just playing around
[00:49] kisielk https://github.com/zeromq/zeromq2/issues
[00:49] kisielk file a bug there with example code
[00:50] Nate75Sanders thanks
[00:55] Nate75Sanders going to make _sure_ that I'm using latest stuff, etc first, though
[01:00] kisielk yeah that's a good idea
[01:01] kisielk although I don't see any closed issues related to push blocking
[01:18] Nate75Sanders heh, i have some serious formatting problems
[01:19] Nate75Sanders editing the issue
[01:21] Nate75Sanders .....and I don't get GitHub's markup
[01:21] Nate75Sanders ugh
[04:16] Nate75Sanders anybody here?
[04:18] Nate75Sanders appreciate it if someone would take a look at https://github.com/zeromq/zeromq2/issues#issue/130 and see if I'm crazy or not
[05:05] the_hulk hi, i an trying to make a simple chat server-client app, zmq_msg_init_size on server site, caused termination saying "Illegal Instruction", what could this mean? any help?
[05:40] guido_g Nate75Sanders: no HWM means: queue all messages as long as memory permits.
[05:41] Nate75Sanders guido_g: Fair enough, but that means the documentation is wrong.
[05:41] guido_g Nate75Sanders: i think the "no downstream nodes" only applies when there _never_ were one
[05:42] guido_g because if there was a connection, then there is a queue and the hwm applies
[05:42] Nate75Sanders guido_g: strange....I definitely think the man page should describe this in more detail
[05:42] guido_g than attach a fix to the issue or post a patch to the ml
[05:42] Nate75Sanders guido_g: could you tell me where you gained this particular understanding of things? so i can read there instead?
[05:43] Nate75Sanders guido_g: I'd like to make _sure_ what you're saying is correct (it sounds like it is).
[05:43] guido_g Nate75Sanders: easy, hang around here for 3 to 4 month, read the things from the core devs that fly by and you'll know :)
[05:44] Nate75Sanders guido_g: understood :)
[05:44] guido_g Nate75Sanders: then write a test case and check what'S going on
[05:44] Nate75Sanders well, i'm halfway there -- based on my bug report
[05:44] Nate75Sanders just need a no-receiver at all case
[05:45] guido_g btw, writing a bug report w/o proof (aka test/show case) isn't worth much
[05:45] Nate75Sanders you know what? actually i think i tested that earlier
[05:45] Nate75Sanders yeah
[05:45] Nate75Sanders i did
[05:45] Nate75Sanders and i just did now, too
[05:46] Nate75Sanders there was never a downstream node
[05:46] Nate75Sanders and it still does NOT block
[05:46] guido_g then attach your test incl. program to the issue, so that it can be verified
[05:46] Nate75Sanders i did
[05:46] Nate75Sanders it's in the bug report
[05:46] Nate75Sanders 2 scripts
[05:46] Nate75Sanders you can run the PUSHer without the PULLer and it refutes what you said earlier
[05:47] Nate75Sanders when you have a PUSHer, if there's no HWM present (whether or not there ever was) the behavior is that it will not block
[05:47] Nate75Sanders sorry, meant to say
[05:48] Nate75Sanders if there's no HWM and no receiver (whether or not there ever was a receiver) the behavior is that it will not block
[05:48] Nate75Sanders so, I'm not sure where you got your info, but it isn't correct
[05:49] Nate75Sanders you can run my code if you don't believe me
[05:49] Nate75Sanders maybe i'm crazy somehow
[05:49] Nate75Sanders but it's in the bug report
[05:50] Nate75Sanders wish sustrik was around....
[05:51] guido_g he will be in an hour or so
[05:51] guido_g he's an early bird
[05:53] guido_g hmm hmm mhmm
[05:53] guido_g if you use bind instead of connect on the push side, it works
[05:54] guido_g i just used "socket.bind("tcp://127.0.0.1:5555")"
[05:54] guido_g and it blocks, as described
[05:55] guido_g and with connect the hwm works too
[05:56] guido_g so it's more a documentation issue on bind vs connect, i'd say
[05:58] Nate75Sanders fair enough
[05:58] guido_g Nate75Sanders: just added that as a comment to your issue
[05:58] Nate75Sanders i had considered that, but not tested it
[05:58] guido_g tz tz tz
[05:58] Nate75Sanders i think in one part of the docks they make a big point to say that bind vs connect shouldn't affect socket semantics
[05:59] Nate75Sanders just the stable vs non-knowable parts of your architecture
[05:59] Nate75Sanders so i really do think that should be dealt with
[05:59] Nate75Sanders but good detective work
[05:59] guido_g before first coffee
[06:00] Nate75Sanders are you in europe?
[06:00] guido_g yes
[06:00] Nate75Sanders italy?
[06:00] guido_g no
[06:00] Nate75Sanders oh :)
[06:00] Nate75Sanders hehe
[06:00] Nate75Sanders I'm in Honolulu
[06:00] Nate75Sanders so it's 8PM here
[06:00] guido_g *sigh*
[06:00] Nate75Sanders "yesterday" :)
[06:00] Nate75Sanders moving soon
[06:01] Nate75Sanders this is a fun place to live, but not good for hackers
[06:01] Nate75Sanders i will miss the weather, though
[06:01] Nate75Sanders truly, truly amazing
[06:01] Nate75Sanders in the winter it's 75 and sunny
[06:01] guido_g given that outside are -7°C i tend to be a bit jealous
[06:01] Nate75Sanders in the summer, it's 85 and sunny
[06:02] guido_g and i need to go to the office soon *shudder*
[06:03] Nate75Sanders What kind of place do you work?
[06:03] guido_g i'm a freelancer, at the moment i'm working at a local bank
[06:04] Nate75Sanders I decided to work for a place that doesn't really do as much code (half code, half sysadmin for an oceanographic science team)
[06:04] Nate75Sanders it's been great
[06:04] guido_g sounds good
[06:04] Nate75Sanders i help them immensely and everything i do is magic
[06:04] Nate75Sanders and i have plenty of free time
[06:04] Nate75Sanders didn't expect this
[06:04] Nate75Sanders i used to work more dev jobs....
[06:04] guido_g sounds even better :)
[06:04] Nate75Sanders they just left me feeling bad
[06:04] Nate75Sanders turning the thing i loved into terrible terrible work
[06:05] Nate75Sanders come home.....hardly interested in the computer....
[06:05] Nate75Sanders now? i'm happy to dive in and work on something
[06:05] Nate75Sanders not tired at the end of a day
[06:05] guido_g that's the price to pay, i know that
[06:05] Nate75Sanders yeah
[06:07] guido_g more coffee
[06:07] Nate75Sanders indeed :)
[06:07] Nate75Sanders don't work too hard today....
[06:07] Nate75Sanders pace yourself
[06:07] guido_g hey, it's a bank
[06:07] Nate75Sanders your 6 hours is easily worth someone else's 9
[06:07] Nate75Sanders deep breaths
[06:07] Nate75Sanders ;)
[06:08] guido_g ahhh...
[06:08] Nate75Sanders mindfulness
[06:08] guido_g i should have mentioned that today is my weekend
[06:08] Nate75Sanders oh, nice
[06:08] guido_g doing only 3 days a week
[06:08] Nate75Sanders sweet
[06:08] guido_g yeah
[06:08] Nate75Sanders hey you have it good, then
[06:09] Nate75Sanders i'm about to go half-time/half-salary myself
[06:09] Nate75Sanders and move
[06:09] guido_g i'm surprised that it worked
[06:09] Nate75Sanders which part?
[06:09] guido_g as i said, it's a bank
[06:09] guido_g the part time thingy
[06:09] Nate75Sanders yeah
[06:09] Nate75Sanders i am surprised too
[06:09] guido_g there not _that_ flexible usually
[06:09] Nate75Sanders yep
[06:09] Nate75Sanders agreed
[06:09] Nate75Sanders my dad runs a bank
[06:10] guido_g ahh so you know a little
[06:11] Nate75Sanders so what do you do with your spare time?
[06:12] guido_g reading funny irc channels on obscure software things, linke brokerless messaging and such
[06:12] guido_g noone will ever need this shit! :)
[06:13] Nate75Sanders haha
[06:13] Nate75Sanders nice
[06:16] guido_g i'd have a project using all this fun stuff -- if i would be better sales guy
[09:21] mikko good morning
[09:27] CIA-20 zeromq2: 03Martin Lucina 07maint * r289b1f5 10/ src/kqueue.cpp :
[09:27] CIA-20 zeromq2: Remove unnecessary cast in kevent_delete
[09:27] CIA-20 zeromq2: Fixes the build on NetBSD where the compiler complains about casting NULL
[09:27] CIA-20 zeromq2: to (int).
[09:27] CIA-20 zeromq2: Signed-off-by: Martin Lucina <mato@kotelna.sk> - http://bit.ly/esACki
[09:27] CIA-20 zeromq2: 03Martin Lucina 07master * r289b1f5 10/ src/kqueue.cpp :
[09:27] CIA-20 zeromq2: Remove unnecessary cast in kevent_delete
[09:27] CIA-20 zeromq2: Fixes the build on NetBSD where the compiler complains about casting NULL
[09:27] CIA-20 zeromq2: to (int).
[09:27] CIA-20 zeromq2: Signed-off-by: Martin Lucina <mato@kotelna.sk> - http://bit.ly/esACki
[09:27] CIA-20 zeromq2: 03Martin Sustrik 07master * r17d7e6e 10/ src/kqueue.cpp :
[09:27] CIA-20 zeromq2: Merge branch 'maint'
[09:28] CIA-20 zeromq2: Remove unnecessary cast in kevent_delete
[09:28] CIA-20 zeromq2: Conflicts:
[09:28] CIA-20 zeromq2: src/kqueue.cpp
[09:28] CIA-20 zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/gqsKEN
[09:34] CIA-20 zeromq2: 03Mikko Koppanen 07master * r5bb0a33 10/ (5 files in 2 dirs): (log message trimmed)
[09:34] CIA-20 zeromq2: Prefix variables with "ac_zmq_"
[09:34] CIA-20 zeromq2: - Added a macro for checking clang compiler
[09:34] CIA-20 zeromq2: - Moved basic compiler checks to a macro
[09:34] CIA-20 zeromq2: - Added a macro for checking if compiler supports a flag
[09:34] CIA-20 zeromq2: - Added --enable-debug flag
[09:34] CIA-20 zeromq2: - Added a macro for running normal autoconf check with compiler flags
[09:47] m exit
[09:58] CIA-20 zeromq2: 03Martin Lucina 07master * rb70d628 10/ (11 files):
[09:58] CIA-20 zeromq2: Documentation updates for 2.1
[09:58] CIA-20 zeromq2: - Clarify ZMQ_LINGER, zmq_close (), zmq_term () relationship
[09:58] CIA-20 zeromq2: - New socket options
[09:58] CIA-20 zeromq2: - Clarify thread safety of sockets and migration between threads
[09:58] CIA-20 zeromq2: - Other minor and spelling fixes
[09:58] CIA-20 zeromq2: Signed-off-by: Martin Lucina <mato@kotelna.sk> - http://bit.ly/hV39nD
[10:12] CIA-20 zeromq2: 03Martin Lucina 07master * r520d621 10/ configure.in :
[10:12] CIA-20 zeromq2: Bump ABI version to 1:0:0 for ZeroMQ 2.1.0 release
[10:12] CIA-20 zeromq2: Signed-off-by: Martin Lucina <mato@kotelna.sk> - http://bit.ly/h94kRn
[10:34] vaadim Hello, all.
[10:34] vaadim Anybody can to help me?
[10:35] vaadim I have a critical error in object.cpp file.
[10:35] mikko vaadim: what is the error?
[10:36] vaadim Version 2.0.10 for windows.
[10:37] vaadim I write test project for client and server applications. Run zmq::queue() function.
[10:37] mikko what is the actual error?
[10:38] vaadim This function call zmq_poll->zmq::app_thread_t::process_commands->process_bind and
[10:38] vaadim in case command_t::bind: process_bind (cmd_.args.bind.in_pipe, cmd_.args.bind.out_pipe, blob_t (cmd_.args.bind.peer_identity, cmd_.args.bind.peer_identity_size));
[10:39] vaadim and cmd_.args.bind.peer_identity have a Bad Ptr == 0x0000000
[10:39] vaadim crash
[10:40] mikko does the same thing happen with github master?
[10:40] mikko can you test?
[10:40] mikko and is this easy to reproduce?
[10:42] vaadim I was try latest version in github, but this version have another problems, and i revert to version 2.0.10 back
[10:42] vaadim I can show my test project
[10:42] mikko what is the problem you had with the latest version?
[10:44] vaadim Current version in github is stable ? I need stable version for commercial project.
[10:45] mikko what i am intersted in what are the problems you are seeing so that they can be fixed
[10:45] mikko i would like you to test the github master to see if the issue with the null pointer has been fixed already
[10:47] vaadim OK. I will try master from github again. And I will come back with another bag's :)
[10:47] vaadim Thank you for help :)
[10:52] CIA-20 zeromq2: 03Steven McCoy 07master * r4a3ed39 10/ (3 files in 2 dirs):
[10:52] CIA-20 zeromq2: Bump OpenPGM to 5.0.92.
[10:52] CIA-20 zeromq2: Signed-off-by: Steven McCoy <steven.mccoy@miru.hk> - http://bit.ly/fLKN5H
[10:57] Steve-o thx Mato
[10:57] CIA-20 zeromq2: 03Martin Lucina 07master * rabc8b5e 10/ NEWS :
[10:57] CIA-20 zeromq2: Update NEWS file for 2.1.0 release
[10:57] CIA-20 zeromq2: Signed-off-by: Martin Lucina <mato@kotelna.sk> - http://bit.ly/hXdp5N
[11:16] rgl w00t 2.1 has hit the streets?
[11:16] mato almost
[11:16] mato due to DNS issues the announcement will go out this evening
[11:17] mato but it's in Git anyhow :-)
[11:18] mikko ill run daily builds now just to see that everything works
[11:19] rgl :)
[11:19] mikko building now
[11:19] rgl I just noticed Sockets may now be migrated between OS threads, as long as the
[11:19] rgl application ensures that a full memory barrier is issued. ... are there any examples how to do it?
[11:21] mikko is 2.1.0 going out as alpha or beta?
[11:22] rgl from the NEWS files its Beta
[11:24] sustrik rgl: when you migrate a thread you have no notify the other thread somehow
[11:24] sustrik the mechanism you use for the notfication will execute the barrier
[11:24] sustrik unless you are doing some black magic
[11:26] rgl I see. alright :D
[11:33] mikko success on all builds this far
[11:33] sustrik goodo
[11:34] mikko icc failed
[11:34] mikko let's see
[11:34] mikko it might be the shutdown stress test failing
[11:34] mikko Bad file descriptor
[11:34] mikko nbytes != -1 (tcp_socket.cpp:197)
[11:34] mikko yes
[11:35] sustrik yep, known issue
[11:36] rgl mikko, where is the build system? got url?
[11:37] mikko http://build.valokuva.org/
[11:38] mikko i got my eye on sunfire server on ebay
[11:38] mikko to add sparc into the build system
[11:38] Steve-o incredibly cheap these days, ~US$100
[11:39] Steve-o but also slow as lard, 100mb NICs often
[11:39] mikko this one is Dual 1.3Ghz Processors
[11:39] mikko i don't mind 100mb nic as it's just for building
[11:39] mikko not really for testing
[11:40] rgl mikko, no msvc build?
[11:40] Steve-o memory is the most important though, don't skimp on that
[11:40] mikko rgl: haven't got a windows running
[11:40] mikko Steve-o: it comes with 4GB
[11:40] vaadim Hello, I am here again.
[11:41] mikko the annoyance is that the build machines are running in finland
[11:41] mikko and i live in london
[11:41] mikko need to either host it here or ship it over
[11:41] sustrik does it make any difference?
[11:41] rgl mikko, http://build.valokuva.org/view/clrzmq/ is using mono then?
[11:41] mikko rgl: yes
[11:41] vaadim mikko: i try the current version from github in my test project
[11:41] mikko vaadim: ok
[11:42] mikko sustrik: i got free hosting in finland
[11:42] mikko that's the only difference
[11:42] sustrik i see
[11:43] rgl mikko, can you add clrmq2 into the clrmq tab too?
[11:43] mikko rgl: done
[11:43] mikko the regex for adding jobs there was outdated
[11:43] vaadim My server stoped with error while i try break zmq_device by call zmq_term() for kill context
[11:43] mikko clrzmq-.*
[11:43] rgl thanks :)
[11:44] mikko sustrik: what is the clean way to stop zmq device?
[11:44] mikko vaadim: i am not sure if that is the way to stop a device
[11:46] sustrik mikko: Ctrl+C?
[11:46] sustrik there's no remote management for devices yet
[11:46] vaadim I read this way in your mailing list archiew
[11:47] vaadim wait a moment...
[11:47] vaadim http://thread.gmane.org/gmane.network.zeromq.devel/2338/focus=2340
[11:48] sustrik vaadim: the devices are dumb pieces of code
[11:48] sustrik basically just a loop that reads from one socket and writes to another one
[11:49] sustrik if you want a better functionality, like a remote management
[11:49] sustrik you can add it there
[11:51] vaadim Sorry, i'm not understand, how to stop loop in device. I need to stop both sockets ?
[11:52] sustrik you just break out of the loop
[11:53] sustrik that's it
[11:53] vaadim How? Kill thread?
[11:53] sustrik while (true) {
[11:53] sustrik ...
[11:53] sustrik if (...)
[11:53] sustrik break;
[11:53] sustrik ...
[11:53] sustrik }
[11:55] vaadim But zmq_device(ZMQ_QUEUE, clients, workers); bon't return control, and i can't make loop myself.
[11:55] sustrik have a look at the source code
[11:55] mikko vaadim: have you looked at the device code?
[11:55] sustrik you just create your own device
[11:55] sustrik and modify it in any way you want
[11:57] vaadim Yeah, well, I'm gonna go build my own device, with blackjack and hookers. In fact, forget the device! :)
[11:59] vaadim Ok, i can append third parameter to my_zmq_device with stop socket.
[12:00] vaadim Thank's.
[16:06] Remoun hi
[16:06] Remoun I'm perusing the guide right now, but I thought I'd ask ahead: Can I overlay some sort of authentication mechanism for workers in a 0MQ-based system?
[16:07] mikko Remoun: i don't see why what would prevent you from doing that
[16:07] Remoun I don't see any reason, either; I just have no idea how yet
[16:07] Remoun But it should be doable (in a transparent manner), right?
[16:07] mikko what do you mean by transparent?
[16:07] mikko you would need to implement it in your code
[16:08] Remoun of course
[16:08] Remoun basically, layering the auth on top of the sockets
[16:08] Remoun So that I don't have to decorate every bit of code using 0MQ sockets with auth handling
[16:09] mikko there are plenty of ways to approach the problem
[16:09] Remoun ... Does that make sense?
[16:09] mikko you could do a small device that handles the authentication
[16:09] mikko and forwards the messages using inproc
[16:10] mikko so your client code would communicate with inproc and the device decorates with authentication info and forwards the messages
[16:13] Remoun sounds good
[16:13] Remoun thanks mikko
[16:24] shales hi, has anyone used eventlet with zmq? I'm wondering if or how eventlet works with zmq.REP sockets or do I have to switch to XREP sockets to avoid the state kept in the REP socket from messing up with multiple threads?
[16:30] mikko shales: sorry, no idea what eventlet is
[16:30] mikko shales: is it ruby?
[16:31] mikko ah, python
[16:31] shales python
[16:31] shales it's ok, someone on #eventlet just gave me an easy workaround
[16:31] shales just give each greenthread its own REP socket
[18:47] jhawk28 Hello. Congrats on the 2.1.0 beta!
[19:58] mikko mato: hmm, does 3.4.6 report that it supports dso visibility?
[20:00] mikko # elif (defined __GNUC__ && __GNUC__ >= 4) || defined __INTEL_COMPILER
[20:00] mikko this error reported makes no sense
[20:00] mato mikko: it shouldn't, i'm just replying asking for more info
[20:00] mato unless RHAT borked it of course :-)
[20:01] mikko but in any case it should not pass __GNUC__ >= 4
[20:01] mato ISTR some problems related to OpenPGM use of __attribute__((visibility)) on old RHAT also
[20:01] mikko downloading centos4 now
[20:01] mato :-)
[20:02] mikko http://cgi.ebay.co.uk/SUN-MICRO-SUNFIRE-V210-RACKMOUNT-SERVER-/230553957004?pt=UK_Computing_Networking_SM&hash=item35ae16328c
[20:02] mikko i got my eye on that
[20:02] mikko to add to build cluster
[20:02] mikko if the price doesn't go up much i'll try to snatch it
[20:04] prokos does ZMQ_SUBSCRIBE filter on the client side or the server side?
[20:04] mikko prokos: client side
[20:05] mato heh, I got offered a Sun Fire 15k for free some time ago, declined on the grounds that I don't have an empty data center and free electricity to run it...
[20:05] mikko if by client you mean the subscriber
[20:05] mikko mato: i'm hoping to put this in the server room at work
[20:05] mikko and as a backup plan im gonna ship it to finland
[20:06] mato well, 50 GBP is damn cheap
[20:06] prokos yes i mean the subscriber.. Is there a reason to not filter on the publisher side?
[20:07] mikko prokos: it's very complicated to implement but i think it's on the list of things to do
[20:11] jhawk28 has anyone seen this: http://sna-projects.com/kafka/
[20:20] mato mikko: I've forwarded you the outputs Chris Patti sent me just now
[20:20] mato mikko: It looks to me like his GCC claims to support -fvisibility=hidden but the results are completely bogus
[20:20] mato mikko: the symbol table in libzmq.so is missing anything which would normally be hidden
[20:21] mikko mato: yes
[20:21] mato mikko: looks like we need an autoconf check for "Checking to see if g++ -fvisibility=hidden actually works" :-)
[20:21] mikko i got centos4 running soon
[20:22] mato no point in fighting with getting it to work on buggy/old compilers, but it shouldn't break completely...
[20:23] mikko mato: does it support the pragma?
[20:23] mikko if it supports -fvisibility
[20:24] mato probably not
[20:24] mato or it's some bastardized RHAT version
[20:26] mikko got centos4 running now
[20:27] mikko yum is _slow_ on centos4
[20:36] mikko mato: gcc version 3.4.6 20060404 (Red Hat 3.4.6-11)
[20:36] mikko visibility works
[20:36] mikko no wait
[20:36] mikko it supports -fvisibility
[20:37] mikko but ignores the attribute
[20:37] mato geez
[20:37] mikko hmm
[20:37] mikko thats not completely true
[20:38] mikko https://gist.github.com/b2f2ef8f5bc92d2688b3
[20:38] mato well, in any case, we can't detect this case in zmq.h sanely
[20:39] mato so we need to account for it in the autoconf tests somehowe
[20:40] mato I reckon for now the easiest thing is to explicitly check the GCC version
[20:40] mato and not enable -fvisibility if it's < 4
[20:40] mikko yes, i was thinking the same
[20:40] mikko let me wrap a macro for that
[20:55] mikko brb
[21:17] prokos f
[22:10] mikko mato: take a look at http://valokuva.org/~mikko/visibility.patch when you got time
[22:41] jhawk28 hey sustrik
[22:43] sustrik hi
[22:47] Guthur is there tests for the new features?
[22:47] Guthur in 2.1.0