IRC Log


Monday January 3, 2011

[Time] NameMessage
[00:47] Skaag shykes_: sounds like an important and also "classical" use case to me, sounds like you need a sort of a "broker"? :-)
[00:48] shykes Correct. Although I already use zmq_forwarder as a broker
[00:50] shykes But before I reinvent the wheel, I'm trying to find out how much of it already exists
[00:50] shykes I'm guessing a lot of people use zeromq for the same purpose as me. How did all these people implement this?
[00:55] shykes Hum, I just found rabbitmq's rmq-0mq plugin. Maybe that can help
[01:00] Skaag we are porting our system from rabbitmq to 0mq
[01:00] Skaag because of scaling issues
[01:00] shykes Yeah, we were using rabbitmq too
[01:00] Skaag I think the case of a "dead" machine is handled well, but a misbehaving machine, that's another story
[01:01] Skaag you'll simply need to implement this yourself, or on the machine itself, decouple the process that receives the messages from the processes that do the actual work
[01:01] shykes yeah
[01:01] shykes sounds like less work than putting rabbitmq back into the mix :)
[01:01] Skaag why are you using 0mq instead of the wicked rabbit if I may ask?
[01:02] Skaag yes, also in my opinion, it is less work.
[01:02] shykes A few reasons
[01:02] shykes 1. We encountered problems with our Celery+RabbitMQ setup
[01:02] Skaag I wish I knew about the shortcomings before I started implementing with rabbit...
[01:02] shykes 2. We hit a major bug in the persister, which made it crash and not come back. Had to recover by hand...
[01:03] Skaag crap
[01:03] Skaag (and now i'm told that our 16 node cluster is considered "very large" in rabbitmq terms...)
[01:03] shykes 3. I hate amqp. It's too complicated
[01:03] Skaag well then we agree at least on 2 and 3, and about 1 is irrelevant to our own system
[01:03] shykes 4. more freedom in the topology
[01:04] Skaag right
[01:04] shykes for example, switch to multicast down the road
[01:04] Skaag ok this gives me validation in my own path
[01:04] Skaag yep
[01:04] Skaag at least within a single DC
[01:04] Skaag I believe multicast over WAN is not feasible and may never be
[01:04] shykes yeah, we're primarily on EC2 for now, so not relevant for us either
[01:21] Skaag I constantly fail to see how EC2 helps with anything other than temporarily huge computations
[01:21] Skaag it's more expensive than standard hosting
[01:21] Skaag by any calculation I made
[03:01] viraptor hi all
[03:02] viraptor could someone tell me where to look for information on zeromq + forking? the docs describe multithreading, but not how I should handle a fork (inherit the context or not)
[03:46] viraptor ehh... found the answer after all... it's "don't share the context with fork"
[05:46] jugg with zmq 2.1.x, sockets can be shared between threads. However, zmq_poll documentation still state that the sockets must belong to the same thread calling zmq_poll. Is that still the case?
[06:09] neopallium jugg: with zmq 2.1 sockets can be moved between threads, but you still can't send/recv from different threads on the same socket.
[06:57] PeterTork Question regarding the Ruby bindings. I am creating a socket, and then trying to extend it with a module. This causes it to fail in ZM.select(arrayOfSockets, nil, nil, 30) with the error `select': uninitialized stream (IOError)
[06:57] PeterTork Removing the s.extend(EmptyModule) fixes this.
[17:55] Seta00 I'm having lots of problems building pyzmq :(
[17:55] Seta00 well, only one problem, but I can't fix it :P
[17:58] mikko what problem are you having?
[18:02] Seta00 mikko, distutils couldn't find my vcvarsall, but I've just fixed it by manually returning the absolute path to it
[18:02] Seta00 now I just need to put the include files on the right dirs
[18:02] Seta00 and everything is good to go (I think)
[18:06] Seta00 now I need a constants.pyd.manifest file
[18:15] Seta00 lol I had to manually create a bunch of manifest files
[18:18] Seta00 hehe finally working
[18:25] Seta00 aaaand they don't work :)
[18:26] Seta00 DLL load failed: The application has failed to start because its side-by-side configuration is incorrect.
[18:50] Seta00 this thing is driving me nuts, I think I'll just use the C++ library
[18:56] sustrik Seta00: try asking on the mailing list, the pyzmq guys can possibly help you
[19:29] mikko sustrik: build currently fails on sunos using gcc3
[19:29] mikko not sure if that is big deal
[19:38] sustrik mikko: there's lot of failures there...
[19:38] mikko what do you mean?
[19:38] mikko you mean sunos and gcc3 to begin with
[19:39] mikko :)
[19:39] sustrik i see 8 red point on valokuva.org
[19:39] sustrik points
[19:39] mikko have i broken something?
[19:39] mikko i'll check
[19:40] mikko github was unavailable it seems
[19:40] sustrik ah, yes
[19:40] sustrik i've noticed that myself
[19:40] sustrik so where should i look for console output?
[19:41] mikko http://build.valokuva.org/job/ZeroMQ2-core-master-SunGCC-solaris10/4/console
[19:41] mikko there at the end
[19:42] mikko i looked at it last night, looks like compiler error to me
[19:44] sustrik yep
[19:44] sustrik let me fix it
[19:49] CIA-21 zeromq2: 03Martin Sustrik 07master * r95eebbf 10/ src/sub.cpp :
[19:49] CIA-21 zeromq2: sunos/gcc3 compilation issue fixed
[19:49] CIA-21 zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> - http://bit.ly/ezkv67
[19:50] sustrik mikko: try now
[21:07] mikko sustrik: i'll kick a build
[21:08] mikko building
[21:16] mikko passes
[21:18] sustrik great
[21:26] Thomas_aboutus I have what I'm sure is a super-noob question, but I've been over the docs a hundred times and can't figure out what I'm doing wrong. If someone can help me that would rock!
[21:26] Thomas_aboutus I am trying to make a simple pub/sub setup. I have the code here: https://gist.github.com/763990 For some reason, the subscriber never receives messages from the publisher. The code in the gist is as simple as I could make it for clarities sake.
[21:36] mikko Thomas_aboutus: both sockets connect
[21:37] mikko change the pub to bind and it should work
[21:40] Thomas_aboutus Wow... Thank you.
[21:40] Thomas_aboutus :)