Tuesday November 8, 2011

[Time] NameMessage
[01:41] amoffatw if i've called multiple .connect(addr) on a socket, is there any way to disconnect a specific address?
[01:57] jhawk28 amoffatw: now
[01:57] jhawk28 no
[01:58] amoffatw jhawk28, oh well, do you know if it's planned?
[01:58] jhawk28 not currently on the published roadmap
[02:00] jhawk28 if you really need it, try to create a convincing case and maybe the devs will add it
[02:00] jhawk28 or you could do it yourself and then submit a pull request
[02:01] amoffatw jhawk28, i was just looking at the github repo. i might just do that
[02:02] amoffatw doesn't sound too difficult in theory, but then again i don't know how it all works under the hood
[02:02] jhawk28 its well laid out
[02:03] jhawk28 the challenge of course is making a good/clean API which is most likely the reason why its not there
[02:04] jhawk28 an alternative would be to create a socket per connect and then use the Poller to select between them
[02:04] jhawk28 you can then close a socket at the connect granularity
[02:05] amoffatw i'll have to look into the Poller more. i'm using pyzmq with eventlet, so i don't know if the poller plays nicely with eventlet
[02:06] jhawk28 unfamiliar with the eventlet stuff, only used python a little
[02:08] jhawk28 eventlet or eventloop?
[02:09] amoffatw jhawk28, eventlet. it's a lightweight threading module. tremendous boon for doing really crazy concurrency
[02:09] jhawk28 eventlet == epoll?
[02:10] jhawk28 thats what the Poller is, a selector for ZMQ Sockets
[02:12] jhawk28 seems like it is more of a threading library
[02:13] amoffatw eventlet is kind of crazy piece of software. it's both lightweight threading and non-blocking I/O
[02:13] amoffatw the idea is that a lightweight thread should only yield when its network IO blocks
[02:13] amoffatw which is a good strategy if you're IO bound
[02:14] amoffatw so it does some magic with c extensions to wrap all of python's socket internals in such a way that this is possible
[02:14] amoffatw it works with zeromq, but i can't wrap my head around if the poller would work. i guess i could write up some tests
[02:15] jhawk28 the poller is just the polling portion (may use epoll underneath)
[02:15] jhawk28 be back in a bit, need to read a bedtime story
[05:11] jwtodd q re durability (probably has been asked many a time) ... been reading up on durable sockets et al, but curious, does this intrinsically support durability of state across zeromq process transitions?
[08:09] mikko jwtodd: you mean persistent storage?
[10:20] alanw123 hi, i hope someone can help me explain why i'm getting an assertion in zmq3 when using xpub/xsub
[10:20] alanw123
[10:20] alanw123 v3.0.2
[10:22] alanw123 Assertion failed: erased == 1 (mtrie.cpp:192)
[10:28] mikko alanw123: i think this might be related to recent mtrie bugs fixed
[10:28] mikko let me find the commit
[10:29] alanw123 mikko: ok, i hope that's the case
[10:30] alanw123 mikko: but i didn't see anything fixes in the code in this area
[10:30] mikko it might be unrelated
[10:30] mikko issue 277
[10:34] alanw123 mikko: ok, we'll give that a try, see if it fixes the problem
[10:40] mikko alanw123: why are you using 3.x ?
[10:41] mikko for sub forwarding?
[10:41] alanw123 y, we've written a sub forwarder device
[10:42] mikko there is a possibility that 3.x is going to go through some radical changes before stable
[10:42] mikko it's good to be aware of that
[10:43] alanw123 hmm, ok
[10:43] alanw123 i thought 3.x was closer to being finished than that
[10:43] mikko alanw123: it's still very much beta
[10:44] alanw123 can you give me an idea of what will change?
[10:44] mikko the label stuff and req/rep protocol
[10:44] alanw123 is it mainly refactorings? or is there additional functionality going in?
[10:44] mikko to make it more compatible with 2.1
[10:44] mikko it hasn't been decided yet
[10:44] alanw123 ok, thanks for the heads-up
[10:45] mikko as far as i know the api changes are going to be fairly small
[10:46] mikko and pub/sub shouldnt change from what it is in 3.x now
[10:46] mikko i still use 2.1 for most of my development
[10:48] alanw123 right
[10:48] mikko it would make sense to join the discussion
[10:48] alanw123 however we're looking at rolling out pubsub on a fairly large scale, so we wanted to take advantage of the lovely subscription forwarding in 3.x :-)
[10:49] mikko there is a debate going on on the lists
[10:49] mikko are you working at spotify?
[10:49] alanw123 yep
[10:49] mikko (based on the hostmask)
[10:50] mikko have you got any sort of write-up how spotify uses zeromq?
[10:50] mikko that would be interesting to read
[10:51] alanw123 yeah that would be a good thing (™)
[10:52] alanw123 but right now, we're not really exploiting the full potential of zmq
[10:52] alanw123 yet
[10:54] alanw123 but we're getting a lot of excitement around the what we can achieve with a good pubsub model
[10:55] eintr i did a write up:
[10:56] eintr i'm especially unhappy with the sharding
[11:19] mikko alanw123: that would be nice
[11:19] mikko alanw123: it's always nice to read about real use-cases
[12:06] sustrik eintr: hi
[12:06] sustrik you still there?
[12:07] eintr sure am
[12:07] eintr mikko: (i'm with alan too btw).. we're actually isolating zmq3 pubsub in a separate topology (and processes), allowing us to be more flexible about running unstable - so only where sub-forwarding is needed, basically. if it breaks due to changes, it's a relatively small event. works pretty well.
[12:08] mikko eintr: regarding the posting, did you see this: ?
[12:08] mikko i wonder if both or either one are hoaxes
[12:08] mikko eintr: what are you using for serialisation?
[12:08] eintr mikko: heh, yeah, i'm just marvelling at the trolling and metatrolling mixed in with random ignorance :P
[12:08] sustrik eintr: i'm a bit confused, the write-up is marked as <SATIRE> but observations seem be spot on
[12:09] mikko sustrik: it's a posting made about mongodb
[12:09] eintr sustrik: lol, hell no.. I just copied and did a searc/replace :)
[12:09] sustrik ah, interesting
[12:09] mikko sustrik: it circulated the internet a few days ago
[12:09] sustrik because some parts seem to fit
[12:09] sustrik asserts...
[12:09] mikko yes, especially the protocol part
[12:09] sustrik the part about overwriting the database was strange though :)
[12:10] mikko sustrik: are you adding asserts in new code btw?
[12:10] mikko or steering away from that?
[12:11] sustrik i am
[12:11] sustrik i have no idea what else to do with unexpected errors
[12:11] eintr sustrik: yeah, maybe, but that's coincidental :).. zmq is well-informed, minimal, easy to understand and foremost *transparent and humble*.
[12:11] sustrik thanks :)
[12:11] mikko sustrik: handle them?
[12:12] sustrik how would you handle unexpected?
[12:12] sustrik it's something you assume it won't happen
[12:12] mikko depends on the unexpected i guess
[12:12] sustrik :)
[12:12] mikko out of memory is tricky
[12:12] mikko but some are more low hanging fruits
[12:12] sustrik yes, that's the special case
[12:12] sustrik mikko: yep, i'm fixing things as i go
[12:12] sustrik but it's slow :(
[12:13] mikko sustrik: i might get mac installed over the christmas holidays
[12:13] sustrik great
[12:13] mikko im going snowboarding for most of december
[12:13] mikko and the place is pretty close to the data centre
[12:13] mikko in finland
[12:13] sustrik nice
[12:14] sustrik i'm probably going to be in korea
[12:14] sustrik consulting
[12:14] sustrik no idea how much time i'll have for 0mq core work
[12:15] mikko is it zeromq related consulting?
[12:15] sustrik ack
[12:15] sustrik covered by nda though
[12:15] mikko hopefully there are some fixes trickling in
[12:15] sustrik yes, i hope so
[12:16] mikko is it a long job?
[12:16] sustrik 2-3 months
[12:17] mikko cool, you probably get quite used to kimchi
[12:17] sustrik actually, i quite like it
[12:17] sustrik looking forward for the local food
[12:17] sustrik where's the datacenter btw?
[12:17] sustrik helsinki?
[12:17] mikko mid-finland
[12:18] sustrik no need for cooling i guess
[12:18] mikko hehe, apart from summer time
[12:19] sustrik hm, you could build a datacenter-powered sauna
[12:20] mato datacenter-powered sauna, wow
[12:20] sustrik eintr: actually, reading the article as is if was about 0mq was quite a useful experience
[12:24] eintr haha, should i search-replace some other rant and see what we else can learn? :)
[12:25] sustrik it's definitely an interesting way to get feedback about the product :)
[12:27] sustrik are you from spotify as well?
[12:37] eintr yup
[12:38] sustrik if you can do a very short writeup of your usage of 0mq (1 paragraph), it would be pretty useful
[12:39] sustrik the project lacks references to actual deployments
[12:39] sustrik financial companies tend to cover it by NDAs
[12:39] sustrik other don't care enough to write about it
[12:40] eintr absolutely, will do.
[12:40] sustrik if you are a new user it's not easy to find out whether anybody uses 0mq for any real work
[12:40] sustrik thanks!
[12:40] sustrik heh
[12:41] sustrik been through that for several times myself
[13:51] CIA-28 libzmq: 03Martin Sustrik 07master * r89962a8 10/ src/req.cpp : Uninitialised FSM state in REQ session fixed (issue 278) ...
[14:16] bayerj I have the following setup: mac os 10.6, zeromq installed via homebrew (2.1.10), pyzmq installed via pip/PyPI. I get this error message when I try to 'import zmq': Any ideas what is going wrong here?
[14:37] mikko bayerj: yes
[14:38] mikko bayerj: it seems that the library is not runtime linkers path (i assume)
[14:38] mikko where does homebrew install it?
[14:38] bayerj it installs it in /usr/local/
[14:38] bayerj can I fix it by setting some environment variable?
[14:39] mikko try this first:
[14:39] mikko otool -L /Library/Python/2.6/site-packages/zmq/core/
[14:40] bayerj ~$ otool -L /Library/Python/2.6/site-packages/zmq/core/ /Library/Python/2.6/site-packages/zmq/core/ /usr/local/lib/libzmq.1.dylib (compatibility version 2.0.0, current version 2.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.11)
[14:41] mikko interesting, it seems to be picking it up
[14:41] mikko i wonder where the symbol error comes from
[14:41] bayerj I just found this:
[14:42] bayerj I will walk through that.