ZeroMq IRC Log

Tuesday November 8, 2011

[Time] Name	Message
[01:41] amoffatw	if i've called multiple .connect(addr) on a socket, is there any way to disconnect a specific address?
[01:57] jhawk28	amoffatw: now
[01:57] jhawk28	no
[01:58] amoffatw	jhawk28, oh well, do you know if it's planned?
[01:58] jhawk28	not currently on the published roadmap
[02:00] jhawk28	if you really need it, try to create a convincing case and maybe the devs will add it
[02:00] jhawk28	or you could do it yourself and then submit a pull request
[02:01] amoffatw	jhawk28, i was just looking at the github repo. i might just do that
[02:02] amoffatw	doesn't sound too difficult in theory, but then again i don't know how it all works under the hood
[02:02] jhawk28	its well laid out
[02:03] jhawk28	the challenge of course is making a good/clean API which is most likely the reason why its not there
[02:04] jhawk28	an alternative would be to create a socket per connect and then use the Poller to select between them
[02:04] jhawk28	you can then close a socket at the connect granularity
[02:05] amoffatw	i'll have to look into the Poller more. i'm using pyzmq with eventlet, so i don't know if the poller plays nicely with eventlet
[02:06] jhawk28	unfamiliar with the eventlet stuff, only used python a little
[02:08] jhawk28	eventlet or eventloop?
[02:09] amoffatw	jhawk28, eventlet. it's a lightweight threading module. tremendous boon for doing really crazy concurrency
[02:09] jhawk28	eventlet == epoll?
[02:10] jhawk28	thats what the Poller is, a selector for ZMQ Sockets
[02:12] jhawk28	seems like it is more of a threading library
[02:13] amoffatw	eventlet is kind of crazy piece of software. it's both lightweight threading and non-blocking I/O
[02:13] amoffatw	the idea is that a lightweight thread should only yield when its network IO blocks
[02:13] amoffatw	which is a good strategy if you're IO bound
[02:14] amoffatw	so it does some magic with c extensions to wrap all of python's socket internals in such a way that this is possible
[02:14] amoffatw	it works with zeromq, but i can't wrap my head around if the poller would work. i guess i could write up some tests
[02:15] jhawk28	the poller is just the polling portion (may use epoll underneath)
[02:15] jhawk28	be back in a bit, need to read a bedtime story
[05:11] jwtodd	q re durability (probably has been asked many a time) ... been reading up on durable sockets et al, but curious, does this intrinsically support durability of state across zeromq process transitions?
[08:09] mikko	jwtodd: you mean persistent storage?
[10:20] alanw123	hi, i hope someone can help me explain why i'm getting an assertion in zmq3 when using xpub/xsub
[10:20] alanw123	http://codepad.org/nomj4WjW
[10:20] alanw123	v3.0.2
[10:22] alanw123	Assertion failed: erased == 1 (mtrie.cpp:192)
[10:28] mikko	alanw123: i think this might be related to recent mtrie bugs fixed
[10:28] mikko	let me find the commit
[10:29] alanw123	mikko: ok, i hope that's the case
[10:30] alanw123	mikko: but i didn't see anything fixes in the code in this area
[10:30] mikko	it might be unrelated
[10:30] mikko	issue 277
[10:34] alanw123	mikko: ok, we'll give that a try, see if it fixes the problem
[10:40] mikko	alanw123: why are you using 3.x ?
[10:41] mikko	for sub forwarding?
[10:41] alanw123	y, we've written a sub forwarder device
[10:42] mikko	there is a possibility that 3.x is going to go through some radical changes before stable
[10:42] mikko	it's good to be aware of that
[10:43] alanw123	hmm, ok
[10:43] alanw123	i thought 3.x was closer to being finished than that
[10:43] mikko	alanw123: it's still very much beta
[10:44] alanw123	can you give me an idea of what will change?
[10:44] mikko	the label stuff and req/rep protocol
[10:44] alanw123	is it mainly refactorings? or is there additional functionality going in?
[10:44] mikko	to make it more compatible with 2.1
[10:44] mikko	it hasn't been decided yet
[10:44] alanw123	ok, thanks for the heads-up
[10:45] mikko	as far as i know the api changes are going to be fairly small
[10:46] mikko	and pub/sub shouldnt change from what it is in 3.x now
[10:46] mikko	i still use 2.1 for most of my development
[10:48] alanw123	right
[10:48] mikko	it would make sense to join the discussion
[10:48] alanw123	however we're looking at rolling out pubsub on a fairly large scale, so we wanted to take advantage of the lovely subscription forwarding in 3.x :-)
[10:49] mikko	there is a debate going on on the lists
[10:49] mikko	are you working at spotify?
[10:49] alanw123	yep
[10:49] mikko	(based on the hostmask)
[10:50] mikko	have you got any sort of write-up how spotify uses zeromq?
[10:50] mikko	that would be interesting to read
[10:51] alanw123	yeah that would be a good thing (â¢)
[10:52] alanw123	but right now, we're not really exploiting the full potential of zmq
[10:52] alanw123	yet
[10:54] alanw123	but we're getting a lot of excitement around the what we can achieve with a good pubsub model
[10:55] eintr	i did a write up: http://pastebin.com/UA8bhuAB
[10:56] eintr	i'm especially unhappy with the sharding
[11:19] mikko	alanw123: that would be nice
[11:19] mikko	alanw123: it's always nice to read about real use-cases
[12:06] sustrik	eintr: hi
[12:06] sustrik	you still there?
[12:07] eintr	sure am
[12:07] eintr	mikko: (i'm with alan too btw).. we're actually isolating zmq3 pubsub in a separate topology (and processes), allowing us to be more flexible about running unstable - so only where sub-forwarding is needed, basically. if it breaks due to changes, it's a relatively small event. works pretty well.
[12:08] mikko	eintr: regarding the posting, did you see this: http://news.ycombinator.com/item?id=3202959 ?
[12:08] mikko	i wonder if both or either one are hoaxes
[12:08] mikko	eintr: what are you using for serialisation?
[12:08] eintr	mikko: heh, yeah, i'm just marvelling at the trolling and metatrolling mixed in with random ignorance :P
[12:08] sustrik	eintr: i'm a bit confused, the write-up is marked as <SATIRE> but observations seem be spot on
[12:09] mikko	sustrik: it's a posting made about mongodb
[12:09] eintr	sustrik: lol, hell no.. I just copied http://pastebin.com/raw.php?i=FD3xe6Jt and did a searc/replace :)
[12:09] sustrik	ah, interesting
[12:09] mikko	sustrik: it circulated the internet a few days ago
[12:09] sustrik	because some parts seem to fit
[12:09] sustrik	asserts...
[12:09] mikko	yes, especially the protocol part
[12:09] sustrik	the part about overwriting the database was strange though :)
[12:10] mikko	sustrik: are you adding asserts in new code btw?
[12:10] mikko	or steering away from that?
[12:11] sustrik	i am
[12:11] sustrik	i have no idea what else to do with unexpected errors
[12:11] eintr	sustrik: yeah, maybe, but that's coincidental :).. zmq is well-informed, minimal, easy to understand and foremost transparent and humble.
[12:11] sustrik	thanks :)
[12:11] mikko	sustrik: handle them?
[12:12] sustrik	how would you handle unexpected?
[12:12] sustrik	it's something you assume it won't happen
[12:12] mikko	depends on the unexpected i guess
[12:12] sustrik	:)
[12:12] mikko	out of memory is tricky
[12:12] mikko	but some are more low hanging fruits
[12:12] sustrik	yes, that's the special case
[12:12] sustrik	mikko: yep, i'm fixing things as i go
[12:12] sustrik	but it's slow :(
[12:13] mikko	sustrik: i might get mac installed over the christmas holidays
[12:13] sustrik	great
[12:13] mikko	im going snowboarding for most of december
[12:13] mikko	and the place is pretty close to the data centre
[12:13] mikko	in finland
[12:13] sustrik	nice
[12:14] sustrik	i'm probably going to be in korea
[12:14] sustrik	consulting
[12:14] sustrik	no idea how much time i'll have for 0mq core work
[12:15] mikko	is it zeromq related consulting?
[12:15] sustrik	ack
[12:15] sustrik	covered by nda though
[12:15] mikko	hopefully there are some fixes trickling in
[12:15] sustrik	yes, i hope so
[12:16] mikko	is it a long job?
[12:16] sustrik	2-3 months
[12:17] mikko	cool, you probably get quite used to kimchi
[12:17] sustrik	actually, i quite like it
[12:17] sustrik	looking forward for the local food
[12:17] sustrik	where's the datacenter btw?
[12:17] sustrik	helsinki?
[12:17] mikko	mid-finland
[12:18] sustrik	no need for cooling i guess
[12:18] mikko	hehe, apart from summer time
[12:19] sustrik	hm, you could build a datacenter-powered sauna
[12:20] mato	datacenter-powered sauna, wow
[12:20] sustrik	eintr: actually, reading the article as is if was about 0mq was quite a useful experience
[12:24] eintr	haha, should i search-replace some other rant and see what we else can learn? :)
[12:25] sustrik	it's definitely an interesting way to get feedback about the product :)
[12:27] sustrik	are you from spotify as well?
[12:37] eintr	yup
[12:38] sustrik	if you can do a very short writeup of your usage of 0mq (1 paragraph), it would be pretty useful
[12:39] sustrik	the project lacks references to actual deployments
[12:39] sustrik	financial companies tend to cover it by NDAs
[12:39] sustrik	other don't care enough to write about it
[12:40] eintr	absolutely, will do.
[12:40] sustrik	if you are a new user it's not easy to find out whether anybody uses 0mq for any real work
[12:40] sustrik	thanks!
[12:40] sustrik	heh
[12:41] sustrik	been through that for several times myself
[13:51] CIA-28	libzmq: 03Martin Sustrik 07master * r89962a8 10/ src/req.cpp : Uninitialised FSM state in REQ session fixed (issue 278) ...
[14:16] bayerj	I have the following setup: mac os 10.6, zeromq installed via homebrew (2.1.10), pyzmq installed via pip/PyPI. I get this error message when I try to 'import zmq': https://gist.github.com/1347829. Any ideas what is going wrong here?
[14:37] mikko	bayerj: yes
[14:38] mikko	bayerj: it seems that the library is not runtime linkers path (i assume)
[14:38] mikko	where does homebrew install it?
[14:38] bayerj	it installs it in /usr/local/
[14:38] bayerj	can I fix it by setting some environment variable?
[14:39] mikko	try this first:
[14:39] mikko	otool -L /Library/Python/2.6/site-packages/zmq/core/error.so
[14:40] bayerj	~$ otool -L /Library/Python/2.6/site-packages/zmq/core/error.so /Library/Python/2.6/site-packages/zmq/core/error.so: /usr/local/lib/libzmq.1.dylib (compatibility version 2.0.0, current version 2.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.11)
[14:41] mikko	interesting, it seems to be picking it up
[14:41] mikko	i wonder where the symbol error comes from
[14:41] bayerj	I just found this: https://github.com/zeromq/pyzmq/issues/77
[14:42] bayerj	I will walk through that.

ZeroMq Home

Tuesday November 8, 2011