ZeroMq IRC Log

Wednesday June 8, 2011

[Time] Name	Message
[00:14] wayneeseguin	Is that bash binding actually implemented somewhere or is it a joke pieterh ?
[00:15] wayneeseguin	If I want to add support to a language that is not there yet, where do I look to do that?
[00:15] wayneeseguin	Do I have to look at others for examples or is there a guide
[05:40] whack	wayneeseguin: I think there's a protocol spec for zeromq - http://rfc.zeromq.org/spec:1 might be it
[06:55] jkp	question: is there a recommended way to implement a many-to-many "broadcast" topology without using multicasting?
[06:55] jkp	im thinking of trying to use zeromq to replace multicasting as a mechanism for broadcasting updates from our servers to our clients
[06:56] jkp	in this scenario, all clients must listen to all servers, but id rather avoid a broker if possible (for variou reasons)
[06:57] guido_g	Ã¸mq has devices for that kind of thing
[06:57] guido_g	devices are small programms with exactly one job, forward messages according to one pattern
[06:57] jkp	guido_g: it does? i read the docs but i didnt see any devices matching that pattern
[06:57] jkp	are they new?
[06:57] guido_g	no
[06:58] jkp	can you name one in particular i should look up?
[06:58] guido_g	there're two kinds of devices: the ones in the library itself and thn some executables
[06:58] guido_g	http://api.zeromq.org/2-1:zmq-device <- lib version
[06:59] guido_g	the executables do come with the v2 of Ã¸mq and are built by default, afair
[06:59] jkp	ahh, it looks like ZMQ_STREAMER would work...but
[06:59] jkp	that still requires a broker to run the device
[07:00] guido_g	of course
[07:00] ianbarber	jkp: if you want many to many without multicast you have to have a device, or you have to have every node know about every other node
[07:00] guido_g	how else should it work?
[07:01] guido_g	complete mesh would kill performance, on app level and on os level
[07:02] jkp	ianbarber: k, i was getting that impression
[07:02] jkp	basically im trying to cope with the following scenario...
[07:02] ianbarber	jkp: that's less a limitation of zeromq and more one of the fact the communication channels are point to point
[07:02] jkp	we have a number of parallel stateless servers, all doing the same thing and sharing state through the db
[07:03] jkp	clients make requests and update data and then all other clients should know when certain updates have happened
[07:03] jkp	we currently use multicasts and have the clients listen to the group / port and reassebmle the messages at the other end
[07:04] jkp	problem is that multicasting is pretty unreliable in a lot of cases and also the code is complex
[07:04] jkp	zeromq looked like it could help here
[07:04] jkp	brb
[07:04] guido_g	sounds like a case for (e)pgm
[07:05] ianbarber	yeah, i think you'll find pgm and zeromq a lot less complex than you may have expected
[07:05] guido_g	one of the best things in Ã¸mq is the pgm support
[07:05] guido_g	because you get most of the mutlicast w/o most of the trouble
[07:08] jkp	yeah, i did some playing around with epgm yesterday
[07:08] jkp	it worked as i would expect...but its multicast based so i guess it just buys me reduced code complexity rather than removing the potential issues of a multicast based solution altogether
[07:09] guido_g	you guess?
[07:09] jkp	heh :)
[07:10] jkp	ok, "it buys me reduced code complexity" :)
[07:10] jkp	better?
[07:10] jkp	:p
[07:10] guido_g	http://code.google.com/p/openpgm/
[07:11] jkp	guido_g: thanks for the pointer :)
[07:11] jkp	i shall do some reading about pgm
[07:11] guido_g	ack
[07:11] jkp	the problem we've had with multicasts as a transport is just getting customers to set their networks up correctly
[07:11] jkp	a lot of times they dont configure their routers right etc and then of course the finger is pointed at our application stack for being crap
[07:12] jkp	(when updates dont arrive as expected)
[07:12] guido_g	it's a cruel world
[07:12] jkp	heh
[07:13] jkp	yeah, but its meant that for a long while we've been thinking we'd like to use something else...something easier to configure correctly from a customer perspective
[07:14] guido_g	think about an application that has to deal w/ hunderts or thousands of endpoints
[07:15] guido_g	an all gos belly up if one port is wrong
[07:15] guido_g	do you really think that is better for you customer (or you)?
[07:19] ianbarber	you could offer 2 modes, have a broker device and have all connect to that, or have multicast for higher performance setups
[07:19] ianbarber	that's really just changing zeromq DSNs, and writing the broker
[07:20] guido_g	not even that, if you can use the zmq_forwarder
[07:21] guido_g	bust sill, lots of tcp connections in one process, that will slow down delivery significantly
[07:41] private_meta	jond: ah good, you see online
[07:42] private_meta	https://gist.github.com/03e2732fc271721e8504 <-- gdb output on redhat when SIGTSTP signal is sent while in "zmq::poll"
[07:52] mrmermaid	Can anyone please help on building libzmq on VS C++ 2010 Express ?
[07:53] mrmermaid	Please point me to some direction as I can only build the .dll file but not the .lib
[08:20] ianbarber	mrmermaid: mikko might be able to help. is there anything similar on build.zero.mq, that may give you a clue
[08:21] mrmermaid	Thank you, I have figured it out
[08:22] mrmermaid	I gave up and install VSC 2008
[08:22] mrmermaid	I checked, and in Properties/Linker/Advanced/Import Library, this is the value that is missing: $(TargetDir)$(TargetName).lib
[08:23] mrmermaid	not sure if this is enough, but compile with 2008 should give a green, posting here so that everyone knows.
[08:35] mikko	mrmermaid: that happens because of the conversion
[08:35] mikko	mrmermaid: for some reason the 2008 project file -> 2010 conversion removes the import library
[08:36] mikko	i will test CMake stuff soon to see if that helps on windows builds
[08:52] mrmermaid	Thank you for your effort.
[09:04] mile	hi guys,
[09:05] mile	I'm getting a "Bad file descriptor
[09:05] mile	nbytes != -1 (mailbox.cpp:242)
[09:05] mile	" error
[09:05] mile	even when I disabled binding
[09:05] mile	any ideas what might cause it?
[09:05] mikko	pieterh: are you here?
[10:14] pieterh	re
[10:14] pieterh	hi mikko
[11:21] mato	hi pieterh
[11:21] mato	pieterh: how was silicon valley?
[12:41] pieterh	hi mato
[12:41] pieterh	I was mostly in downtown SFO
[12:41] pieterh	it's pretty intense, every cafe is filled with guys working on their starups
[12:41] pieterh	&startups
[13:28] Seta00	hmm, I think I'm going to write a ZMQ Groovy wrapper
[13:29] Seta00	so you can have two Actors in two different machines communicating with the same Actor syntax
[13:29] Seta00	this is going to be interesting
[13:35] tproa	i read that as "gravy wrapper"
[14:02] ianbarber	literally every time i think 'oh i need a zero.mq/? shortcode for that link' pieter has already made one :)
[14:30] Seta00	anyone here uses jzmq?
[14:34] pieterh	ianbarber: lol, I'm sure you can think of some new ones
[14:34] pieterh	Seta00: I know some folk who use jzmq, yes
[14:35] mikko	pieterh: we should be all set with jira data now
[14:35] mikko	just needs fiddling the projects around
[14:35] mikko	(LIBZMQX) is the fixed one)
[14:35] pieterh	mikko: did Atlassian fix the messed up comments?
[14:35] mikko	yes
[14:35] pieterh	oh, wow, great!
[14:36] mikko	their importer plugin is broken
[14:37] pieterh	so what's the next step, Mikko?
[14:38] mikko	pieterh: renaming or deleting LIBZMQ
[14:38] mikko	and renaming LIBZMQX to LIBZMQ
[14:38] pieterh	let's go for it
[14:38] mikko	i can do it this evening
[14:38] mikko	or you can do it any time
[14:38] pieterh	I can do it now
[14:38] pieterh	mato: you around?
[14:40] pieterh	I see people have been adding issues to libzmqx
[14:42] mikko	pieterh: nope
[14:43] mikko	pieterh: theyve been copied from LIBZMQ
[14:43] mikko	i think
[14:43] mikko	216 - N
[14:43] pieterh	ah, cause I didn't see them in libzmq
[14:43] pieterh	is it possible to change the project key?
[14:44] mato	pieterh: i was going to do it, just waiting for confirmation from mikko
[14:44] mato	which just arrived
[14:44] mato	pieterh: mikko: i'll do it now ...
[14:44] pieterh	mato: ok, I've renamed the two projects and set an avatar / URL for the new libzmq
[14:44] pieterh	you can take over...
[14:44] mato	ok
[14:45] pieterh	I'm going to create a czmq project and we can offer this to other binding authors who want it
[14:45] mato	pieterh: yes, i've already put intro text into the dashboard telling people the tracker is open for any zmq projects
[14:46] pieterh	cool
[14:49] mikko	this concludes the migration?
[14:49] mikko	apart from the https cert
[14:49] mikko	https cert / custom domain
[14:49] mato	mikko: almost, can you ask in the support request if they can re-key LIBZMQX to LIBZMQ?
[14:50] mato	mikko: that and e-mail notifications which i will setup next ...
[14:59] mikko	mato: i'll ask if they can rekey it
[15:00] mato	mikko: this forum post suggests it can't be done easily: http://forums.atlassian.com/thread.jspa?messageID=257376506
[15:00] mato	mikko: but maybe they know a way
[15:01] mikko	mato: export and reimport ?
[15:02] mikko	i don't see why that wouldnt work
[15:02] mato	we'll see what they say
[15:02] mato	i'll leave the "please don't add issues" banner up for now
[15:04] pieterh	mato: afair from previous use of Jira, it can be done with an export and import and manual edit in between
[15:04] pieterh	mato: IMO that banner should go, it'll confuse people
[15:04] mato	pieterh: not if we're going to do one more export/reimport
[15:05] mato	pieterh: the only problem is, this is all hosted and only atlassian can do import/export it seems
[15:05] pieterh	mato: I guess so, but I'm sending people to that page now for czmq as well
[15:05] pieterh	ah, right
[15:05] pieterh	bleh
[15:05] mato	pieterh: that's why this is all taking so long ...
[15:05] mato	pieterh: otherwise we'd have had it all sorted ages ago.
[15:05] pieterh	that's why there was no import/export option anywhere
[15:05] mikko	it's a trade-off
[15:05] mikko	takes longer but we dont need to host it
[15:06] pieterh	well, notifications seem to work
[15:06] pieterh	finally, this makes it worthwhile :)
[15:07] mato	notifications?
[15:07] mato	those aren't even set up yet :)
[15:09] mato	pieterh: anyhow, so we need to wait for atlassian to do the re-key
[15:09] pieterh	yeah
[15:09] pieterh	mato: default notification scheme works more or less
[15:10] mato	pieterh: you're getting emails? i thought that wasn't turned on yet
[15:10] pieterh	I didn't check it in detail but it's sending me emails for my own issues, which is fine
[15:10] mato	ok
[15:10] pieterh	just select 'default notification scheme' for project
[15:10] mato	right
[15:10] pieterh	github really don't get it
[15:10] pieterh	how can smart people be so stupid
[15:10] mato	github for some reason intentionally want just a tracker for small (a few people) projects
[15:11] pieterh	ah, yes, it's not open source, of course
[15:11] mato	neither is jira :-)
[15:11] pieterh	github support explicitly said they won't do email notifications cause they don't like email
[15:11] pieterh	indeed, they're reducing the email notifications already in place, it seems
[15:11] mato	pieterh: how was the trip to california?
[15:12] pieterh	mato: it was great
[15:12] pieterh	there are a lot of folk using 0MQ in California and Washington state
[15:12] mato	real users, cool
[15:12] pieterh	real users
[15:12] mato	shame they're so far away...
[15:12] cremes	pieterh: are they targeting any particular platforms?
[15:13] pieterh	I kind of summarized it here: http://unprotocols.org/blog:7
[15:14] mato	?
[15:15] mato	that doesn't tell me anything about our users :-)
[15:15] mikko	this makes me wonder about 3.0
[15:15] mikko	if we have a lot of active users
[15:16] pieterh	well, the point is the use case
[15:16] pieterh	it's cloud and mobile
[15:16] mato	what use case? there's no use case in there
[15:16] mato	stuff on the cloud talks to ... other stuff ... :-)
[15:16] pieterh	you want specific verticals?
[15:16] mikko	what is happening to identities in 3.0?
[15:17] pieterh	mikko: nothing yet, afaik
[15:17] mato	no, i just asked what are users are mostly doing, since you met with a bunch of them
[15:17] cremes	i'm curious about linux versus osx versus windows versus os390, etc
[15:17] mato	ruby vs. python vs. perl vs. java vs....
[15:17] cremes	that too
[15:17] pieterh	ok, specifics: mostly Linux, mostly Python, Ruby, and C/C++, and mostly EC2 or similar
[15:17] mato	mostly web 2.0 startups or also others?
[15:18] pieterh	Microsoft are looking at 0MQ for their clouds, so that's obviously not Linux but they're the only ones
[15:18] mikko	we should have sockopt for being able to disable identities on pub side
[15:18] pieterh	mostly startups that want to build massive cloud services
[15:18] mikko	or maybe a callback when client connects with identity
[15:18] mato	right, so we're fashionable :-)
[15:18] pieterh	think of... mongohq, simplegeo, etc. (these don't use 0MQ but are typical use cases)
[15:18] pieterh	distributed cloud databases
[15:18] pieterh	that's what mobile apps need
[15:18] cremes	great
[15:18] pieterh	the key points are cost and scalability
[15:19] pieterh	it has to be extremely cheap and extremely scalable
[15:19] pieterh	clouds make their money from advertising, basically
[15:19] mato	hmph
[15:19] pieterh	meaning you have to handle a million users to make $1,000 profit (or somesuch)
[15:20] pieterh	the figures are not real, but you get the point
[15:20] mato	ok, any people running real businesses? :-)
[15:20] pieterh	yes, but all in startup mode afaik
[15:20] pieterh	the cycle is obviously 12-18 months and 0MQ/2.x has only been stable for 6 months
[15:20] cremes	any area in particular where the established community should focus?
[15:21] cremes	docs..
[15:21] pieterh	cremes: yes
[15:21] cremes	stability...
[15:21] cremes	new features
[15:21] pieterh	think clouds talking to mobile devices...
[15:21] pieterh	security
[15:21] pieterh	https
[15:21] pieterh	that's the number one request
[15:21] cremes	ok
[15:21] pieterh	number two is probably "should not crash when given bad data"
[15:22] pieterh	there's no number three afaict
[15:22] cremes	yes, it needs to be impervious to fuzzing
[15:22] pieterh	ye
[15:22] pieterh	s
[15:22] mikko	and prevent other DoS possibilities
[15:22] cremes	anything about javascript or 0mq in the browser?
[15:22] pieterh	cremes: nope
[15:22] mikko	like being able to disable explicit identities on server-side
[15:22] cremes	interesting...
[15:22] pieterh	I'm sure some people want to get 0MQ natively in the browser
[15:23] pieterh	there's an actionscript stack
[15:23] pieterh	but it's moot... the real problems are security and firewalls
[15:23] cremes	sure, but apparently none of the startups (think "lots 'o startup cash") want it so it may not get much attention
[15:23] pieterh	oh, request number 2 was actually subports, now that I think about it
[15:24] pieterh	it can be really annoying to have to open 5 ports across a firewall
[15:24] cremes	agreed
[15:24] cremes	but it hasn't really hurt my use of 0mq yet
[15:24] cremes	so i don't feel that particular itch :)
[15:24] pieterh	the consensus in SFO was "named subports in one process"
[15:25] pieterh	anyhow, the interesting news is that there's nothing else like 0MQ for this space
[15:25] pieterh	people like dotcloud started with RabbitMQ then switched to 0MQ
[15:25] pieterh	some others considered writing their own TCP fabric, then abandoned that
[15:25] cremes	yes, most people i have talked with have tried amqp and abandoned it
[15:26] cremes	some went 0mq, others with homegrown
[15:26] pieterh	I expect we'll get a lot of amqp refugees in about 6-12 months as amqp/1.0 emerges
[15:26] cremes	very few have gone with commercial (29west, tibco, etc)
[15:26] pieterh	the main questions will be stuff like "how do I do topic routing"
[15:26] pieterh	we may want to provide standard answers to that
[15:26] cremes	answer: with a device
[15:26] pieterh	i don't see subscription upstreaming working soon
[15:27] pieterh	well, a device that acts like an amqp topic exchange
[15:27] pieterh	not trivial, but not rocket science either
[15:27] cremes	right
[15:27] cremes	i have some vague interest in using 0mq as a foundation for an actor library
[15:28] pieterh	one thing I noticed is that people stick with the version of 0MQ they first use in production
[15:28] cremes	pretty much everything i build is a small single function client (or server)
[15:28] cremes	which maps to the actor model quite nicely
[15:28] pieterh	do you mix them together using inproc?
[15:28] cremes	i use whatever transport makes sense except for the pgm-based ones
[15:28] pieterh	right
[15:28] cremes	so yes, inproc is used
[15:29] pieterh	its a shame we didn't collect useful devices (and device designs) for reuse
[15:30] cremes	that may happen over time; it's not like the ideas aren't still out there
[15:36] nilsh	is a libzmq crash expected on icmp "Destination Net Unreachable" ?
[15:37] cremes	nilsh: no
[15:37] cremes	did it print an assertion?
[15:37] nilsh	errno == ECONNREFUSED \|\| errno == ECONNRESET \|\| errno == ETIMEDOUT \|\| errno == EHOSTUNREACH (tcp_connecter.cpp:300)
[15:42] nilsh	maybe the errno for "Destination Net Unreachable" should be added there to be ok?
[15:42] mikko	yes
[15:48] nilsh	ok i will create a bug report for ths
[17:39] mikko	jira looks good now
[17:39] mikko	mato: can you check?
[17:46] yukonbob	hello, #zeromq
[17:47] mato	mikko: checking...
[17:47] mato	mikko: seems mostly fine
[18:04] mato	mikko: ok, i've deleted the banner asking people not to add stuff
[18:04] mato	mikko: i think we're good to go
[18:04] mikko	cool
[18:04] mikko	still need to handle the domain at some point
[18:04] mato	yes, later