ZeroMq IRC Log

Wednesday May 26, 2010

[Time] Name	Message
[02:09] jugg	is there any documentation/threads discussing the choice of lgpl3 for zeromq?
[04:25] sustrik	cremes: by all means update the faq
[04:26] sustrik	jugg: what exactly are you interested in?
[04:26] sustrik	lgpl vs. proprietary?
[04:30] jugg	v3 is pretty invasive, so before bringing up any concerns, I'm interested in previous discussion to avoid re-hashing those issues/concerns.
[04:32] sustrik	invasive? it's lgpl not gpl
[04:32] sustrik	what are you trying to do?
[04:33] jugg	lgplv3 is gplv3 with certain exceptions made.
[04:33] sustrik	yup, the exceptions result in "you can do whatever you want with the application using the library"
[04:35] zedas	hey, on linux I'm finding zmq_poll kills my app thread, but i can't quite find out why. here's the code: http://mulletdb.com/artifact?name=35bf3679dcef35fdefbfe19130a9d5abe1d4ee03
[04:35] zedas	it's just one thread running a poll loop, but gdb shows that the thread running zmq_poll exits while the two running epoll donot
[04:36] sustrik	zedas: looks like the only way to exit the loop is through the die message
[04:37] zedas	http://dpaste.de/dYJs/ heres' the backtrace in the threads
[04:38] sustrik	aha, or zmq_poll can return error
[04:38] zedas	sustrik_: i thought too, but the thread exits immediately, it doesn't return back to the caller of zmq_poll.
[04:38] jugg	The main concern with v3 is that it controls the hardware system. If, for instance, I wanted to lease/sell a hw system (box), and do so at a low cost because support contracts would be put in place - but including a v3 (lgpl/gpl) piece of software prevents me from locking down the software that is run on the box. I'm happy to provide the source to said software, but I'm not happy to allow other uses of said box. The v3 specfi
[04:38] jugg	cally tries to prevent this (tivo). So, using zeromq in such a platform is not possible.
[04:38] sustrik	then the loop would exit
[04:38] sustrik	can you check the actual error code?
[04:38] zedas	yep, but i don't get an exit
[04:38] zedas	so i put break points right after the zmq_poll call, also tried getting the return value
[04:39] zedas	and not getting it. just boom.
[04:39] zedas	so, insize zmq.cpp there's code that gets the app_thread
[04:39] sustrik	hm, the backtrace looks like it's still inside zmq_poll
[04:39] sustrik	blocked maybe?
[04:40] zedas	oh i stopped it
[04:40] zedas	right at the poll so i can see what thread is dying
[04:40] zedas	so basically i run that for a while, then pound it with a client
[04:40] zedas	works fine but after a few thousand iterations that thread just exits right after the poll and doesn't come back to the caller
[04:41] sustrik	strange
[04:41] zedas	so, i'm wondering if i wanted to try to catch that where would i breakpoint? inside zmq.cpp?
[04:42] sustrik	what about stepping through the zmq_poll implementation and checking where exactly it exists?
[04:42] zedas	i'm thinking that it might actually be exiting, but g++ might optimizing out the actual exit
[04:42] sustrik	zedas, by all means compile with CXXFLAGS="-g -O0"
[04:42] zedas	well, i have to run it a shitload :-)
[04:42] zedas	so it'd be a lot of stepping
[04:43] sustrik	try turning the optimisation off first
[04:43] zedas	yep doing that
[04:43] zedas	also, maybe strace?
[04:44] sustrik	jugg: i am not a IP lawyer :) please, do post your question to the mailing list
[04:44] sustrik	imatix should answer this
[04:44] sustrik	zedas: possibly
[04:45] jugg	ok
[05:01] zedas	sustrik_: nope still doing it
[05:03] sustrik	zedas: it's really strange, i cannot think of thread unexpectedly exiting aside of some serious stack overwrite
[05:03] sustrik	anyway
[05:03] sustrik	sanity check: can you add a printf after the loop?
[05:03] sustrik	just to be 100% sure that the thread disappers rather then exits?
[05:05] zedas	sustrik_: exactly,
[05:05] zedas	sustrik_: yep done that, breakpoints, everything. i'm now thinking memory/stack overflow
[05:06] zedas	doing one more thing next. i do a while(true) so it never exits and see what i get
[05:07] zedas	sustrik_: you're right, this has to be a stack overflow. i shall valgrind it.
[05:08] sustrik	try that -- i really haven't seen it behave like this
[05:08] sustrik	something strange going on
[05:42] zedas	sustrik_: yep, stack smash. there's a piece of another library i'm using that had a stray free where it didn't belong.
[05:42] zedas	removed it and so far it seems to be stable, although leaking. i'll let you know if it holds true
[05:43] sustrik	zedas: great, thanks
[11:08] sjampoo	have been lurking on the mailinglist, zmq_wait_fd looks REALLY useful. Just to make sure i understand this correct. I can poll on the fd with just regular EPOLL and when something is ready i can just call zmq_recv, right?
[11:14] sustrik	sjampoo: it's a bit more complex
[11:14] sustrik	when fd is ready
[11:14] sustrik	it means "something had happened" within 0MQ library
[11:15] sustrik	then you have to check whether there messages available
[11:15] sustrik	but maybe there are no messages
[11:15] sjampoo	Aha.
[11:16] sjampoo	Ok, well it would still help me in some way, i could do the main polling with EPOLL and on a signal i could do a non blocking zmq_poll
[11:16] sustrik	sjampoo: exactly
[11:16] sjampoo	wonder how the performance would differ compared to just building around zmq_poll
[11:16] sustrik	+ the overhead of epoll :)
[11:17] sjampoo	Yes, it would be a bit slower for 0MQ sockets but most of my data is through regular sockets unfortunately
[11:17] sjampoo	And i experienced that using EPOLL is faster for that.
[11:17] sustrik	definitely
[11:18] sjampoo	in this situation i don't really care if the signal fd is for a socket or a context
[11:18] sjampoo	It could even be part of the Poll set
[11:18] sjampoo	but anyway it all makes me happy :)
[11:18] sustrik	great :)
[11:27] jugg	lines 129 and 131 seems to be swapped: http://github.com/mato/zeromq-examples/blob/master/zmq-camera.c from the documentation, it seems the local loopback should be using zmq_bind, and the remote end point should be using zmq_connect. Am I confused?
[11:30] sustrik	jugg: i am not familiar with the example, but it used to work
[11:30] sustrik	why do you believe it's swapped?
[11:36] jugg	zmq_bind - assign a local address to a socket, zmq_connect - connect a socket to a peer address (more detail in the docs), const char local_camera[] = "inproc://local-camera"; and the comment above the two lines in question.
[11:37] jugg	swapped meaning that zmq_bind should be used on local_camera, and zmq_connect should be used on sender_args->endpoint
[11:38] sustrik	ah
[11:38] sustrik	afaiu "local camera" means that it runs in the same process
[11:39] sustrik	"assign a local address to a socket" means local NIC (on this box)
[11:39] sustrik	local != local :)
[11:39] sustrik	the comment is confusing
[11:40] sustrik	maybe local_camera should be renamed to my_camera?
[11:42] jugg	I think I need to better understand the differences between bind/connect...
[11:43] sustrik	the documentation should be improved...
[11:43] sustrik	the last wording we came with was:
[11:43] sustrik	"bind" creates an endpoint
[11:44] sustrik	"connect" establishes a connection to the endpoint
[11:44] sustrik	does that make more sense?
[11:51] jugg	yes, assuming my understanding is correct :P where are endpoints discussed in the documentation?
[11:53] sjampoo	jugg, i am not really sure if there is any specific documentation about that.
[11:53] sjampoo	But i had trouble understanding that as well
[11:53] sjampoo	I had an eureka moment when i saw there 0MQ allows you to design a messaging system at two different levels
[11:54] sjampoo	First the infrastructure, who connects to who (bind/connect)
[11:54] sjampoo	Ie, multiple clients usually connect to a single server which binds to a single port
[11:54] sjampoo	Then at the second level you have the data flow
[11:55] sjampoo	Ie, you can bind to port and subscribe to all message OR publish to it
[11:57] sustrik	sjampoo, yes, that's missing from the docs
[11:58] jugg	hmm, "who connects to who" may hit on some of the missunderstanding for me... is it the case that in a P2P socket, one has to bind, the other connect?
[11:58] jugg	sustrik: diagrams
[11:58] sustrik	jugg: yes
[11:59] sustrik	what kind of diagrams?
[12:00] jugg	connection and data flow
[12:01] jugg	do you have a term for an entity implementing zmq? eg. client or ?
[12:01] jugg	there are various roles of course, client, service, broker etc
[12:03] jugg	but anyway, diagrams showing zmq entities, which one uses connect, which uses bind, and the type of socket used, and data flow between them. ??
[12:04] sustrik	jugg: i fear the connection diagrams would be dead trivial
[12:04] sustrik	showing two nodes
[12:04] sustrik	one doing bind
[12:04] sustrik	another one connect
[12:04] jugg	perfect
[12:04] sustrik	:)
[12:04] jugg	:)
[12:05] pieterh	i'm making new documentation
[12:05] pieterh	it'll be on the website as wiki text
[12:05] pieterh	so anyone can join in and edit
[12:05] jugg	I think I'm getting passed the point of needing the diagrams with this discussion, but newbies are a blank slate, bind,connect,socket have no meaning. Simple diagrams quickly get us passed the blank slate state. :)
[12:05] pieterh	@jugg: ack
[12:05] pieterh	my big advantage is that i'm a newbie :-)
[12:06] pieterh	slates don't come much blanker than me
[12:06] sustrik	i think sjampoo's comment about having two separate phases (establishing connections / sending messages) is important
[12:06] jugg	:)
[12:06] sustrik	if user doesn't get that he's lost
[12:06] pieterh	@sustrik: normally these would be formalized in user-level APIs, no?
[12:07] sustrik	connect/bind vs. send/recv
[12:07] pieterh	topology being hidden
[12:07] jugg	sustrik: your original clarification above: create endpoint, connect to endpoint was a big help as well.
[12:08] pieterh	@jugg: we argued about this for a while, I insist that 0MQ does have connections, except they're not TCP connections but 0MQ connections
[12:08] jugg	my original read of the documentation was that it was necessary to use two different functions depending if I was connecting locally or remotely.
[12:08] pieterh	the local/remote explanation is misleading
[12:08] sustrik	yes, that's confusing
[12:08] pieterh	you can connect to a local socket
[12:09] pieterh	it's about creating directed 0MQ connections (arrows)
[12:09] sustrik	zmq_connect(3) and zmq_bind(3) should be fixed
[12:09] pieterh	in my blank slate mind, anyhow
[12:09] pieterh	@sustrik: it would be helpful IMO to stop saying that "0MQ does not have connections"
[12:10] pieterh	it is freaky to have to "connect" and then think "there are no connections"
[12:10] jugg	now, one more clarification point... even if I bind a socket, I can still send - ie, bind/recv and connect/send are not coupled, correct?
[12:11] jugg	that being to pieterh's point
[12:11] sjampoo	jugg, exactly
[12:11] sustrik	yes
[12:11] pieterh	you can always send but it won't go anywhere, right?
[12:11] jugg	sjampoo: sorry, that was your point. :)
[12:11] sustrik	it's either queued or dropped
[12:11] sustrik	depending on the messaging pattern
[12:11] pieterh	right
[12:12] jugg	does bind/connect have to do more with queues?
[12:12] pieterh	bind creates an endpoint and connect creates a 0MQ connection to that endpoint
[12:12] pieterh	once the connection exists, messages may flow across it
[12:12] jugg	yes, what is an endpoint?
[12:12] pieterh	an endpoint is a socket that accepts connections :-)
[12:13] pieterh	it is a connection endpoint
[12:13] pieterh	not a messaging endpoint
[12:13] sustrik	it can also be a device
[12:13] pieterh	topology construction, ball & socket
[12:13] pieterh	devices can have sockets and ... dare I say it... balls
[12:14] sjampoo	balls?
[12:14] pieterh	i am abusing "socket" now... :-(
[12:14] sjampoo	heh
[12:15] sjampoo	oh, as in a bone joint...
[12:15] jugg	so, in p2p terms, either peer can accept connections (bind) or make a connection (connect), regardless of the direction the connection was made, both peers can send/recv messages over that connection.
[12:15] pieterh	@jugg, last part depends on socket types
[12:15] pieterh	but otherwise, yes
[12:16] jugg	socket type p2p
[12:16] pieterh	usually but not always the "client" connects to the "server"
[12:16] pieterh	but you can usefully have cases where the "server" will connect to the "client", e.g. to get out from firewalls
[12:17] jugg	yes, the peer accepting connections (bind) is the "server", the one making the connection (connect) is the "client"
[12:17] pieterh	it's the common model
[12:17] jugg	I don't think any of my last three comments are incorrect? (hopefully)
[12:17] pieterh	(of course we can do lots of weird stuff but knowing that is not helpful to newbies)
[12:18] pieterh	@jugg, no, but the last one was totally incorrect!
[12:18] pieterh	:-)
[12:18] jugg	poor me :)
[12:20] jugg	ok, the socket type doesn't dictate the server/client role?
[12:21] jugg	eg. I can set up a ZMQ_PUB socket, and it could be used in zmq_bind or zmq_connect, yes?
[12:21] pieterh	@jugg, yes
[12:21] pieterh	a publisher can connect to a subscriber
[12:21] pieterh	but that's a weird usecase which I'd want to hide from newbies
[12:22] jugg	and that goes for the rest of the socket types as well? socket type is bind/connect agnostic?
[12:22] pieterh	yes
[12:22] jugg	ok, things are falling into place. slowly... :)
[12:22] pieterh	@sustrik: i'm likely to say stupid things, please correct me if I'm wrong...
[12:23] sustrik	you are right, np
[12:23] pieterh	nice!
[12:23] jugg	blind leading the blind, how nice :)
[12:23] sustrik	now you see the problems with docs i mentioned last week
[12:23] pieterh	@jugg: i have one eye slightly open... can just about make out the crevasses
[12:23] pieterh	@sustrik: yes, it's pretty clear we have a lot of work to do documenting the basics
[12:24] pieterh	i'll have a first draft ready real soon
[12:26] jugg	ok, best practice documentation, or tradeoff discussion on setting up the network topology - ie, deciding who should bind, who should connect.
[12:27] jugg	eg, how can I assure that messages I'm sending out are not dropped into oblivion if the other side of the connection doesn't yet exist?
[12:28] sustrik	yes
[12:28] pieterh	@jugg, yes
[12:29] jugg	yes, I can assure this?
[12:29] pieterh	yes, documenting best practice
[12:29] pieterh	as to how to assure it, it depends on the socket types
[12:30] pieterh	best practice in some scenarios is "drop stuff you can't deliver"
[12:32] jugg	sorry, slightly lost, this is what you are currently working on documenting, or it is already documented somewhere, if so, where? :)
[12:33] pieterh	@jugg: i am working on that documentation now
[12:34] jugg	cool
[12:34] pieterh	i'll put it up as wiki text with space for comments and/or arbitrary contributions
[12:44] cremes	jugg: regarding your question about assuring messages you send are not dropped because the other side doesn't exist yet...
[12:44] cremes	here is how i solve that
[12:44] cremes	if i am going to have a PUB socket deliver data, i don't allow it to start until there is at least one request
[12:44] cremes	so i pair up my PUB socket with a REP socket that i bind to an endpoint
[12:45] cremes	clients that want data can connect to my REP socket with a REQ socket and send me a request
[12:45] cremes	upon receipt, i begin publishing
[12:46] cremes	that "pattern" has been working well
[12:48] jugg	what does "pair up" mean?
[12:49] jugg	using an zmq concept, or simply as an internal meta relationship?
[12:49] cremes	"pair up" means that my server uses both PUB and REP sockets to provide services; REP for receiving the first request, PUB to send the data
[12:49] cremes	internal meta relationship
[12:49] jugg	ok, that makes sense.
[12:50] cremes	i use REQ/REP sockets to transition my servers through a state machine and other socket types for mass-data delivery
[12:51] cremes	the semantics of each 0mq socket type make this easier to think about nwo that i have gotten past the bind/connect confusion :)
[12:54] jugg	so any quick pointers on choosing bind/connect and socket types? Currently, I have several nodes, four of them are chained in a one way message passing n1->n2->n3->n4. Another node (n0) exists that receives requests from any other nodes and responds to those messages n1@->n0, n2@->n0, n3@->n0, n4@->n0 (the @ deliniating response to -> request). Also, n3 must subscribe to publications by n0 (which is the only publication from n0).
[12:55] jugg	the REQ/REP seems obvious for the nX@->n0 interface
[12:55] jugg	But I'm uncertain whether to use PUB/SUB or P2P for the chaining.
[12:56] jugg	I think what cremes just described makes good sense for the n3 subscription to n0.
[12:57] jugg	I'd probably go with bind/connect based on message direction for the chaining, the receiving binding, the sender connecting. But I have no basis for that other than simplicity of thinking.
[13:00] jugg	hmm, n1-4 also send messages to n0 which don't require a response.
[13:00] sustrik	cremes: connecting PUB socket to REP socket is not a good idea
[13:01] sustrik	why not have REQ instead of PUB?
[13:01] jugg	I don't think he is doing that?
[13:01] sustrik	ah
[13:01] sustrik	pair
[13:01] sustrik	!= connect
[13:02] sustrik	sorry for confusion
[13:05] jugg	ipc vs tcp loopback, any obvious reason to go one way or the other?
[13:05] sustrik	ipc is faster on linux
[13:07] sustrik	pieterh: here you go - how to run perf tests: http://www.zeromq.org/results:perf-howto
[13:07] pieterh	thanks!
[13:08] jugg	I think in the nX@->n0 interface, n0 will bind, nX will connect.
[13:09] jugg	is there a performance difference in the chaining example if I use PUB/SUB vs P2P
[13:09] cremes	jugg: right, n0 should probably bind and let the other nodes connect to it
[13:10] cremes	jugg: p2p is kind of broken right now; use PUB
[13:10] sustrik	right
[13:11] sustrik	n1-n4 is a processing pipeline?
[13:11] jugg	yes
[13:12] sustrik	have a look at UPSTREAM/DOWNSTREAM sockets then
[13:12] sustrik	that's what they are meant for
[13:12] sustrik	summary:
[13:13] sustrik	n1->n2->n3->n4: UPSTREAM/DOWNSTREAM
[13:13] sustrik	nX@->n0: REQ/REP
[13:13] sustrik	publications from n0: PUB/SUB
[13:16] jugg	hmm, so each of n1-n4 could be load balanced. or if only one instance of each n1-n4 exists, it'll simply behave as a one way message passing?
[13:18] sustrik	it's load-balancing among one service :)
[13:18] sustrik	= simply pass the message to next node
[13:18] jugg	yes, ok. :)
[13:18] sustrik	however you can add more nodes in the future if needed
[13:19] jugg	why is "component" emphasised in the docs, does it mean something special? It isn't just a node like any other?
[13:19] sustrik	where's that?
[13:20] jugg	" Parallelized pipeline pattern" for the socket type
[13:20] sustrik	no idea
[13:20] sustrik	probably a typo
[13:21] sustrik	mato: any reason for that?
[13:21] jugg	and is "messaging pattern" and "socket type" something different?
[13:21] sustrik	yes, patter is say "request/reply"
[13:21] sustrik	it consists of REQ and REP socket types
[13:21] jugg	ah
[13:22] jugg	that is obvious, I was confusing myself.
[13:24] jugg	or perhaps the text: "The following messaging patterns are defined" could be changed to something like "The following messaging patterns are defined describing which socket types should be used for binding and connecting to an endpoint"
[13:24] jugg	or not...
[13:25] jugg	trying to processing too much info :)
[13:26] jugg	sustrik: thanks for the summary above
[13:26] sustrik	jugg: feel free to submit documentation patches
[13:28] jugg	I assume the docs on the website correspond to the zeromq2/docs/*.txt (I've been reading the website)
[13:28] jugg	and it looks like patches should be sent to the mailing list vs pull requests via github?
[13:29] sustrik	via mailing list, stating it's submitted under MIT license
[13:29] sustrik	pieter: the people who join the wiki cannot edit pages
[13:30] sustrik	is that deliberate?
[13:30] sustrik	pieterh:
[13:30] pieterh	sustrik: ja?
[13:30] sustrik	see above
[13:30] pieterh	i see, hang on
[13:30] pieterh	it's normal for some categories, not for others
[13:30] pieterh	let me double check, what categories are you talking about?
[13:31] jugg	http://www.zeromq.org/area:licensing well that answers my gpl3 tivo question
[13:31] sustrik	http://www.zeromq.org/build-v20
[13:32] pieterh	@sustrik: pages in the default area need to be moved
[13:32] pieterh	sorry, default catergory
[13:32] pieterh	and the area: category is not open to all people to edit
[13:32] pieterh	the wiki probably needs redesigning, we should discuss this
[13:33] pieterh	@jugg: tivoization is a perfectly good business model for some people
[13:34] sustrik	???!!!
[13:34] sustrik	i cannot edit nav:side :(
[13:34] pieterh	sustrik: ? hang on
[13:35] jugg	meh, zeromq licensing makes no sense.
[13:36] pieterh	sustrik: if you can't edit a page it's probably because you are not logged in
[13:36] pieterh	@jugg: what do you mean?
[13:37] pieterh	@sustrik: you are site admin, always have been, and have full edit rights over everything
[13:38] sustrik	oops
[13:38] sustrik	it's ok
[13:40] jugg	pieterh: well, ignoring the stupidity of gpl3 anti-tivo, and looking only at my contributions must be MIT licensed (purely for the benefit of imatix), I myself and others can only use contributions under lgpl3. Basically, this means that contributors are working for free for imatix. Normally open source projects, contributs work for free for the project. This is not the case here.
[13:40] jugg	sorry for grammar/spelling, I'm getting tired... :)
[13:41] pieterh	@jugg, fair points
[13:41] pieterh	however imatix works for free for you too
[13:41] pieterh	so it's not an unfair deal to ask that we can try to make a viable business
[13:41] pieterh	you don't need to give us patches
[13:42] pieterh	however if you want us to maintain your patches in the official product, we ask for MIT licensing OR copyright transfer
[13:43] pieterh	you can always just redistribute your modified libraries under the LGPL, period
[13:43] pieterh	as for the GPL3's "stupidity"... we disagree
[13:43] pieterh	i've been using the GPL since... 1991 or so for my work
[13:44] pieterh	and the principle of enforcing "share-alike" is important IMO
[13:45] pieterh	what do you see as the problem with this model?
[13:46] sustrik	anti-tivo caluse in gpl3 is not about share-alike, it's about free (as is speech) hardware
[13:46] pieterh	why do you say so?
[13:47] pieterh	it is about preventing people from not sharing
[13:47] pieterh	it is about enforcing the spirit of the GPL: you use my work, you share improvements
[13:47] sustrik	you can redistribute the source and still lock you hw device no?
[13:47] pieterh	Tivo hacked the GPL to find a way around that
[13:47] sustrik	i am not an expert anyway
[13:48] pieterh	The GPL does not try to force anything on HW producers
[13:48] pieterh	it is purely a license for using someone's work
[13:48] sustrik	ok, ianal
[13:49] pieterh	ianal either but have been working with licenses for close on two decades
[13:49] pieterh	the dual-license option is actually win-win
[13:50] pieterh	commercial firms can still build no-share products but they pay for the right to escape, so they are not cheating
[13:50] pieterh	and contributors are assured that their work is not closed off
[13:51] pieterh	@jugg: are you still around?
[13:51] jugg	pieterh: gpl3 tivo is purely about enforcing open hardware. It has nothing to do with software. ESR himself states that he is scared that hardware vendors will implement restrictions that only allow running of signed software, preventing open source software from running on that hardware. His aim is to ensure the hardware paltform remains open. The GPL2 was just fine in ensure the opening of software.
[13:52] pieterh	ESR has nothing to do with GPL... however let me do some more research
[13:52] pieterh	are you asking that we move to GPL2?
[13:53] pieterh	that is not going to happen, I can say it categorically
[13:53] pieterh	:-)
[13:53] jugg	appologizes certainly isn't ESR :)
[13:54] pieterh	maybe you mean Eben Moglen ?
[13:54] jugg	anyway, ignoring my slipup of ESR vs Richard Stallman.
[13:55] jugg	http://gplv3.fsf.org/rms-why.html
[13:57] pieterh	so to review the tivo issue
[13:57] pieterh	first, tivo began using modified GPL software and not providing the source code
[13:57] pieterh	e.g. using a modified 0MQ and not providing patches
[13:58] pieterh	then they accepted to provide their modifications
[13:58] jugg	to your earlier points, I do not begrudge imatix from making a viable business. Certainly from imatix's stand point, requesting MIT or Copyright transfer makes perfect sense.
[13:58] pieterh	but they introduced hardware checking on their boxes so you could not remix those modifications
[13:58] pieterh	so you'd get e.g. a 0MQ with a driver for some network card
[13:58] pieterh	but you couldn't improve that driver because the Tivo borked
[13:58] jugg	pieterh: them not providing source code isn't fixed by gpl3.
[13:59] pieterh	no, but it prevents them using the GPL3 code at all
[13:59] pieterh	OR they must not prevent remixing (on their boxes)
[13:59] pieterh	i concede...
[13:59] pieterh	that it's about stopping people running arbitrary code on a box that they think they own but actually don't
[13:59] jugg	them locking the hw to only run their signed version of the code is their perogative imo. As long as they provide the modified source, I don't care of that source can't be run on their hardware.
[13:59] pieterh	you don't care
[14:00] pieterh	i do :-)
[14:00] pieterh	now, why i care is not ideological
[14:00] jugg	why? Go build your own hardware that matches theirs and run the modified code on it.
[14:00] pieterh	nah, i sell software :-)
[14:00] pieterh	if a firm is bothered enough (and has money enough) to lock their hardware
[14:00] pieterh	then they can buy commercial licenses
[14:00] pieterh	and thus fund 0MQ development
[14:01] pieterh	i see GPL3 as an ultracapitalist tool
[14:01] jugg	how does them preventing modified versions of the software from running on their hardware hurt you?
[14:01] pieterh	it's not really the point
[14:01] jugg	?!
[14:01] pieterh	the point is, we're a business in the end
[14:02] pieterh	we need to find ways of getting money from people who use 0MQ for profit
[14:02] pieterh	making software like 0MQ is expensive, though obviously a community like ours makes a huge difference
[14:03] pieterh	it lets us leverage our investment
[14:03] jugg	so, tivoication is simply an indicator for you of the depth of the hw companies pocket?
[14:03] pieterh	i think that's accurate, yes
[14:03] pieterh	sorry to be blunt :-)
[14:03] pieterh	i've been making free software since 1991 like i said
[14:03] jugg	blunt is clear. :)
[14:03] pieterh	and i love the ideological aspects
[14:04] pieterh	but they do seem to reflect economic realities that also work
[14:04] pieterh	forcing those with deep pockets to put money on the table is win-win
[14:05] pieterh	i do really appreciate the work that the FSF (Eben Moglen, mainly) put into GPL3
[14:05] jugg	and at the same time kill the companies without deep pockets who's only way to make money is by using tivoication to drive service and support contracts.
[14:05] pieterh	kill is a harsh word
[14:06] pieterh	how can our license terms kill a firm?
[14:06] pieterh	they can just write their own software
[14:06] pieterh	or buy commercial software
[14:06] jugg	which would kill them :)
[14:06] pieterh	or buy a commercial license from us
[14:06] pieterh	sorry, but i pay people
[14:06] pieterh	and when they send me invoices i don't say, "you're killing me!"
[14:06] pieterh	that... would be... silly
[14:06] jugg	:)
[14:06] pieterh	indeed
[14:07] sustrik	sorry to interrupt you, i'm just interested -- what's the point of tivoising a hw device?
[14:07] sustrik	i mean business point
[14:07] pieterh	it's a strange world when simply proposing to charge a modest amount for a good product translates into accusations of premeditated murder... :-)
[14:07] pieterh	sustrik: it means you can cripple the hardware
[14:07] sustrik	i mean business point
[14:07] pieterh	run software on it that locks down features
[14:07] pieterh	unlock features for money
[14:07] jugg	sustrik: it means that you can lease or sell hw at a low cost in exchange for ongoing service contracts.
[14:08] sustrik	a-ha
[14:08] sustrik	understood
[14:08] sustrik	thanks
[14:08] pieterh	e.g. you buy a mobile phone that has VoIP locked/disabled
[14:08] pieterh	if i can get the source code, i can unlock it, run a patched version
[14:08] pieterh	the Tivo solution is that the hw won't run that modified / improved version
[14:08] jugg	it means a sofware company can survive, by providing a hw platform that enables them to make money on their software.
[14:09] pieterh	jugg: do you work for such a company? just curious
[14:09] pieterh	I could argue: it means a firm can take work done by others and not contribute anything back
[14:09] pieterh	it is specifically about taking free software, don't forget
[14:09] jugg	pieterh: not currently.
[14:09] jugg	pieterh: absolutely not true, they must contribute back the source they've modified.
[14:10] pieterh	true
[14:10] jugg	Just because they provided a hardware platform doesn't mean because you wrote the software that you have the right to that hw paltform.
[14:10] pieterh	i misstated myself, sorry
[14:10] pieterh	well, put this another way
[14:10] pieterh	a commercial license can be for 1 euro
[14:10] pieterh	it really depends
[14:10] jugg	but that is what the gpl3 wants to state. it wants to state that someone who wrote software must have control over the hardware that software is ran on. rediculous.
[14:11] pieterh	this is only one part of the GPL3
[14:11] jugg	you write the software, you get control over the software.
[14:11] jugg	pieterh: I agree, which is why a gpl2.5 is needed.
[14:11] pieterh	actually, you get the software (as a user) in a box, you get the right to hack that box
[14:11] pieterh	the point is the sharing of knowledge enforced by private contract
[14:12] pieterh	when we invest in 0MQ we tell box makers:
[14:12] pieterh	- if you use OUR software (and we speak in name of our contributors too) on your box
[14:12] jugg	pieterh: absolutely not. the box is leased, you have no right to it. Or the box is sold at a very low cost with the expectation that ongoing service is provided for that box.
[14:12] pieterh	- then you MUST allow people to improve that software AND run the improvements on your box
[14:12] pieterh	simple
[14:12] pieterh	- OR you pay us something and we'll make an exception for you
[14:12] pieterh	seems simple enough
[14:13] pieterh	and, honestly, it seems totally fair
[14:13] pieterh	look, @jugg, if/when you work for a firm that finds itself in this situation, let me know
[14:14] pieterh	and we'll bend over backwards to get as close to that nominal 1 Euro as possible
[14:14] pieterh	:-)
[14:14] pieterh	or perhaps make a specific exception to the Tivo parts for that firm
[14:15] jugg	:) Like I said, I don't begrudge imatix anything, but the idealogical concept that someone can write software and believe by the use of that software they have the ability to assert some right and control over the hardware is asinine. (to be blunt)
[14:16] pieterh	@jugg: there is clearly a conflict of interests
[14:16] jugg	anyway... sorry for dragging this channel through all of that. :)
[14:16] pieterh	recall that the tivo clauses came from real conflicts, not some theoretical debate
[14:17] pieterh	and in the end, if a developer wants to license his work under a license that says, "you must wear pink while singing 'When the Saints'", that's their choice
[14:17] pieterh	private contract
[14:18] pieterh	there is no post-facto modification, no ambush, no traps
[14:18] jugg	no, as you've stated yourself, it came from the ability to reach into another companies pockets. The gpl2 was complied with in the end, source was provided. Nothing further was necessary imo. (although I had no idea zmq was part of this, so I surely stepped on the hornets nest!)
[14:18] pieterh	@jugg: that is my view on it, it's not what drove the FSF's changes
[14:19] sustrik	what about turning it upsde down: idealogical concept that someone can make hadware and believe by the use of that hardware they have the ability to assert some right and control over the software is asinine :)
[14:19] pieterh	the GPL3 was in part about patents, in part about people hacking the word "distribute" and in part about issues like this
[14:19] pieterh	again
[14:19] pieterh	someone produces a work and licenses their work
[14:19] pieterh	other parties can accept / reject these licenses
[14:20] pieterh	there is no "asinine" just "I accept" or "I do not accept"
[14:24] jugg	my only issue with gpl3 is the tivo part. The rest, patents and the like, trully deal with the software freedom. The touted aspect of the GPL is about software freedom not about a tool for making money. So when a company uses the GPL as a tool for making money, I can't meet that argument on equal ground. Take the GPL terms and relabel it as your own license - it has the same affect without confusion as to why it is being use
[14:24] jugg	d. By using the "GPL" name, then you are stating that your interested in software freedom and open source.
[14:24] pieterh	@jugg: it is not practical to define custom licenses
[14:25] pieterh	except on a case by case basis
[14:25] pieterh	if we tried to take the GPL3 and customize it
[14:25] jugg	I never said customize, I said rename.
[14:25] pieterh	then we would force every single serious firm to do a full legal review of our license
[14:25] jugg	actually I said "relabel" :)
[14:25] pieterh	?
[14:25] pieterh	what are you actually proposing? I've lost the plot now
[14:26] pieterh	freedom is not about NOT making money
[14:26] pieterh	freedom is precisely about a more effective market in knowledge
[14:36] jugg	I think I need to wrap this up on my end, but to try to clarify the lost plot... the gpl is about software freedom, not about a tool to make money, in fact quite the opposite. So, if a company uses the name "GPL" it is stating that the GPL licensed software is about freedom of the software. If that isn't the case, and the GPL terms are being used as a way to ensure alertnate licensing deals, then that company (imo) should re
[14:36] jugg	package the GPL terms under a different name.
[14:37] jugg	That said I'm trying to make money from software as well, so I'm not in ideological alignment with the purpose of the GPL myself in all instances. However, when I used the GPL as I often do, it is about the software only.
[14:37] pieterh	this would create an enormous amount of extra work for many people
[14:37] jugg	what extra work?
[14:37] pieterh	today, any firm wanting to use 0MQ in their apps must do a legal review of the license terms
[14:37] pieterh	it is a contract, don't forget
[14:37] pieterh	the LGPL is a standard contract
[14:38] pieterh	reviewed once, used many times
[14:38] pieterh	if we make a custom license it must be reviewed specifically
[14:38] pieterh	that is called "legal clearance" and it is expensive
[14:38] pieterh	lawyer hours and days and weeks
[14:38] pieterh	sufficiently expensive that many firms will simply refuse to use "open source" that does not have a standard licence
[14:39] pieterh	we would be shooting ourselves in the foot if we did what you propose
[14:39] pieterh	suggestion registered... and rejected... sorry! :-)
[14:40] jugg	Funny, too bad GNU didn't think of a way to scope out the pockets of firms trying to use the GPL to make money and figure out a way to get money out of those pockets... :)
[14:42] pieterh	@jugg: well, the FSF is a charity
[14:42] pieterh	not a business
[14:42] pieterh	there have been some very large businesses built around the GPL
[14:43] jugg	there I go confusing my terms again, I really should stop already. :)
[14:44] pieterh	think of... Red Hat, or Cygnus Solutions
[14:44] pieterh	iMatix is actually a corporate sponsor of the FSF
[14:53] jugg	please don't take any of this as argument directly against imatix. My main argument is against the GPL3 tivoization clause. The fact that imatix uses it as a way to force commercial licensing deals is compeltely imatix's prerogative. But at an idealogical level, I associate the name GPL with free software, not as a tool to make money. As such, at an idealogical level, I'd like companies not to use the name GPL for licensi
[14:53] jugg	ng software in this way as it creates confusion.
[14:54] jugg	obviously, that ideology isn't going to change imatix's stance... :)
[14:54] pieterh	@jugg, the argument is a useful one and I certainly don't take it personally
[14:55] pieterh	iMatix has been using free software licenses since 1996
[14:55] pieterh	we used to write our own (our old SFL license is one of those recognized by the FSF)
[14:55] pieterh	and we've tried many variations...
[14:56] pieterh	GPL is the one that works, and I've spent enough time with the FSF and thinking about the rationale to understand why
[14:56] pieterh	it is not money vs. free, that is a false dichotomy
[14:56] pieterh	it is about creating the most efficient knowledge economy in which all contributors get the most out of their work
[14:57] pieterh	all parties have the choice to stay out of that economy
[14:57] pieterh	but if they participate, they follow the rules
[14:58] pieterh	there is no ideology, just more or less efficient contracts
[14:59] pieterh	s/efficient/successful/
[14:59] pieterh	sorry to be soapboxing, I think we're boring the channel now :-)
[14:59] pieterh	hey guise, I want to rename all the socket types!
[15:00] guido_g	hehehe
[15:00] pieterh	ZMQ_DONT_TRY_THIS
[15:00] pieterh	ZMQ_OMG_ITS_AMAZING
[15:00] pieterh	and ZMQ_MAGIC_HAPPENS
[15:00] sjampoo	Hmmm, what would ZMQ_MAGIC_HAPPENS do?
[15:00] guido_g	and of course: ZMQ_IT_MIGHT_EVEN_WORK
[15:00] pieterh	"I tried the ZMQ_DONT_TRY_THIS socket and my app crashed?"
[15:01] pieterh	sjampoo: that depends on the time of day
[15:01] jugg	certainly, but I think the change of rules between gpl2 and gpl3 is significant, and it leaves a lot of us who were part of the gpl2 economy, who helped build that enconmy, in an ackward place. Anyway, I agree, time to move on. Thanks for the discussion, I appreciate knowing where zeromq stands in this regards.
[15:01] sjampoo	excellent, i was just looking for something like that
[15:02] pieterh	@sjampoo: if that doesn't work, try guido_g's new ZMQ_IT_MIGHT_EVEN_WORK
[15:02] sjampoo	pieterh, thanks for the tip!
[15:02] pieterh	np... hang on, i need to update the FAQ
[15:03] pieterh	create a special release of 0MQ where all the names are... weird
[15:03] pieterh	somewhere, someone would use it and not get the joke, and would, like months later come back
[15:04] pieterh	"I'm using ZMQ_NULL_WRITES_ENABLED and losing messages, is that normal?"
[15:18] jugg	in zmq_socket messaging patterns request-reply description, the terms "client" and "service" is used, I assume the defines a node that uses zmq_connect or zmq_bind respectively?
[15:19] sustrik	jugg: client is REQ socket, service is REP socket
[15:24] jugg	ok, but "A socket of type ZMQ_REQ is used by a client" would seem to say, a client already exists and a client would use a socket type of ZMQ_REQ, which if just reading the documentation makes me ask, what is a client? (hence, so I did)
[15:25] sustrik	ok, more precisely: client is an application that uses REQ socket
[15:25] jugg	Am I abusing the term node?
[15:25] jugg	eh, I didn't use "node" this time. ignore me.
[15:26] sustrik	:)
[15:26] sustrik	anyway, the problem with this definition of client
[15:26] sustrik	is that if a single app uses both REQ and REP sockets
[15:26] sustrik	it's both client and server at once
[15:27] guido_g	you're in the clients role when you request a service, i'd say
[15:27] sustrik	documentation should probably refrain on defining categories of applications
[15:27] sustrik	application is just application
[15:27] sustrik	it may have some sockets open
[15:28] jugg	"Each request sent is load-balanced among all connected services." facinating, so for that last to be true, the "services" would be connecting to the client's bound endpoint.
[15:28] sustrik	or you can connect multiple times
[15:28] sustrik	0mq allows multiple connects
[15:28] sustrik	as well as multiple binds
[15:29] sustrik	guido_g: yes, but that kind of distinction is good for tutorial, not the formal definition
[15:30] guido_g	huh?
[15:30] guido_g	why so?
[15:30] sustrik	because application can be both client and service
[15:30] guido_g	this is why i said "role"
[15:31] sustrik	ah, got it
[15:31] guido_g	one application can take on multiplke roles
[15:31] sustrik	yes
[15:31] guido_g	some apps even at the same time (threading on multicore)
[15:33] sustrik	multiple roles for same app are pretty common, say an application is both _publisher_ of stock quotes and trade execution _service_
[15:33] guido_g	exactly
[15:56] jugg	rc = zmq_bind(s, "ipc:///tmp/a/0"); fails, but rc = zmq_bind(s, "ipc:///tmp/0"); succeeds. Is it expected that only the directory part must exist?
[15:57] jugg	s/only //
[15:57] jugg	"/tmp/" exists on my system, "/tmp/a/" does not
[15:58] sustrik	it translates directly to UNIX domain sockets
[15:58] sustrik	have a look what do they allow and what they don't
[16:12] jugg	And I'm off for now. Thanks for all of the help/discussion. :)
[17:11] zedas	sustrik: hey, there may be a memory leak in the I/O thread: http://dpaste.de/gM4s/ that's valgrind output from a test server
[17:12] zedas	looks like zmq::pipe_t is making ram and then it's not getting freed properly. not sure exactly where though so thought you might have insight.
[17:17] zedas	sustrik: could it be because the ~pipe_t hasn't run?
[18:51] cremes	are functions zmq_sleep, zmq_stopwatch_start and zmq_stopwatch_stop part of the official zmq api? should non-C language bindings expose them?
[18:56] mato	cremes: nope, they're not
[18:56] cremes	mato: ok
[18:57] mato	cremes: they're in there because of the perf tests, and should probably be moved somewhere
[18:57] sjampoo	Yah, they are handy for perf tests. PyZMQ removed stopwatch at first, but then the performance results wheren't really comparable so they have been moved back in.
[18:58] cremes	sounds reasonable
[18:58] mato	sustrik: good discussion about connect/bind and endpoints, i will rewrite the relevant portions of the reference
[18:58] mato	sjampoo: don't rely on them, they might go away...
[18:59] mato	we don't want people to the the wrong idea that we're building a portability library
[18:59] sjampoo	aren't you guys there already ;)
[19:00] sjampoo	What are the future plans? I think i saw some flirting with moving into the kernel.
[19:01] mato	that's part of it, but a long way away
[19:01] mato	sustrik is the expert on the grand vision
[19:01] mato	deploying 0mq netwide
[19:02] mato	take a look here: http://250bpm.wikidot.com/local--files/start/towards.pdf
[19:02] sjampoo	ah thanks!
[20:05] sustrik	zedas: thanks for reporting it
[20:05] sustrik	i'll have a look at it tomorrow (late evening here)
[20:07] sustrik	sjampoo: basically, the idea is to allow for distributed networks of nodes with no centralised control
[20:07] sustrik	think of a marketplace for computational services
[20:08] sustrik	a world-wide cloud
[20:08] sustrik	but that's pretty far ahead
[20:14] zedas	sustrik: if i can figure it out i'll shoot you a patch
[20:14] zedas	otherwise you know better than me probably
[20:14] sustrik	zedas: thanks
[21:07] sustrik	sjampoo: let me give an example
[21:08] sustrik	zed have created a database accessible via 0mq
[21:08] sustrik	he may have used it just as a convenient way to provide network access
[21:08] sustrik	but:
[21:09] sustrik	think of three companies providing storage space using mulletdb
[21:09] sustrik	you can obviously connect to one of them
[21:09] sustrik	but you can put a device in a middle and let it load balance your requests among the three
[21:09] sustrik	once we implement priorities
[21:10] sustrik	you can even say "store the data primarily at A, but it it's not available, use B"
[21:10] sustrik	etc.
[21:21] mikko	would be cool to create such a system
[21:21] mikko	and use apache zookeeper for syncing