Wednesday May 26, 2010

[Time] NameMessage
[02:09] jugg is there any documentation/threads discussing the choice of lgpl3 for zeromq?
[04:25] sustrik cremes: by all means update the faq
[04:26] sustrik jugg: what exactly are you interested in?
[04:26] sustrik lgpl vs. proprietary?
[04:30] jugg v3 is pretty invasive, so before bringing up any concerns, I'm interested in previous discussion to avoid re-hashing those issues/concerns.
[04:32] sustrik invasive? it's lgpl not gpl
[04:32] sustrik what are you trying to do?
[04:33] jugg lgplv3 is gplv3 with certain exceptions made.
[04:33] sustrik yup, the exceptions result in "you can do whatever you want with the application using the library"
[04:35] zedas hey, on linux I'm finding zmq_poll kills my app thread, but i can't quite find out why. here's the code:
[04:35] zedas it's just one thread running a poll loop, but gdb shows that the thread running zmq_poll exits while the two running epoll donot
[04:36] sustrik zedas: looks like the only way to exit the loop is through the die message
[04:37] zedas heres' the backtrace in the threads
[04:38] sustrik aha, or zmq_poll can return error
[04:38] zedas sustrik_: i thought too, but the thread exits immediately, it doesn't return back to the caller of zmq_poll.
[04:38] jugg The main concern with v3 is that it controls the hardware system. If, for instance, I wanted to lease/sell a hw system (box), and do so at a low cost because support contracts would be put in place - but including a v3 (lgpl/gpl) piece of software prevents me from locking down the software that is run on the box. I'm happy to provide the source to said software, but I'm not happy to allow other uses of said box. The v3 specfi
[04:38] jugg cally tries to prevent this (tivo). So, using zeromq in such a platform is not possible.
[04:38] sustrik then the loop would exit
[04:38] sustrik can you check the actual error code?
[04:38] zedas yep, but i don't get an exit
[04:38] zedas so i put break points right after the zmq_poll call, also tried getting the return value
[04:39] zedas and not getting it. just boom.
[04:39] zedas so, insize zmq.cpp there's code that gets the app_thread
[04:39] sustrik hm, the backtrace looks like it's still inside zmq_poll
[04:39] sustrik blocked maybe?
[04:40] zedas oh i stopped it
[04:40] zedas right at the poll so i can see what thread is dying
[04:40] zedas so basically i run that for a while, then pound it with a client
[04:40] zedas works fine but after a few thousand iterations that thread just exits right after the poll and doesn't come back to the caller
[04:41] sustrik strange
[04:41] zedas so, i'm wondering if i wanted to try to catch that where would i breakpoint? inside zmq.cpp?
[04:42] sustrik what about stepping through the zmq_poll implementation and checking where exactly it exists?
[04:42] zedas i'm thinking that it might actually be exiting, but g++ might optimizing out the actual exit
[04:42] sustrik zedas, by all means compile with CXXFLAGS="-g -O0"
[04:42] zedas well, i have to run it a shitload :-)
[04:42] zedas so it'd be a lot of stepping
[04:43] sustrik try turning the optimisation off first
[04:43] zedas yep doing that
[04:43] zedas also, maybe strace?
[04:44] sustrik jugg: i am not a IP lawyer :) please, do post your question to the mailing list
[04:44] sustrik imatix should answer this
[04:44] sustrik zedas: possibly
[04:45] jugg ok
[05:01] zedas sustrik_: nope still doing it
[05:03] sustrik zedas: it's really strange, i cannot think of thread unexpectedly exiting aside of some serious stack overwrite
[05:03] sustrik anyway
[05:03] sustrik sanity check: can you add a printf after the loop?
[05:03] sustrik just to be 100% sure that the thread disappers rather then exits?
[05:05] zedas sustrik_: exactly,
[05:05] zedas sustrik_: yep done that, breakpoints, everything. i'm now thinking memory/stack overflow
[05:06] zedas doing one more thing next. i do a while(true) so it never exits and see what i get
[05:07] zedas sustrik_: you're right, this *has* to be a stack overflow. i shall valgrind it.
[05:08] sustrik try that -- i really haven't seen it behave like this
[05:08] sustrik something strange going on
[05:42] zedas sustrik_: yep, stack smash. there's a piece of another library i'm using that had a stray free where it didn't belong.
[05:42] zedas removed it and so far it seems to be stable, although leaking. i'll let you know if it holds true
[05:43] sustrik zedas: great, thanks
[11:08] sjampoo have been lurking on the mailinglist, zmq_wait_fd looks REALLY useful. Just to make sure i understand this correct. I can poll on the fd with just regular EPOLL and when something is ready i can just call zmq_recv, right?
[11:14] sustrik sjampoo: it's a bit more complex
[11:14] sustrik when fd is ready
[11:14] sustrik it means "something had happened" within 0MQ library
[11:15] sustrik then you have to check whether there messages available
[11:15] sustrik but maybe there are no messages
[11:15] sjampoo Aha.
[11:16] sjampoo Ok, well it would still help me in some way, i could do the main polling with EPOLL and on a signal i could do a non blocking zmq_poll
[11:16] sustrik sjampoo: exactly
[11:16] sjampoo wonder how the performance would differ compared to just building around zmq_poll
[11:16] sustrik + the overhead of epoll :)
[11:17] sjampoo Yes, it would be a bit slower for 0MQ sockets but most of my data is through regular sockets unfortunately
[11:17] sjampoo And i experienced that using EPOLL is faster for that.
[11:17] sustrik definitely
[11:18] sjampoo in this situation i don't really care if the signal fd is for a socket or a context
[11:18] sjampoo It could even be part of the Poll set
[11:18] sjampoo but anyway it all makes me happy :)
[11:18] sustrik great :)
[11:27] jugg lines 129 and 131 seems to be swapped: from the documentation, it seems the local loopback should be using zmq_bind, and the remote end point should be using zmq_connect. Am I confused?
[11:30] sustrik jugg: i am not familiar with the example, but it used to work
[11:30] sustrik why do you believe it's swapped?
[11:36] jugg zmq_bind - assign a local address to a socket, zmq_connect - connect a socket to a peer address (more detail in the docs), const char local_camera[] = "inproc://local-camera"; and the comment above the two lines in question.
[11:37] jugg swapped meaning that zmq_bind should be used on local_camera, and zmq_connect should be used on sender_args->endpoint
[11:38] sustrik ah
[11:38] sustrik afaiu "local camera" means that it runs in the same process
[11:39] sustrik "assign a local address to a socket" means local NIC (on this box)
[11:39] sustrik local != local :)
[11:39] sustrik the comment is confusing
[11:40] sustrik maybe local_camera should be renamed to my_camera?
[11:42] jugg I think I need to better understand the differences between bind/connect...
[11:43] sustrik the documentation should be improved...
[11:43] sustrik the last wording we came with was:
[11:43] sustrik "bind" creates an endpoint
[11:44] sustrik "connect" establishes a connection to the endpoint
[11:44] sustrik does that make more sense?
[11:51] jugg yes, assuming my understanding is correct :P where are endpoints discussed in the documentation?
[11:53] sjampoo jugg, i am not really sure if there is any specific documentation about that.
[11:53] sjampoo But i had trouble understanding that as well
[11:53] sjampoo I had an eureka moment when i saw there 0MQ allows you to design a messaging system at two different levels
[11:54] sjampoo First the infrastructure, who connects to who (bind/connect)
[11:54] sjampoo Ie, multiple clients usually connect to a single server which binds to a single port
[11:54] sjampoo Then at the second level you have the data flow
[11:55] sjampoo Ie, you can bind to port and subscribe to all message OR publish to it
[11:57] sustrik sjampoo, yes, that's missing from the docs
[11:58] jugg hmm, "who connects to who" may hit on some of the missunderstanding for me... is it the case that in a P2P socket, one has to bind, the other connect?
[11:58] jugg sustrik: diagrams
[11:58] sustrik jugg: yes
[11:59] sustrik what kind of diagrams?
[12:00] jugg connection and data flow
[12:01] jugg do you have a term for an entity implementing zmq? eg. client or ?
[12:01] jugg there are various roles of course, client, service, broker etc
[12:03] jugg but anyway, diagrams showing zmq entities, which one uses connect, which uses bind, and the type of socket used, and data flow between them. ??
[12:04] sustrik jugg: i fear the connection diagrams would be dead trivial
[12:04] sustrik showing two nodes
[12:04] sustrik one doing bind
[12:04] sustrik another one connect
[12:04] jugg perfect
[12:04] sustrik :)
[12:04] jugg :)
[12:05] pieterh i'm making new documentation
[12:05] pieterh it'll be on the website as wiki text
[12:05] pieterh so anyone can join in and edit
[12:05] jugg I think I'm getting passed the point of needing the diagrams with this discussion, but newbies are a blank slate, bind,connect,socket have no meaning. Simple diagrams quickly get us passed the blank slate state. :)
[12:05] pieterh @jugg: ack
[12:05] pieterh my big advantage is that i'm a newbie :-)
[12:06] pieterh slates don't come much blanker than me
[12:06] sustrik i think sjampoo's comment about having two separate phases (establishing connections / sending messages) is important
[12:06] jugg :)
[12:06] sustrik if user doesn't get that he's lost
[12:06] pieterh @sustrik: normally these would be formalized in user-level APIs, no?
[12:07] sustrik connect/bind vs. send/recv
[12:07] pieterh topology being hidden
[12:07] jugg sustrik: your original clarification above: create endpoint, connect to endpoint was a big help as well.
[12:08] pieterh @jugg: we argued about this for a while, I insist that 0MQ *does* have connections, except they're not TCP connections but 0MQ connections
[12:08] jugg my original read of the documentation was that it was necessary to use two different functions depending if I was connecting locally or remotely.
[12:08] pieterh the local/remote explanation is misleading
[12:08] sustrik yes, that's confusing
[12:08] pieterh you can connect to a local socket
[12:09] pieterh it's about creating directed 0MQ connections (arrows)
[12:09] sustrik zmq_connect(3) and zmq_bind(3) should be fixed
[12:09] pieterh in my blank slate mind, anyhow
[12:09] pieterh @sustrik: it would be helpful IMO to stop saying that "0MQ does not have connections"
[12:10] pieterh it is freaky to have to "connect" and then think "there are no connections"
[12:10] jugg now, one more clarification point... even if I bind a socket, I can still send - ie, bind/recv and connect/send are not coupled, correct?
[12:11] jugg that being to pieterh's point
[12:11] sjampoo jugg, exactly
[12:11] sustrik yes
[12:11] pieterh you can always send but it won't go anywhere, right?
[12:11] jugg sjampoo: sorry, that was your point. :)
[12:11] sustrik it's either queued or dropped
[12:11] sustrik depending on the messaging pattern
[12:11] pieterh right
[12:12] jugg does bind/connect have to do more with queues?
[12:12] pieterh bind creates an endpoint and connect creates a 0MQ connection to that endpoint
[12:12] pieterh once the connection exists, messages may flow across it
[12:12] jugg yes, what is an endpoint?
[12:12] pieterh an endpoint is a socket that accepts connections :-)
[12:13] pieterh it is a connection endpoint
[12:13] pieterh not a messaging endpoint
[12:13] sustrik it can also be a device
[12:13] pieterh topology construction, ball & socket
[12:13] pieterh devices can have sockets and ... dare I say it... balls
[12:14] sjampoo balls?
[12:14] pieterh i am abusing "socket" now... :-(
[12:14] sjampoo heh
[12:15] sjampoo oh, as in a bone joint...
[12:15] jugg so, in p2p terms, either peer can accept connections (bind) or make a connection (connect), regardless of the direction the connection was made, both peers can send/recv messages over that connection.
[12:15] pieterh @jugg, last part depends on socket types
[12:15] pieterh but otherwise, yes
[12:16] jugg socket type p2p
[12:16] pieterh usually but not always the "client" connects to the "server"
[12:16] pieterh but you can usefully have cases where the "server" will connect to the "client", e.g. to get out from firewalls
[12:17] jugg yes, the peer accepting connections (bind) is the "server", the one making the connection (connect) is the "client"
[12:17] pieterh it's the common model
[12:17] jugg I don't think any of my last three comments are incorrect? (hopefully)
[12:17] pieterh (of course we can do lots of weird stuff but knowing that is not helpful to newbies)
[12:18] pieterh @jugg, no, but the last one was totally incorrect!
[12:18] pieterh :-)
[12:18] jugg poor me :)
[12:20] jugg ok, the socket type doesn't dictate the server/client role?
[12:21] jugg eg. I can set up a ZMQ_PUB socket, and it could be used in zmq_bind or zmq_connect, yes?
[12:21] pieterh @jugg, yes
[12:21] pieterh a publisher can connect to a subscriber
[12:21] pieterh but that's a weird usecase which I'd want to hide from newbies
[12:22] jugg and that goes for the rest of the socket types as well? socket type is bind/connect agnostic?
[12:22] pieterh yes
[12:22] jugg ok, things are falling into place. slowly... :)
[12:22] pieterh @sustrik: i'm likely to say stupid things, please correct me if I'm wrong...
[12:23] sustrik you are right, np
[12:23] pieterh nice!
[12:23] jugg blind leading the blind, how nice :)
[12:23] sustrik now you see the problems with docs i mentioned last week
[12:23] pieterh @jugg: i have one eye slightly open... can just about make out the crevasses
[12:23] pieterh @sustrik: yes, it's pretty clear we have a lot of work to do documenting the basics
[12:24] pieterh i'll have a first draft ready real soon
[12:26] jugg ok, best practice documentation, or tradeoff discussion on setting up the network topology - ie, deciding who should bind, who should connect.
[12:27] jugg eg, how can I assure that messages I'm sending out are not dropped into oblivion if the other side of the connection doesn't yet exist?
[12:28] sustrik yes
[12:28] pieterh @jugg, yes
[12:29] jugg yes, I can assure this?
[12:29] pieterh yes, documenting best practice
[12:29] pieterh as to how to assure it, it depends on the socket types
[12:30] pieterh best practice in some scenarios is "drop stuff you can't deliver"
[12:32] jugg sorry, slightly lost, this is what you are currently working on documenting, or it is already documented somewhere, if so, where? :)
[12:33] pieterh @jugg: i am working on that documentation now
[12:34] jugg cool
[12:34] pieterh i'll put it up as wiki text with space for comments and/or arbitrary contributions
[12:44] cremes jugg: regarding your question about assuring messages you send are not dropped because the other side doesn't exist yet...
[12:44] cremes here is how i solve that
[12:44] cremes if i am going to have a PUB socket deliver data, i don't allow it to start until there is at least one request
[12:44] cremes so i pair up my PUB socket with a REP socket that i bind to an endpoint
[12:45] cremes clients that want data can connect to my REP socket with a REQ socket and send me a request
[12:45] cremes upon receipt, i begin publishing
[12:46] cremes that "pattern" has been working well
[12:48] jugg what does "pair up" mean?
[12:49] jugg using an zmq concept, or simply as an internal meta relationship?
[12:49] cremes "pair up" means that my server uses both PUB and REP sockets to provide services; REP for receiving the first request, PUB to send the data
[12:49] cremes internal meta relationship
[12:49] jugg ok, that makes sense.
[12:50] cremes i use REQ/REP sockets to transition my servers through a state machine and other socket types for mass-data delivery
[12:51] cremes the semantics of each 0mq socket type make this easier to think about nwo that i have gotten past the bind/connect confusion :)
[12:54] jugg so any quick pointers on choosing bind/connect and socket types? Currently, I have several nodes, four of them are chained in a one way message passing n1->n2->n3->n4. Another node (n0) exists that receives requests from any other nodes and responds to those messages n1@->n0, n2@->n0, n3@->n0, n4@->n0 (the @ deliniating response to -> request). Also, n3 must subscribe to publications by n0 (which is the only publication from n0).
[12:55] jugg the REQ/REP seems obvious for the nX@->n0 interface
[12:55] jugg But I'm uncertain whether to use PUB/SUB or P2P for the chaining.
[12:56] jugg I think what cremes just described makes good sense for the n3 subscription to n0.
[12:57] jugg I'd probably go with bind/connect based on message direction for the chaining, the receiving binding, the sender connecting. But I have no basis for that other than simplicity of thinking.
[13:00] jugg hmm, n1-4 also send messages to n0 which don't require a response.
[13:00] sustrik cremes: connecting PUB socket to REP socket is not a good idea
[13:01] sustrik why not have REQ instead of PUB?
[13:01] jugg I don't think he is doing that?
[13:01] sustrik ah
[13:01] sustrik pair
[13:01] sustrik != connect
[13:02] sustrik sorry for confusion
[13:05] jugg ipc vs tcp loopback, any obvious reason to go one way or the other?
[13:05] sustrik ipc is faster on linux
[13:07] sustrik pieterh: here you go - how to run perf tests:
[13:07] pieterh thanks!
[13:08] jugg I think in the nX@->n0 interface, n0 will bind, nX will connect.
[13:09] jugg is there a performance difference in the chaining example if I use PUB/SUB vs P2P
[13:09] cremes jugg: right, n0 should probably bind and let the other nodes connect to it
[13:10] cremes jugg: p2p is kind of broken right now; use PUB
[13:10] sustrik right
[13:11] sustrik n1-n4 is a processing pipeline?
[13:11] jugg yes
[13:12] sustrik have a look at UPSTREAM/DOWNSTREAM sockets then
[13:12] sustrik that's what they are meant for
[13:12] sustrik summary:
[13:13] sustrik n1->n2->n3->n4: UPSTREAM/DOWNSTREAM
[13:13] sustrik nX@->n0: REQ/REP
[13:13] sustrik publications from n0: PUB/SUB
[13:16] jugg hmm, so each of n1-n4 could be load balanced. or if only one instance of each n1-n4 exists, it'll simply behave as a one way message passing?
[13:18] sustrik it's load-balancing among *one* service :)
[13:18] sustrik = simply pass the message to next node
[13:18] jugg yes, ok. :)
[13:18] sustrik however you can add more nodes in the future if needed
[13:19] jugg why is "component" emphasised in the docs, does it mean something special? It isn't just a node like any other?
[13:19] sustrik where's that?
[13:20] jugg " Parallelized pipeline pattern" for the socket type
[13:20] sustrik no idea
[13:20] sustrik probably a typo
[13:21] sustrik mato: any reason for that?
[13:21] jugg and is "messaging pattern" and "socket type" something different?
[13:21] sustrik yes, patter is say "request/reply"
[13:21] sustrik it consists of REQ and REP socket types
[13:21] jugg ah
[13:22] jugg that is obvious, I was confusing myself.
[13:24] jugg or perhaps the text: "The following messaging patterns are defined" could be changed to something like "The following messaging patterns are defined describing which socket types should be used for binding and connecting to an endpoint"
[13:24] jugg or not...
[13:25] jugg trying to processing too much info :)
[13:26] jugg sustrik: thanks for the summary above
[13:26] sustrik jugg: feel free to submit documentation patches
[13:28] jugg I assume the docs on the website correspond to the zeromq2/docs/*.txt (I've been reading the website)
[13:28] jugg and it looks like patches should be sent to the mailing list vs pull requests via github?
[13:29] sustrik via mailing list, stating it's submitted under MIT license
[13:29] sustrik pieter: the people who join the wiki cannot edit pages
[13:30] sustrik is that deliberate?
[13:30] sustrik pieterh:
[13:30] pieterh sustrik: ja?
[13:30] sustrik see above
[13:30] pieterh i see, hang on
[13:30] pieterh it's normal for some categories, not for others
[13:30] pieterh let me double check, what categories are you talking about?
[13:31] jugg well that answers my gpl3 tivo question
[13:31] sustrik
[13:32] pieterh @sustrik: pages in the default area need to be moved
[13:32] pieterh sorry, default catergory
[13:32] pieterh and the area: category is not open to all people to edit
[13:32] pieterh the wiki probably needs redesigning, we should discuss this
[13:33] pieterh @jugg: tivoization is a perfectly good business model for some people
[13:34] sustrik ???!!!
[13:34] sustrik i cannot edit nav:side :(
[13:34] pieterh sustrik: ? hang on
[13:35] jugg meh, zeromq licensing makes no sense.
[13:36] pieterh sustrik: if you can't edit a page it's probably because you are not logged in
[13:36] pieterh @jugg: what do you mean?
[13:37] pieterh @sustrik: you are site admin, always have been, and have full edit rights over everything
[13:38] sustrik oops
[13:38] sustrik it's ok
[13:40] jugg pieterh: well, ignoring the stupidity of gpl3 anti-tivo, and looking only at my contributions must be MIT licensed (purely for the benefit of imatix), I myself and others can only use contributions under lgpl3. Basically, this means that contributors are working for free for imatix. Normally open source projects, contributs work for free for the project. This is not the case here.
[13:40] jugg sorry for grammar/spelling, I'm getting tired... :)
[13:41] pieterh @jugg, fair points
[13:41] pieterh however imatix works for free for you too
[13:41] pieterh so it's not an unfair deal to ask that we can try to make a viable business
[13:41] pieterh you don't need to give us patches
[13:42] pieterh however if you want us to maintain your patches in the official product, we ask for MIT licensing OR copyright transfer
[13:43] pieterh you can always just redistribute your modified libraries under the LGPL, period
[13:43] pieterh as for the GPL3's "stupidity"... we disagree
[13:43] pieterh i've been using the GPL since... 1991 or so for my work
[13:44] pieterh and the principle of enforcing "share-alike" is important IMO
[13:45] pieterh what do you see as the problem with this model?
[13:46] sustrik anti-tivo caluse in gpl3 is not about share-alike, it's about free (as is speech) hardware
[13:46] pieterh why do you say so?
[13:47] pieterh it is about preventing people from not sharing
[13:47] pieterh it is about enforcing the spirit of the GPL: you use my work, you share improvements
[13:47] sustrik you can redistribute the source and still lock you hw device no?
[13:47] pieterh Tivo hacked the GPL to find a way around that
[13:47] sustrik i am not an expert anyway
[13:48] pieterh The GPL does not try to force anything on HW producers
[13:48] pieterh it is purely a license for using someone's work
[13:48] sustrik ok, ianal
[13:49] pieterh ianal either but have been working with licenses for close on two decades
[13:49] pieterh the dual-license option is actually win-win
[13:50] pieterh commercial firms can still build no-share products but they pay for the right to escape, so they are not cheating
[13:50] pieterh and contributors are assured that their work is not closed off
[13:51] pieterh @jugg: are you still around?
[13:51] jugg pieterh: gpl3 tivo is purely about enforcing open hardware. It has nothing to do with software. ESR himself states that he is scared that hardware vendors will implement restrictions that only allow running of signed software, preventing open source software from running on that hardware. His aim is to ensure the hardware paltform remains open. The GPL2 was just fine in ensure the opening of software.
[13:52] pieterh ESR has nothing to do with GPL... however let me do some more research
[13:52] pieterh are you asking that we move to GPL2?
[13:53] pieterh that is not going to happen, I can say it categorically
[13:53] pieterh :-)
[13:53] jugg appologizes certainly isn't ESR :)
[13:54] pieterh maybe you mean Eben Moglen ?
[13:54] jugg anyway, ignoring my slipup of ESR vs Richard Stallman.
[13:55] jugg
[13:57] pieterh so to review the tivo issue
[13:57] pieterh first, tivo began using modified GPL software and not providing the source code
[13:57] pieterh e.g. using a modified 0MQ and not providing patches
[13:58] pieterh then they accepted to provide their modifications
[13:58] jugg to your earlier points, I do not begrudge imatix from making a viable business. Certainly from imatix's stand point, requesting MIT or Copyright transfer makes perfect sense.
[13:58] pieterh but they introduced hardware checking on their boxes so you could not remix those modifications
[13:58] pieterh so you'd get e.g. a 0MQ with a driver for some network card
[13:58] pieterh but you couldn't improve that driver because the Tivo borked
[13:58] jugg pieterh: them not providing source code isn't fixed by gpl3.
[13:59] pieterh no, but it prevents them using the GPL3 code at all
[13:59] pieterh OR they must not prevent remixing (on their boxes)
[13:59] pieterh i concede...
[13:59] pieterh that it's about stopping people running arbitrary code on a box that they think they own but actually don't
[13:59] jugg them locking the hw to only run their signed version of the code is their perogative imo. As long as they provide the modified source, I don't care of that source can't be run on their hardware.
[13:59] pieterh you don't care
[14:00] pieterh i do :-)
[14:00] pieterh now, why i care is not ideological
[14:00] jugg why? Go build your own hardware that matches theirs and run the modified code on it.
[14:00] pieterh nah, i sell software :-)
[14:00] pieterh if a firm is bothered enough (and has money enough) to lock their hardware
[14:00] pieterh then they can buy commercial licenses
[14:00] pieterh and thus fund 0MQ development
[14:01] pieterh i see GPL3 as an ultracapitalist tool
[14:01] jugg how does them preventing modified versions of the software from running on their hardware hurt you?
[14:01] pieterh it's not really the point
[14:01] jugg ?!
[14:01] pieterh the point is, we're a business in the end
[14:02] pieterh we need to find ways of getting money from people who use 0MQ for *profit*
[14:02] pieterh making software like 0MQ is expensive, though obviously a community like ours makes a huge difference
[14:03] pieterh it lets us leverage our investment
[14:03] jugg so, tivoication is simply an indicator for you of the depth of the hw companies pocket?
[14:03] pieterh i think that's accurate, yes
[14:03] pieterh sorry to be blunt :-)
[14:03] pieterh i've been making free software since 1991 like i said
[14:03] jugg blunt is clear. :)
[14:03] pieterh and i love the ideological aspects
[14:04] pieterh but they do seem to reflect economic realities that also work
[14:04] pieterh forcing those with deep pockets to put money on the table is win-win
[14:05] pieterh i do really appreciate the work that the FSF (Eben Moglen, mainly) put into GPL3
[14:05] jugg and at the same time kill the companies without deep pockets who's only way to make money is by using tivoication to drive service and support contracts.
[14:05] pieterh kill is a harsh word
[14:06] pieterh how can our license terms kill a firm?
[14:06] pieterh they can just write their own software
[14:06] pieterh or buy commercial software
[14:06] jugg which would kill them :)
[14:06] pieterh or buy a commercial license from us
[14:06] pieterh sorry, but i pay people
[14:06] pieterh and when they send me invoices i don't say, "you're killing me!"
[14:06] pieterh that... would be... silly
[14:06] jugg :)
[14:06] pieterh indeed
[14:07] sustrik sorry to interrupt you, i'm just interested -- what's the point of tivoising a hw device?
[14:07] sustrik i mean business point
[14:07] pieterh it's a strange world when simply proposing to charge a modest amount for a good product translates into accusations of premeditated murder... :-)
[14:07] pieterh sustrik: it means you can cripple the hardware
[14:07] sustrik i mean business point
[14:07] pieterh run software on it that locks down features
[14:07] pieterh unlock features for money
[14:07] jugg sustrik: it means that you can lease or sell hw at a low cost in exchange for ongoing service contracts.
[14:08] sustrik a-ha
[14:08] sustrik understood
[14:08] sustrik thanks
[14:08] pieterh e.g. you buy a mobile phone that has VoIP locked/disabled
[14:08] pieterh if i can get the source code, i can unlock it, run a patched version
[14:08] pieterh the Tivo solution is that the hw won't run that modified / improved version
[14:08] jugg it means a sofware company can survive, by providing a hw platform that enables them to make money on their software.
[14:09] pieterh jugg: do you work for such a company? just curious
[14:09] pieterh I could argue: it means a firm can take work done by others and not contribute anything back
[14:09] pieterh it is specifically about taking free software, don't forget
[14:09] jugg pieterh: not currently.
[14:09] jugg pieterh: absolutely not true, they must contribute back the source they've modified.
[14:10] pieterh true
[14:10] jugg Just because they provided a hardware platform doesn't mean because you wrote the software that you have the right to that hw paltform.
[14:10] pieterh i misstated myself, sorry
[14:10] pieterh well, put this another way
[14:10] pieterh a commercial license can be for 1 euro
[14:10] pieterh it really depends
[14:10] jugg but that is what the gpl3 wants to state. it wants to state that someone who wrote software must have control over the hardware that software is ran on. rediculous.
[14:11] pieterh this is only one part of the GPL3
[14:11] jugg you write the software, you get control over the software.
[14:11] jugg pieterh: I agree, which is why a gpl2.5 is needed.
[14:11] pieterh actually, you get the software (as a user) in a box, you get the right to hack that box
[14:11] pieterh the point is the sharing of knowledge enforced by private contract
[14:12] pieterh when we invest in 0MQ we tell box makers:
[14:12] pieterh - if you use OUR software (and we speak in name of our contributors too) on your box
[14:12] jugg pieterh: absolutely not. the box is leased, you have no right to it. Or the box is sold at a very low cost with the expectation that ongoing service is provided for that box.
[14:12] pieterh - then you MUST allow people to improve that software AND run the improvements on your box
[14:12] pieterh simple
[14:12] pieterh - OR you pay us something and we'll make an exception for you
[14:12] pieterh seems simple enough
[14:13] pieterh and, honestly, it seems totally fair
[14:13] pieterh look, @jugg, if/when you work for a firm that finds itself in this situation, let me know
[14:14] pieterh and we'll bend over backwards to get as close to that nominal 1 Euro as possible
[14:14] pieterh :-)
[14:14] pieterh or perhaps make a specific exception to the Tivo parts for that firm
[14:15] jugg :) Like I said, I don't begrudge imatix anything, but the idealogical concept that someone can write software and believe by the use of that software they have the ability to assert some right and control over the hardware is asinine. (to be blunt)
[14:16] pieterh @jugg: there is clearly a conflict of interests
[14:16] jugg anyway... sorry for dragging this channel through all of that. :)
[14:16] pieterh recall that the tivo clauses came from real conflicts, not some theoretical debate
[14:17] pieterh and in the end, if a developer wants to license his work under a license that says, "you must wear pink while singing 'When the Saints'", that's their choice
[14:17] pieterh private contract
[14:18] pieterh there is no post-facto modification, no ambush, no traps
[14:18] jugg no, as you've stated yourself, it came from the ability to reach into another companies pockets. The gpl2 was complied with in the end, source was provided. Nothing further was necessary imo. (although I had no idea zmq was part of this, so I surely stepped on the hornets nest!)
[14:18] pieterh @jugg: that is my view on it, it's not what drove the FSF's changes
[14:19] sustrik what about turning it upsde down: idealogical concept that someone can make hadware and believe by the use of that hardware they have the ability to assert some right and control over the software is asinine :)
[14:19] pieterh the GPL3 was in part about patents, in part about people hacking the word "distribute" and in part about issues like this
[14:19] pieterh again
[14:19] pieterh someone produces a work and licenses their work
[14:19] pieterh other parties can accept / reject these licenses
[14:20] pieterh there is no "asinine" just "I accept" or "I do not accept"
[14:24] jugg my only issue with gpl3 is the tivo part. The rest, patents and the like, trully deal with the software freedom. The touted aspect of the GPL is about software freedom not about a tool for making money. So when a company uses the GPL as a tool for making money, I can't meet that argument on equal ground. Take the GPL terms and relabel it as your own license - it has the same affect without confusion as to why it is being use
[14:24] jugg d. By using the "GPL" name, then you are stating that your interested in software freedom and open source.
[14:24] pieterh @jugg: it is not practical to define custom licenses
[14:25] pieterh except on a case by case basis
[14:25] pieterh if we tried to take the GPL3 and customize it
[14:25] jugg I never said customize, I said rename.
[14:25] pieterh then we would force every single serious firm to do a full legal review of our license
[14:25] jugg actually I said "relabel" :)
[14:25] pieterh ?
[14:25] pieterh what are you actually proposing? I've lost the plot now
[14:26] pieterh freedom is not about NOT making money
[14:26] pieterh freedom is precisely about a more effective market in knowledge
[14:36] jugg I think I need to wrap this up on my end, but to try to clarify the lost plot... the gpl is about software freedom, not about a tool to make money, in fact quite the opposite. So, if a company uses the name "GPL" it is stating that the GPL licensed software is about freedom of the software. If that isn't the case, and the GPL *terms* are being used as a way to ensure alertnate licensing deals, then that company (imo) should re
[14:36] jugg package the GPL terms under a different name.
[14:37] jugg That said I'm trying to make money from software as well, so I'm not in ideological alignment with the purpose of the GPL myself in all instances. However, when I used the GPL as I often do, it is about the software only.
[14:37] pieterh this would create an enormous amount of extra work for many people
[14:37] jugg what extra work?
[14:37] pieterh today, any firm wanting to use 0MQ in their apps must do a legal review of the license terms
[14:37] pieterh it is a contract, don't forget
[14:37] pieterh the LGPL is a standard contract
[14:38] pieterh reviewed once, used many times
[14:38] pieterh if we make a custom license it must be reviewed specifically
[14:38] pieterh that is called "legal clearance" and it is expensive
[14:38] pieterh lawyer hours and days and weeks
[14:38] pieterh sufficiently expensive that many firms will simply refuse to use "open source" that does not have a standard licence
[14:39] pieterh we would be shooting ourselves in the foot if we did what you propose
[14:39] pieterh suggestion registered... and rejected... sorry! :-)
[14:40] jugg Funny, too bad GNU didn't think of a way to scope out the pockets of firms trying to use the GPL to make money and figure out a way to get money out of those pockets... :)
[14:42] pieterh @jugg: well, the FSF is a charity
[14:42] pieterh not a business
[14:42] pieterh there have been some very large businesses built around the GPL
[14:43] jugg there I go confusing my terms again, I really should stop already. :)
[14:44] pieterh think of... Red Hat, or Cygnus Solutions
[14:44] pieterh iMatix is actually a corporate sponsor of the FSF
[14:53] jugg please don't take any of this as argument directly against imatix. My main argument is against the GPL3 tivoization clause. The fact that imatix uses it as a way to force commercial licensing deals is compeltely imatix's prerogative. But at an idealogical level, I associate the name GPL with free software, not as a tool to make money. As such, at an idealogical level, I'd like companies not to use the name GPL for licensi
[14:53] jugg ng software in this way as it creates confusion.
[14:54] jugg obviously, that ideology isn't going to change imatix's stance... :)
[14:54] pieterh @jugg, the argument is a useful one and I certainly don't take it personally
[14:55] pieterh iMatix has been using free software licenses since 1996
[14:55] pieterh we used to write our own (our old SFL license is one of those recognized by the FSF)
[14:55] pieterh and we've tried many variations...
[14:56] pieterh GPL is the one that works, and I've spent enough time with the FSF and thinking about the rationale to understand why
[14:56] pieterh it is not money vs. free, that is a false dichotomy
[14:56] pieterh it is about creating the most efficient knowledge economy in which all contributors get the most out of their work
[14:57] pieterh all parties have the choice to stay out of that economy
[14:57] pieterh but if they participate, they follow the rules
[14:58] pieterh there is no ideology, just more or less efficient contracts
[14:59] pieterh s/efficient/successful/
[14:59] pieterh sorry to be soapboxing, I think we're boring the channel now :-)
[14:59] pieterh hey guise, I want to rename all the socket types!
[15:00] guido_g hehehe
[15:00] pieterh ZMQ_DONT_TRY_THIS
[15:00] pieterh ZMQ_OMG_ITS_AMAZING
[15:00] pieterh and ZMQ_MAGIC_HAPPENS
[15:00] sjampoo Hmmm, what would ZMQ_MAGIC_HAPPENS do?
[15:00] guido_g and of course: ZMQ_IT_MIGHT_EVEN_WORK
[15:00] pieterh "I tried the ZMQ_DONT_TRY_THIS socket and my app crashed?"
[15:01] pieterh sjampoo: that depends on the time of day
[15:01] jugg certainly, but I think the change of rules between gpl2 and gpl3 is significant, and it leaves a lot of us who were part of the gpl2 economy, who helped build that enconmy, in an ackward place. Anyway, I agree, time to move on. Thanks for the discussion, I appreciate knowing where zeromq stands in this regards.
[15:01] sjampoo excellent, i was just looking for something like that
[15:02] pieterh @sjampoo: if that doesn't work, try guido_g's new ZMQ_IT_MIGHT_EVEN_WORK
[15:02] sjampoo pieterh, thanks for the tip!
[15:02] pieterh np... hang on, i need to update the FAQ
[15:03] pieterh create a special release of 0MQ where all the names are... weird
[15:03] pieterh somewhere, someone would use it and not get the joke, and would, like months later come back
[15:04] pieterh "I'm using ZMQ_NULL_WRITES_ENABLED and losing messages, is that normal?"
[15:18] jugg in zmq_socket messaging patterns request-reply description, the terms "client" and "service" is used, I assume the defines a node that uses zmq_connect or zmq_bind respectively?
[15:19] sustrik jugg: client is REQ socket, service is REP socket
[15:24] jugg ok, but "A socket of type ZMQ_REQ is used by a client" would seem to say, a client already exists and a client would use a socket type of ZMQ_REQ, which if just reading the documentation makes me ask, what is a client? (hence, so I did)
[15:25] sustrik ok, more precisely: client is an application that uses REQ socket
[15:25] jugg Am I abusing the term node?
[15:25] jugg eh, I didn't use "node" this time. ignore me.
[15:26] sustrik :)
[15:26] sustrik anyway, the problem with this definition of client
[15:26] sustrik is that if a single app uses both REQ and REP sockets
[15:26] sustrik it's both client and server at once
[15:27] guido_g you're in the clients role when you request a service, i'd say
[15:27] sustrik documentation should probably refrain on defining categories of applications
[15:27] sustrik application is just application
[15:27] sustrik it may have some sockets open
[15:28] jugg "Each request sent is load-balanced among all connected services." facinating, so for that last to be true, the "services" would be connecting to the client's bound endpoint.
[15:28] sustrik or you can connect multiple times
[15:28] sustrik 0mq allows multiple connects
[15:28] sustrik as well as multiple binds
[15:29] sustrik guido_g: yes, but that kind of distinction is good for tutorial, not the formal definition
[15:30] guido_g huh?
[15:30] guido_g why so?
[15:30] sustrik because application can be both client and service
[15:30] guido_g this is why i said "role"
[15:31] sustrik ah, got it
[15:31] guido_g one application can take on multiplke roles
[15:31] sustrik yes
[15:31] guido_g some apps even at the same time (threading on multicore)
[15:33] sustrik multiple roles for same app are pretty common, say an application is both _publisher_ of stock quotes and trade execution _service_
[15:33] guido_g exactly
[15:56] jugg rc = zmq_bind(s, "ipc:///tmp/a/0"); fails, but rc = zmq_bind(s, "ipc:///tmp/0"); succeeds. Is it expected that only the directory part must exist?
[15:57] jugg s/only //
[15:57] jugg "/tmp/" exists on my system, "/tmp/a/" does not
[15:58] sustrik it translates directly to UNIX domain sockets
[15:58] sustrik have a look what do they allow and what they don't
[16:12] jugg And I'm off for now. Thanks for all of the help/discussion. :)
[17:11] zedas sustrik: hey, there may be a memory leak in the I/O thread: that's valgrind output from a test server
[17:12] zedas looks like zmq::pipe_t is making ram and then it's not getting freed properly. not sure exactly where though so thought you might have insight.
[17:17] zedas sustrik: could it be because the ~pipe_t hasn't run?
[18:51] cremes are functions zmq_sleep, zmq_stopwatch_start and zmq_stopwatch_stop part of the official zmq api? should non-C language bindings expose them?
[18:56] mato cremes: nope, they're not
[18:56] cremes mato: ok
[18:57] mato cremes: they're in there because of the perf tests, and should probably be moved somewhere
[18:57] sjampoo Yah, they are handy for perf tests. PyZMQ removed stopwatch at first, but then the performance results wheren't really comparable so they have been moved back in.
[18:58] cremes sounds reasonable
[18:58] mato sustrik: good discussion about connect/bind and endpoints, i will rewrite the relevant portions of the reference
[18:58] mato sjampoo: don't rely on them, they might go away...
[18:59] mato we don't want people to the the wrong idea that we're building a portability library
[18:59] sjampoo aren't you guys there already ;)
[19:00] sjampoo What are the future plans? I think i saw some flirting with moving into the kernel.
[19:01] mato that's part of it, but a long way away
[19:01] mato sustrik is the expert on the grand vision
[19:01] mato deploying 0mq netwide
[19:02] mato take a look here:
[19:02] sjampoo ah thanks!
[20:05] sustrik zedas: thanks for reporting it
[20:05] sustrik i'll have a look at it tomorrow (late evening here)
[20:07] sustrik sjampoo: basically, the idea is to allow for distributed networks of nodes with no centralised control
[20:07] sustrik think of a marketplace for computational services
[20:08] sustrik a world-wide cloud
[20:08] sustrik but that's pretty far ahead
[20:14] zedas sustrik: if i can figure it out i'll shoot you a patch
[20:14] zedas otherwise you know better than me probably
[20:14] sustrik zedas: thanks
[21:07] sustrik sjampoo: let me give an example
[21:08] sustrik zed have created a database accessible via 0mq
[21:08] sustrik he may have used it just as a convenient way to provide network access
[21:08] sustrik but:
[21:09] sustrik think of three companies providing storage space using mulletdb
[21:09] sustrik you can obviously connect to one of them
[21:09] sustrik but you can put a device in a middle and let it load balance your requests among the three
[21:09] sustrik once we implement priorities
[21:10] sustrik you can even say "store the data primarily at A, but it it's not available, use B"
[21:10] sustrik etc.
[21:21] mikko would be cool to create such a system
[21:21] mikko and use apache zookeeper for syncing