IRC Log

Thursday August 12, 2010

[Time] NameMessage
[06:16] CIA-20 zeromq2: 03Martin Sustrik 07wip-shutdown * radb8ecb 10/ (7 files): dezombification procedure fixed - http://bit.ly/bgEdXg
[09:43] mato pieterh: Git tip for you: You manually reverted the zmq_deviced changes when you could have just done "git revert <commit>"
[09:44] mato pieterh: Which basically applies the changes from <commit> in reverse and writes the log message (Reverted "<title>" commit <commit> or similar) for you.
[09:51] pieterh mato: yes, you're right
[09:51] pieterh worth documenting as well
[09:51] pieterh i knew it but forgot
[09:52] sustrik pieterh: would you be willing to help with the linux kongress article
[09:52] sustrik ?
[09:52] pieterh sustrik: i already sent you that several days ago
[09:52] sustrik yep, that's the first part
[09:52] pieterh ? that's already 3x too long... :-)
[09:52] sustrik i have to write next one till 15th
[09:52] sustrik i'll shave the excess load off
[09:52] pieterh you have multiple articles to present?
[09:53] sustrik nope, it's just one, but i want it to consist of 2 parts
[09:53] sustrik 1. introduction to 0mq
[09:53] pieterh ah, you didn't explain this
[09:53] mato sustrik: you could try asking them nicely if it's at all possible to deliver it later...
[09:53] mato sustrik: worth a try, no?
[09:53] sustrik maybe
[09:53] sustrik still have 3 days...
[09:53] pieterh sustrik: I don't mind helping if you tell me what you want
[09:53] pieterh :-)
[09:54] sustrik pieter: i'll send you a concept via email
[09:54] sustrik wait a sec
[09:55] pieterh sustrik: ok, and the first part is at http://www.zeromq.org/docs:introduction
[09:55] pieterh there are some fixes done in parallel with fixes to the Guide
[09:56] sustrik ok, i'll do the editing part
[10:02] sustrik gh, gh, i am trying to formulate what i want to say...
[10:02] sustrik irc would be better than email
[10:02] sustrik after showing how 0mq works (based on simple examples) i would like to convey the large picture
[10:03] sustrik that is, in short, inteligent endpoints and dumb intermediaries
[10:03] sustrik standard internet end-to-end principle
[10:03] sustrik it should be explained on example
[10:04] sustrik there are 3 examples i can think of
[10:04] pieterh who is 'gh'? :-)
[10:04] sustrik it's expression of desperation
[10:04] sustrik :)
[10:05] sustrik 1. i can start with moglen's quote about ditributed servies, move to how diaspora should work
[10:05] sustrik and show that 0mq is a generic infrastructure for that kind interconnect
[10:05] pieterh would you take a second to read the new introduction text i wrote for the guide?
[10:05] pieterh it might be part of what you want to say already
[10:05] sustrik yep, i read it
[10:05] sustrik it's ok but it's too abstract
[10:06] pieterh its the problem spec
[10:06] sustrik i need an example, that's the best way to convey the idea
[10:06] pieterh just one question first
[10:06] mato sustrik: what about that narrative you wrote?
[10:06] sustrik it's good but clumsy
[10:06] mato sustrik: so, polish it
[10:06] sustrik that was my option 2
[10:06] mato sustrik: it's already there, a good starting point
[10:07] pieterh sustrik: do you think the user guide should try to explain the big picture up front or not?
[10:07] pieterh do people care?
[10:07] sustrik not really
[10:07] sustrik maybe a different article
[10:07] pieterh maybe
[10:07] pieterh so why do you want to explain this to your audience?
[10:07] mato pieterh: yeah, this was my main feeling from the user guide at the moment... it's good, but quite longwinded and i was missing more of a "practical" feel
[10:07] sustrik because i want involve kernel developers
[10:08] sustrik those are not interested what 0mq offers from user's point of view
[10:08] mato pieterh: but to be honest, i've not really read it, this is just an off-the-cuff feeling
[10:08] sustrik they cannot use it anyway
[10:08] pieterh mato: sure, it's to read, not use as reference...
[10:08] mato pieterh: yeah, but even then the assumption is the person is reading it with the intent of doing something practical
[10:08] sustrik what they need to understand to get interested is exactly the big picture
[10:08] mato pieterh: that's all...
[10:08] pieterh sustrik: ok, you need to establish who your audience is and what you want them to take away with them
[10:09] sustrik audience: kernel developers
[10:09] sustrik message: this stuff should be in kernel
[10:09] pieterh mato: ... it swings around, if i start with code people want more exposition, and if there's exposition, people ask for more practicality
[10:09] pieterh sustrik: ok that's clear
[10:09] pieterh then i'd suggest starting with eben moglen's speech
[10:10] pieterh take it from there
[10:10] mato pieterh: a format i've found useful is that of e.g. the rails books, and indeed many of the "practical" books that exist today...
[10:10] sustrik yes, i had the same idea
[10:10] pieterh linux kernel developers are well inside the GPL camp
[10:10] mato pieterh: which is, you build a practical system, or more than one, from simpler to more complex
[10:10] mato pieterh: as you go along, you then explain the hard bits, but not upfront
[10:10] mato pieterh: just a suggestion...
[10:10] pieterh mato: yes, i know
[10:11] pieterh i agree too
[10:11] pieterh this was in fact the plan
[10:11] mato based from experience i get the most out of books like that when the subject matter is complex
[10:11] pieterh in between "basic concepts" and the rest should come a worked example
[10:11] pieterh i'm not sure how much of the basic concepts remains, then
[10:12] pieterh it does not matter, the text is highly fungible
[10:12] pieterh this is really phase 1, capture
[10:12] pieterh sustrik: i'm not sure the intro i wrote for you will work, then
[10:13] sustrik it edit it
[10:13] sustrik but it should be there
[10:13] sustrik because otherwise people would be simply confused
[10:13] pieterh ok, what i suggest is you hack that text as you like, put bullets where you want more detail, etc.
[10:13] sustrik it shows "what the thing is"
[10:13] pieterh then pass it to me and i'll continue the work
[10:13] sustrik ok
[10:13] pieterh i'm playing with gist.github.com
[10:14] sustrik so let me explain what the part 2 should say
[10:14] pieterh shoot
[10:14] sustrik starting with the moglen's quote looks good
[10:14] sustrik that brings us to diaspora as a comprehensible example of end-to-end messaging
[10:14] pieterh i like "diaspora"
[10:14] sustrik it's easy to grasp because everyone knws how facebook works
[10:15] pieterh facebook is diaspora? or not diaspora?
[10:15] sustrik and can understand the centralised=>distributed transition
[10:15] sustrik it is not
[10:15] pieterh aha
[10:15] pieterh sustrik: we need to invent a killer app for 0MQ
[10:15] sustrik but it's an generally comprehensible example to show
[10:15] pieterh call it diaspora
[10:15] sustrik how centralised messaging applications can be decetralised
[10:16] sustrik keep in mind i have just few minutes to present it
[10:16] sustrik so the example has to be extremely easy to grasp
[10:16] sustrik mato: that's the problem with my article
[10:16] sustrik it's message is not that obvious
[10:16] sustrik as to grasp it in 10 mins
[10:16] pieterh you won't present the article as such, you'll talk and the article gets published, right?
[10:16] sustrik yes
[10:17] pieterh so would you start by ignoring all existing text
[10:17] sustrik but there should be some correleation i suppose
[10:17] pieterh and write down your speech in bullet points
[10:17] pieterh standard rule for short speech:
[10:17] pieterh - explain what you are going to say
[10:17] pieterh - say it, in three parts
[10:17] pieterh - explain what you just said
[10:18] jsimmons 0mq in the kernel ey
[10:18] sustrik the whole speach is 45 mins i think
[10:18] sustrik so it's not that short
[10:18] pieterh 10 or 45? /me is confused now
[10:18] sustrik the problem is that to present the kernel stuff
[10:18] sustrik i have to introduce 0mq first
[10:18] jsimmons what would the advantage of having 0mq in the kernel as opposed to an external library?
[10:18] sustrik which will take say 15 mins
[10:19] sustrik jsimmons: i'll explain in part 3 :)
[10:19] pieterh sustrik: i can't follow any longer
[10:19] sustrik then i need to explain why the stuff is *importatnt*
[10:19] sustrik say another 15 mins
[10:19] sustrik then say 5 mins as for "why it should be implemented at kernel level"
[10:19] sustrik + question
[10:19] sustrik questions
[10:19] sustrik pieterh: summary:
[10:20] pieterh sustrik: irc is not a database
[10:20] sustrik 1. example of 0mq, how it works
[10:20] sustrik 2. why is it important
[10:20] sustrik 3. why it should be in kernal
[10:20] sustrik 4. Q & A
[10:20] sustrik :)
[10:20] pieterh so you have your three points
[10:21] sustrik 1 is done
[10:21] sustrik 3 i can write myself
[10:21] sustrik 2 ???
[10:21] pieterh profit, obviously
[10:21] sustrik it needs some kind of comprehensible narrative
[10:21] mato sustrik: 2) is your narrative, polished with a good "cloud" example
[10:21] pieterh why is it important?
[10:21] sustrik based on example
[10:21] pieterh why is 0mq important?
[10:22] mato sustrik: since you want to start with eben moglens quote
[10:22] mato sustrik: it would seem that a narrative based around a cloud example is good
[10:22] mato pieterh: incidentally, did you read the narrative sustrik wrote?
[10:22] sustrik whatever
[10:22] pieterh sustrik: i'm asking for an ideology of sorts
[10:22] sustrik the point is that there's only ~15 mins to present it
[10:22] sustrik so it has to be dead simple
[10:22] jsimmons 1) what is the current situation 2) what is the omq solution 3) why it needs to be included at kernel level. It being important should be blatantly obvious from your excellent example.
[10:23] pieterh mato: ??? lookup function failed
[10:23] mato pieterh: 1st paper on www.250bpm.com
[10:24] sustrik http://www.250bpm.com/joe
[10:24] sustrik but it's too complex imo
[10:24] mato sustrik: nope, assume kernel developers are smart
[10:24] pieterh sustrik: obviously i think 0MQ is important but can we assume people share our assumptions?
[10:24] sustrik that's why i wanted to give a real example
[10:24] sustrik like diaspora
[10:25] sustrik it's about freedom, the other example (my paper) is about business
[10:25] pieterh ok, freedom is a good hook
[10:25] sustrik but i have no example that's both about business and freedom and can be explained in 15 mins
[10:25] mato sustrik: decentralized facebook?
[10:25] pieterh yes, this is possible
[10:26] sustrik easy to grasp
[10:26] mato sustrik: remember, everyone knows what facebook does
[10:26] pieterh mato: "kernel developers are smart"
[10:26] pieterh you don't need facebook as a reference
[10:26] sustrik you can even mention "groups"
[10:26] jsimmons yeah but diaspora is a large complicated example
[10:26] sustrik as an example of a device
[10:26] sustrik jsimmons: any better idea?
[10:26] pieterh freedom is about communication
[10:26] pieterh and the cost of communication
[10:26] pieterh centralization is anti-freedom
[10:27] mato pieterh: ja, but using facebook as a reference makes it completely obvious
[10:27] mato pieterh: no harm in that...
[10:27] pieterh sure but what's the alternative to centralized facebook?
[10:27] jsimmons yeah but your software should stand on its own. I think brining politics into it is a mistake.
[10:27] jsimmons and a large complicated non-materialised example, too. on the diaspora thing.
[10:27] mato jsimmons: good point.
[10:28] sustrik yes, but what's the alternative
[10:28] jsimmons You need something concrete where people can see one thing that ain't zeromq and one thing that is and go wow that's so much better
[10:28] pieterh yes
[10:28] pieterh you can take it one step more abstract
[10:28] pieterh one centralized vs. one distributed
[10:28] sustrik yes
[10:28] pieterh but no more than that
[10:28] sustrik yes
[10:28] pieterh so what's the most successful distributed system?
[10:29] sustrik IP
[10:29] sustrik web
[10:29] pieterh that every linux kernel developer knows and loves?
[10:29] pieterh :-)
[10:29] mato IP actually
[10:29] pieterh git
[10:29] mato yeah, git :)
[10:29] jsimmons git, I thought that too pieterh.
[10:29] pieterh git
[10:29] sustrik the question is how to base a narrative on IP
[10:29] pieterh now imagine you could turn every centralized app into a git
[10:29] pieterh all it requires is freedom to communicate
[10:30] sustrik i have an idea
[10:30] sustrik think of dystopia instead of utopia:
[10:30] mato one important point about the freedom to communicate bit - this is what the internet was originally designed to do
[10:30] sustrik what if web was centralised?
[10:30] pieterh mato: absolutely
[10:30] sustrik each link has to be resolved by a centeralised entity
[10:30] mato now, 0mq enables you to use the net for what it was designed for, for certain problems
[10:30] pieterh sustrik: i don't think you need to explain why centralization is bad
[10:30] pieterh people get that
[10:30] mato something along those lines anyway
[10:31] pieterh especially your audience
[10:31] pieterh problem is that the cost of distribution is still tooooooo high
[10:31] pieterh no-one doubts eben's message
[10:31] pieterh but it's almost impossible to see how to make it
[10:31] sustrik right
[10:31] jsimmons I think you're talking too abstractly. If they're kernel developers they're developers, give them something that works.
[10:31] sustrik so the message is "this is IT"
[10:32] pieterh yes
[10:32] pieterh this is IT
[10:32] pieterh did you read my Fixing the World intro?
[10:32] sustrik yes
[10:32] sustrik it misses example
[10:32] pieterh yes
[10:32] pieterh eben at the start, example after, and you have part two
[10:32] sustrik exactly
[10:32] pieterh though I'd move that to part 1
[10:33] pieterh part 2 is then how 0mq works
[10:33] pieterh part 3 is then why it must be in the kernel
[10:33] pieterh /works/ is used/
[10:33] sustrik pt3 is easy
[10:33] pieterh yes
[10:33] sustrik i wanted to have intro to 0mq first so that people can make more sense of the example in part2
[10:34] pieterh problem spec has to come before solution
[10:34] sustrik what i wanted to show was"
[10:34] sustrik 1. You can do network programming easy with 0mq
[10:34] sustrik (no big ideas there, just "how to write client sercer in 10 mins")
[10:35] pieterh 0. Your freedom depends on easy network programming
[10:35] pieterh 2. Your kernel is the best way to get this to the whole world
[10:35] sustrik 2. Here's how it grows to ludicrous scales
[10:35] pieterh you can point out that in 5 years' time Linux will be the only kernel
[10:35] sustrik :)
[10:36] pieterh this is the structure i'd use, to do evangelizing
[10:36] pieterh 0. dystopia, unless...
[10:36] sustrik sure, why not
[10:36] pieterh 1. perfect solution, look
[10:37] pieterh 2. we need you to make this happen
[10:37] sustrik yes, can be
[10:37] sustrik still, what should be the example
[10:37] sustrik i still like the social networking thing
[10:37] pieterh you can simply list centralized examples
[10:37] pieterh and distributed examples
[10:37] pieterh these are smart people
[10:37] sustrik i can explicitly NOT mention facebook and diaspora
[10:37] sustrik if we want ot keep it neutral
[10:38] pieterh centralized development practices vs. linux kernel process
[10:38] pieterh centralized mobile phone networks vs. VoIP
[10:38] sustrik sure, the question is how that maps to 0mq
[10:38] pieterh doesn't matter
[10:38] sustrik just listing examples won't do
[10:38] jsimmons just drop the speech, put your desktop on the big screen and implement a massively scalable example x in the few minutes you have, then have everybody in the room grab it and make a ad-hoc distributed network that does something cool. finish with a quick speech about why it should be in the kernel. :)
[10:38] pieterh doesn't have to map directly
[10:38] pieterh one level of abstraction will work fine
[10:39] sustrik jsimmons: :)
[10:39] pieterh everyone there is wondering, as you give examples, "yes, yes, yes but how can I make distributed systems?"
[10:39] pieterh which you answer right afterwards
[10:39] sustrik what would be the answer?
[10:40] pieterh 5-line python example
[10:40] sustrik showing a client/server example is not exactly it
[10:40] pieterh yes it's fine
[10:40] jsimmons To me if I were going to talk to kernel developers I'd be taking codes. and they'd better be good ones too.
[10:40] sustrik that's what people call "centralised"
[10:40] pieterh you need to clearly explain that the only reason for centralization
[10:40] pieterh is the high cost of doing network programming
[10:40] pieterh if that is lower
[10:40] sustrik what's missing is the idea of interjecting devices in the middle
[10:40] pieterh decentralization is inevitable
[10:41] pieterh i don't agree you need to speak of scale
[10:41] pieterh devices, etc. who cares
[10:41] pieterh :-)
[10:41] mato wrong
[10:41] sustrik agreed
[10:41] pieterh don't talk about your technology
[10:41] mato that's the *major* point
[10:41] mato that's the *major* point
[10:41] mato that's the *major* point
[10:41] mato :-)
[10:41] pieterh you think it is
[10:41] pieterh i disagree
[10:41] pieterh you can't jump from Eben'
[10:41] pieterh Eben's speech to devices in 1 hour
[10:42] pieterh you will lose them
[10:42] sustrik you don't have to go into detail
[10:42] pieterh this is detail
[10:42] pieterh that's my view, anyhow
[10:42] sustrik but it should be clear that 0mq can be used to produce distributed systems
[10:42] pieterh 45 minutes goes by *really quickly*
[10:42] sustrik if the only thing you show is client/server, you haven't proven your point
[10:43] sustrik i know
[10:43] pieterh you can show something like this, hang on...
[10:43] sustrik diagrams maybe
[10:43] pieterh http://www.zeromq.org/docs:user-guide#toc24
[10:43] pieterh it's still a trivial code example
[10:43] pieterh but shows scale
[10:43] sustrik yes, distribution tree
[10:44] pieterh but don't go into theories of dumb networks and smart endpoints etc.
[10:44] sustrik that's good but should be shown in the context of an example
[10:44] pieterh yes
[10:44] pieterh data distribution by itself is a very important use case
[10:44] sustrik say decentralised social networking
[10:44] pieterh forget request-reply
[10:44] pieterh right
[10:44] pieterh updates
[10:44] sustrik the box on the top is you
[10:44] pieterh sure
[10:44] sustrik the boxes in the bottom are your friends
[10:44] jsimmons I presume everyone will have computers?
[10:44] pieterh ok, i have it
[10:44] pieterh :-)
[10:44] sustrik the boxes in the middle are groups
[10:44] pieterh you can make this live
[10:45] pieterh you'll like this
[10:45] sustrik jsimmons: no idea
[10:45] pieterh decentralized twitter
[10:45] pieterh push updates to all
[10:45] pieterh subscribe to anyone
[10:45] sustrik no central authority
[10:45] pieterh i'm going to use this as the example in the Guide
[10:45] pieterh no more fail whale
[10:45] sustrik actualy yes, twitter is even better than facebook
[10:45] sustrik as it is simpler
[10:46] pieterh yup cause it's a pure push
[10:46] jsimmons what's the free twitter :)
[10:46] pieterh fritter?
[10:46] sustrik :)
[10:46] mato pieterh: people have already implemented this, look at the ostatus/statusnet/identi.ca crowd
[10:46] jsimmons sustrik, find out, I'd presume so. If you could whip up a sample that everyone can compile real quick and instantly become part of a distributed network
[10:46] pieterh mato: yes, but not in 10 lines of python
[10:47] mato pieterh: sure, i'm just pointing out that it exists, and taking that into account is useful
[10:47] sustrik jsimmons: not enough time for that i think
[10:47] pieterh sustrik: just describe it and show one chart
[10:47] mato pieterh: "here's a quick reimplementation of statusnet in 10 lines of python" :-)
[10:47] pieterh that'll prove the point
[10:47] pieterh mato: sure
[10:47] pieterh except i have no idea what statusnet is
[10:48] mato pieterh: open source and decentralized twitter afaik
[10:48] sustrik distributed trwitter i assume :)
[10:48] pieterh sure
[10:48] mato see, /me is a database :-)
[10:48] sustrik ok, can you write that down?
[10:48] pieterh sustrik: you speaking to me?
[10:48] sustrik yes
[10:48] pieterh "that" is the structure?
[10:48] pieterh ok
[10:49] sustrik i mean the text
[10:49] sustrik moglen/twitter thing
[10:49] pieterh ok... i'll give it a shot
[10:49] mato sustrik, pieterh: i think you should mention at least in passing that 0mq is also applicable to a distributed facebook
[10:49] sustrik thanks, you'll help me tremendously
[10:49] mato no detail, but worth mentioning
[10:50] sustrik i can do that, np
[10:50] mato hmm
[10:51] pieterh sustrik: what I will do is expand the text i already wrote into what feels workable for 45 minutes of talking, ok?
[10:51] mato another interesting example would be a modern reimplementation of irc
[10:51] mato using 0mq
[10:51] jsimmons If you had a python script hosted on your machine, ready to go and you just publish your ip at the start. only problem is they might be too interested in that to listen to you :)
[10:51] mato since irc is inherently both peer to peer and distributed, and has the concept of intermediate servers afaik
[10:51] pieterh mato: indeed, but this is about cutting down, not adding more
[10:51] mato i.e. devices
[10:51] mato pieterh: sure, i'm just thinking aloud
[10:51] sustrik pieterh: slightly less
[10:52] sustrik time for Q & A should be allowed
[10:52] mato pieterh: /me makes a note in database (self) to think about a "irc over 0mq" example
[10:52] pieterh sustrik: yes, ok, 30 mins but please do not read the text aloud!
[10:52] pieterh :-)
[10:52] mato pieterh: with the whole hog including devices and the p2p bits
[10:52] pieterh mato: i think there are some killer 0mq applications waiting to be born
[10:53] sustrik pieterh: no, i am improvising when speaking
[10:53] sustrik but i need something to publish in proceedings
[10:53] pieterh sustrik: excellent, this'll work
[10:53] pieterh i have to write some code first, will work on the text later
[10:53] pieterh it is a busy weekend and my wife is going to go to hospital at any moment to have our baby
[10:53] sustrik jsimmons: yes, that's the problem, it's hard to convey the message in 30 mins even if people are not busy compiling stuff
[10:54] sustrik are you able to make it till 15th?
[10:54] sustrik if not, i'll do it myself
[10:55] pieterh 15th, yeah, today probably
[10:55] pieterh at least an outline, there are parts you'll want to add
[10:55] sustrik ok, great, thanks
[10:55] pieterh thank me when its done :-)
[10:55] sustrik ok
[12:01] AndrewBC heh
[12:59] pieterh sustrik: i have a really good example
[13:04] CIA-20 zeromq2: 03Martin Sustrik 07wip-shutdown * r28c8b5b 10/ (8 files): one more dezombification bug fixed - http://bit.ly/bkK0xS
[13:04] sustrik pieterh: shoot
[13:04] pieterh well... email
[13:04] pieterh essentially distributed, and successful as such
[13:04] pieterh slowly being replaced by gmail
[13:05] pieterh i have an explanation why
[13:05] sustrik ?
[13:05] pieterh which is taken from one of my books in progress
[13:05] pieterh basically gmail is a mediocre client
[13:05] pieterh mato: just put away the knee-jerk reaction for a second
[13:06] mato trolled!
[13:06] pieterh gmail solves the most vital problem for (normal) users
[13:06] pieterh that is, spam
[13:06] pieterh how it does that is by sharing knowledge of "this is spam"
[13:06] pieterh 20 people mark a message as spam, 20M never see it
[13:06] pieterh now in theory you COULD do that for distributed email
[13:06] pieterh people have kind of... tried... but failed
[13:06] pieterh it requires massive distribution of data
[13:07] pieterh and really rapidly, no point it arriving after people already got the spam
[13:07] sustrik pieterh: it's not an uncontroversial example
[13:07] pieterh "high cost of writing connected code" makes it practically impossible to solve problems
[13:07] pieterh that centralized systems can solve trivially
[13:07] sustrik what we need is somethig obvious
[13:07] pieterh why is this controversial?
[13:07] mato pieterh: i think you're very wrong about gmail and spam
[13:07] pieterh mato: you don't use gmail, do you
[13:07] mato pieterh: tell me if you're interested in why...
[13:08] sustrik people have to think about, then they can either agree or disagree
[13:08] mato pieterh: i don't use gmail for other reasons
[13:08] pieterh mato: no, i'm interested in examples that compare distributed vs. centralized services
[13:08] sustrik that takes valuable time
[13:08] mato pieterh: but i believe there are several flaws in your hypothesis
[13:08] mato pieterh: about spam & gmail
[13:08] sustrik what we need is something that people just say: "ah, ok"
[13:09] mato pieterh: anyway, i'm happy to expand on why exactly if you're interested
[13:09] pieterh i'm listening to suggestions of distributed vs. centralized services where cost of messaging is the differentiating factor
[13:09] pieterh mato: you don't use gmail so it's like a virgin discussing sex
[13:09] pieterh fun to watch but not useful at this stage
[13:09] pieterh sorry
[13:10] sustrik the cost is irrelevant to the topic imo
[13:10] sustrik what's needed is an obvious example
[13:10] pieterh sustrik: cost of messaging is the key factor
[13:10] pieterh people do want to build distributed systems but can't
[13:10] pieterh IMO
[13:10] sustrik i am trying to explain not sell
[13:10] pieterh ok, whatever
[13:10] AndrewBC I tried to set up email on my server once
[13:10] pieterh i'll continue my text, do what you like with it afterwards
[13:10] mato pieterh: right, i will go fuck off then, seeing as you're obviously not interested in my opinion
[13:11] pieterh mato: trolled! :-)
[13:11] pieterh lol
[13:11] mato 1:1
[13:11] pieterh yo mama
[13:11] pieterh ok, tell me
[13:12] mato ok, first, i've yet to see anyone claim that spam can be solved with distributed automation using a single corpus
[13:12] mato maybe this is what google are doing, maybe not
[13:13] mato having said that, spam is a mostly solved problem without that anyway
[13:13] mato i have been using a spam filter (CRM114 specifically) on all my mail for years now
[13:13] mato you train it more or less efortlessly and it works
[13:13] pieterh i recall endless (endless!) problems with spam before moving to gmail and letting it handle all my email
[13:14] mato sure, because you didn't choose the right tool
[13:14] mato i tried various things, picked this one, it worked
[13:14] pieterh that kind of proves my point
[13:14] AndrewBC This is why users choose gmail instead of doing it themselves.
[13:14] AndrewBC The pains of doing it right.
[13:14] pieterh i don't have time, almost no-one does, to become an expert in "tools" just to use email
[13:14] pieterh you represent a very select part of a very select elite
[13:15] mato yes, sure, it's one of the reasons
[13:15] pieterh my thesis is that requiring people to find and use those tools is a failure
[13:15] mato of course
[13:15] pieterh and Google's trick is to share the corpus
[13:15] AndrewBC Writing a whole thesis on that? :)
[13:15] pieterh and that can be done except messaging is too expensive
[13:15] pieterh AndrewBC: I was writing a book...
[13:15] mato so you think yahoo, hotmail, etc do not share the corpus?
[13:15] pieterh one of several...
[13:15] pieterh mato: they did not do this as well, Google used their experience from search
[13:16] mato pieterh: the point i'm trying to make is that gmail is not that special, at least not for spam
[13:16] mato there's something else which in my opinion makes it special
[13:16] AndrewBC pieterh, oh you were serious. Add a couple of degrees of incredulity to my original question then. :P
[13:16] pieterh AndrewBC: it's half a page in a book of 250 pages...
[13:16] mato which is, search, and through search the use of email as a database :-)
[13:16] pieterh mato: yes, indeed, you are right
[13:16] AndrewBC Oh see, now you've gone and ruined all my fun
[13:17] pieterh AndrewBC: fun, you want fun? Down the hall, room 5b
[13:17] mato pieterh: and that search has nothing to do with distributed systems really
[13:17] pieterh mato: i agree
[13:17] AndrewBC I have a feeling that I'm in for an argument
[13:17] pieterh AndrewBC: no, this is insults. arguments are up in 11c
[13:18] AndrewBC Oh, okay. :D
[13:18] jsimmons and in gmail where you have a centralised client. as opposed to email's annoying decentralised model, as far as client<>server goes anyway.
[13:18] pieterh mato: point taken, i shall have lunch and reflect on this
[13:18] mato yes... lunch...
[13:19] jsimmons although I'm really just talking about POP, IMAP has other issues
[13:19] pieterh well apart from searching, spam filtering, huge mailboxes, labels, no fuss, what has Gmail EVER DONE FOR US?
[13:19] pieterh it's a serious question :-)
[13:19] mato pieterh: i would also be very interested to actually hear someone from google say if they do use the distributed method for flagging spam
[13:19] pieterh mato: trade secrets, i'm almost certain
[13:20] pieterh and it's not distributed, really
[13:20] jsimmons gmail android pieterh. best mobile email client eva
[13:20] AndrewBC I hope they do, because I'm such a good spam tagger
[13:20] jsimmons now with copy+paste!
[13:20] pieterh :-)
[13:20] jsimmons I don't get any spam to tag :(
[13:20] AndrewBC Haha, sucker.
[13:21] AndrewBC Wait.. that's not right.
[13:21] mato mine. all mine. :-)
[13:21] jsimmons only cus you keep the details so you can get cheap viagra on the down low
[13:21] AndrewBC farming the pharma
[13:21] mato :-)
[13:23] sustrik mato: wip-shutdown
[13:23] mato sustrik: jaaa?
[13:23] sustrik it's in the state where termination is delayed till messages are sent
[13:23] sustrik then it fails or freezes :)
[13:23] sustrik but at least the basic thing looks workign
[13:24] mato goody good
[13:24] mato so needs lots of testing, right?
[13:24] sustrik yes, the code is completely rewritten
[13:24] mato ok
[13:25] mato sustrik: unlikely i will get to it before next week
[13:25] sustrik sure, no haste
[13:25] mato sustrik: i have to deliver this c++ thingy by tomorrow
[13:25] sustrik np
[13:25] mato sustrik: so, next week i can do some basic testing
[13:26] mato sustrik: i'd also like to announce that work on the list
[13:26] mato sustrik: and make it destined for 2.1
[13:26] mato sustrik: that's if you agree...
[13:27] sustrik yes, sure, i do
[13:42] pieterh_lunch ok, mato
[13:43] pieterh why do people tend to build centralized services?
[13:43] mato pieterh: well, for search the answer is it's a hard problem which we don't know how to decentralize well
[13:44] mato pieterh: (search == web search)
[13:44] pieterh yes, search
[13:44] pieterh but why in general do people tend to build centralized services?
[13:44] mato control? power? profit?
[13:44] mato tell me...
[13:45] pieterh part is the control instinct
[13:45] pieterh the movie industry is highly centralized
[13:45] pieterh so that people can control distribution
[13:45] pieterh the fashion industry is highly decentralized
[13:45] pieterh it's 4x larger
[13:45] pieterh part is the simplicity of a star network
[13:46] pieterh conceptually, much easier to grok
[13:46] pieterh skype is hard to understand as p2p, easy as centralized service
[13:46] mato for the end user, yes...
[13:46] pieterh you go somewhere for data, data does not come to you
[13:47] pieterh but i think the reason is even more stupid than that
[13:47] pieterh here's the results of my lunch rumination
[13:47] pieterh - it is really hard to build a working communications stack
[13:48] pieterh - the only one that kind of worked widely, by the 90s as we got all that network capacity was HTTP
[13:48] pieterh - therefore every programmer reaches for HTTP as their first choice
[13:48] mato um, IP, surely?
[13:48] pieterh no-one programs IP and it's not a stack
[13:48] mato huh?
[13:48] pieterh - therefore every architect thinks in terms of HTTP and centralized services
[13:49] soren python-ipy
[13:49] mato pieterh: it's called "sockets" :-)
[13:49] pieterh exactly
[13:49] pieterh its a real pain to use sockets in anger
[13:49] soren Oh, that sort of IP library. Not python-ipy then :)
[13:49] pieterh thus a $bn messaging middleware industry
[13:49] pieterh we know the list of problems you have to solve
[13:49] pieterh it's endless and horrible
[13:49] pieterh ja?
[13:50] mato more or less
[13:50] pieterh thus, people reach for what works
[13:50] pieterh SOAP, XMLrpc, etc.
[13:50] mato yes, of course
[13:50] pieterh which leads them all to centralized services
[13:50] mato it's always easier to take something that "mostly works"
[13:50] pieterh yes
[13:50] pieterh so i see 0MQ as the antidote to HTTP
[13:51] mato interesting line of thought...
[13:51] mato yes.
[13:51] mato i like it
[13:51] pieterh HTTP as level7 stack is toxic to humanity
[13:51] pieterh ok
[13:51] mato sustrik: you reading this? pieterh has a nice point
[13:51] pieterh the contrast is nice: sync vs. async, patterns, knowledge in centre vs. edges, etc.
[13:52] mato of course it's all blue-sky future, but the point is 0MQ *could* be the antidote to all this use of HTTP-as-universal-messaging
[13:52] pieterh yes, that's it
[13:52] pieterh i don't see a conspiracy to create facebook
[13:52] pieterh its just been easier up to now
[13:53] pieterh ok, me needs coffee now
[13:53] mato well, actually, there is... since the gravy train is more obvious with a centralized facebook
[13:53] mato versus eben moglen's wall-wart distributed server thingy
[13:54] mato i wouldn't go so far to call it a conspiracy, it's just the plain old capitalist profit motive
[13:54] mato 1.5 billion users using centralized facebook == obvious $$$$$$
[13:55] pieterh mato: that ignores my example that distributed systems are worth many more $$$$$$
[13:55] mato pieterh: maybe, yes, i'm just comparing the facebook example
[13:56] pieterh if we want to position 0MQ as "internet scale messaging" it's nice to consider that we have two internets
[13:56] pieterh like two parallel realities or futures
[13:56] pieterh one, centralized. two, distributed.
[13:56] pieterh one, HTTP. two, 0MQ.
[13:56] pieterh and that is why 0MQ MUST go into the Kernel!
[13:56] pieterh :-)
[13:56] mato :-)
[13:57] mato pieterh: ok, back to work, /me wants to get this $%#^%# c++ working
[13:57] mato almost there...
[13:57] pieterh ack
[13:57] pieterh thanks btw
[13:58] mato np
[13:58] cremes any word from the ietf guys on websocket framing?
[13:59] pieterh cremes: discussion paused, but i think the "consensus" is at a 0MQ-style frame
[13:59] cremes good to hear
[13:59] cremes at some point i would like to be able to use 0mq from the browser
[13:59] pieterh yes, it would be good to tunnel over websockets
[13:59] cremes similar framing will only make that easier i would think
[14:00] pieterh indeed
[14:00] cremes are you aware of any 0mq javascript projects?
[14:00] pieterh though i was thinking of a very simple way to bridge HTTP to 0MQ
[14:00] cremes i haven't seen any here on irc or the ML, but you might know of some below-the-radar projects
[14:00] pieterh i have a design
[14:00] pieterh but no time to start on it
[14:01] cremes heh
[14:01] pieterh basically, AMQP stripped down to run as native HTTP
[14:01] pieterh not even RESTful, just plain HTTP methods
[14:01] pieterh request-reply, pubsub, pipeline
[14:01] cremes right
[14:01] pieterh it would not be 0MQ but it would feed smoothly into a 0MQ network
[14:02] pieterh which is just as good IMO
[14:02] pieterh plus you get all the benefits of HTTP
[14:02] cremes you probably need a paying customer to find time for it... :)
[14:02] pieterh yup
[14:02] pieterh as usual
[14:02] pieterh pure R&D is a waste of time, it produces things no-one uses
[14:03] pieterh lol
[15:02] sustrik mato: yes, read it now
[15:02] sustrik pieter is right
[15:02] sustrik http is misused in terrible ways
[15:09] dos000 http is used as a transport protocol .. when it is an app protocol
[15:10] dos000 and a beautifully simple app protocol
[17:00] moorooboorai hello, this a good time and place to raise an issue on installing the erlang bindings?
[17:16] cremes moorooboorai: i recommend you ask and see; it may be hours but if you stay in the channel you may eventually get a response from an erlang person
[17:17] moorooboorai hi cremes: thanks, I will take my time ;-)
[17:18] moorooboorai main problem: the installation instructions as described on http://www.zeromq.org/bindings:erlang don't work out
[17:19] moorooboorai the git-extracted sources don't include the ./configure - using autconf on it (running Ubuntu here) doesn't help either
[17:19] cremes moorooboorai: try searching the ML for an answer too
[17:20] moorooboorai will do, but was kinda hoping for follow up on the 'running within 5 minutes' slogan...
[17:21] cremes moorooboorai: heh, yeah... try it with the C, python or ruby bindings then :)
[17:23] moorooboorai yeah, right...that's not what I am after, of course..
[17:24] moorooboorai really a shame, though. Think Pieter's writing's are very convincing. Was coming from RabbitMQ/AMQP and found exactly the right words in Pieter's 'rant' on AMQP. I am sure my issue will just be a minor glitch.
[17:25] sustrik pieterh, mato, all: here's a first shot on introduction: http://www.250bpm.com/hits
[17:26] cremes moorooboorai: i took a look at the erlang bindings page; it notes that it requires a feature not present in the 2.0.7 release
[17:26] cremes if you are trying to track master right now, then i'm not surprised the erlang bindings blow up
[17:26] cremes master is undergoing a lot of changes right now and is probably moving faster than the erlang-bindings folks are tracking it
[17:27] cremes that's one of the downsides of immature libraries, unfortunately
[17:27] sustrik moorooboorai: as for now use sustrik/zeromq2 branch
[17:28] sustrik it's kind of freezes on exit
[17:28] sustrik but should be ok for prototyping
[17:28] sustrik the whole thing is going to be settled down shortly
[17:30] moorooboorai Thanks, but I am using the sustrick/zeromq2 branch.
[17:30] moorooboorai As per the instructions.
[17:31] sustrik ah
[17:31] sustrik sorry, i've misread
[17:31] sustrik you have to run ./autogen.sh first
[17:31] sustrik that will generate the configure script
[17:32] moorooboorai okay, cloning again now
[17:34] moorooboorai Thanks! It's building now. Thinks the instructions could use some updates?
[17:34] sustrik sure, if you find it confusing, fix the instructions
[17:34] moorooboorai Also for the part where the command "git git@github.com..." is used. It ends up with a permission denied.
[17:35] sustrik :(
[17:35] moorooboorai Can I fix them? If so, then I will do that of course.
[17:35] sustrik click join wiki button on the left bottom of the page
[17:35] sustrik once you've joined you can edit documents
[17:44] moorooboorai have made the changes; if anything else pops up I will come back here
[17:44] moorooboorai Thanks for the help so far!
[17:44] sustrik you are welcome
[19:09] pieterh re
[19:11] pieterh moorooboorai: thanks for fixing the Erlang page
[19:12] pieterh sustrik: the introduction makes sense but it takes 2-3 readings to understand the point
[19:12] pieterh maybe i'm just tired :-)
[19:15] sustrik hm
[19:15] sustrik what do you think is unclear?
[19:18] sustrik i should probably state the point fists, then discuss it, then re-state it at the end
[19:27] pieterh sustrik: your core argument is that distributed systems are more successful
[19:27] pieterh this argument is easily disproven, unfortunately
[19:27] pieterh at least superficially
[19:27] sustrik ?
[19:28] pieterh 100% adoption of GSM (centralized), 100% adoption of centralized movie distribution, 90% adoption of Windows, 100% of Euro in Eurozone, etc.
[19:28] moorooboorai pieterh: glad to be of service - truly hope zeromq lives up to my (now) great expectations!
[19:28] sustrik true, i drop the 100% thing
[19:28] pieterh moorooboorai: i'm interested to know what you appreciated about my amqp analysis
[19:29] pieterh sustrik: you are going to argue that a distributed service bus is essential, right?
[19:30] pieterh then you step into economics and politics even
[19:30] sustrik yes, but i have no time for that
[19:30] pieterh exactly
[19:30] sustrik i'll try to keep it minimal
[19:30] sustrik jsut an introduction like this one
[19:30] sustrik then some technical example
[19:31] sustrik then the kernel thing
[19:31] pieterh drop the claim that tcp/ip dominates because it's distributed, that is not accurate
[19:31] sustrik why so?
[19:31] pieterh it is simply "natural monopoly"
[19:31] sustrik if it was cetnralised who would use it?
[19:31] pieterh 220v is centralized
[19:31] pieterh euro is centralized
[19:31] pieterh i repeat all the examples
[19:32] pieterh it dominates because it was cheaper than its competitors
[19:32] sustrik which is consequence of being distributed
[19:32] pieterh it's a consequence mainly of having no patents
[19:32] pieterh no license fees
[19:33] pieterh ok, 110 patents on the RJ45 socket, but not the digital part
[19:33] moorooboorai pieterh: first one has to do with style: you were able to keep a very lenghty argument interesting to read; whilst reading the user guide I am - again - pleasantly surprised by your style of writing. It is humorous, takes a broad view and still singles out the subject.
[19:33] sustrik sure, that two
[19:33] sustrik too
[19:33] pieterh whereas, for example, Microsoft launched MSN precisely to try to kill IP
[19:33] sustrik but if there was a single party in control it would try to get as much money from it as possible
[19:34] pieterh so IP enabled a competitive market whereas MSN et al did not
[19:34] sustrik exactly
[19:34] sustrik and the competitive market got the price down to zero
[19:34] pieterh yes
[19:34] pieterh as will always happen in a friction free market
[19:34] sustrik if it weren't distributed there wouldn't be a competitive market
[19:35] pieterh that is correct and that's the chain of logic
[19:35] pieterh you can't skip the intermediate step
[19:35] pieterh IMO
[19:35] sustrik which one is that?
[19:35] pieterh the economics of it
[19:35] sustrik eh?
[19:35] sustrik i mean what would you skip?
[19:36] pieterh sorry, i'll try to clarify
[19:36] moorooboorai pieterh: second one is concerned content-wise: you nailed the 'feeling' that slowly crept upon me. Seeing the guys from RabbitMQ faltering in keeping the pace made me worry big time. Then stumbling upon your argument (pinpointed by the wiki on AMQP) which suddenly gave me a very keen sense on the real reasons behind that faltering. It's very much reminiscent of the company politics you enter into in big enterprises. So point
[19:36] moorooboorai being: you've persuaded me to give zeromq a try, not on the technical specs but on it's apparent "better" organizational setting.
[19:36] pieterh sustrik: a second, let me re-read your abstract
[19:37] pieterh moorooboorai: this is sadly where AMQP seems to have landed
[19:39] pieterh moorooboorai: BTW, welcome to our little community, and thanks for not fleeing when it didn't work right away
[19:40] sustrik yes, the argument can be made much clearer if i focus on the single fact: distributed architecture creates a competitive market whereas centralised architecture does not
[19:40] sustrik consequently, markets created on distributed architectures flourish
[19:40] sustrik and the price is drawn down
[19:40] sustrik which increases adoption etc.
[19:41] sustrik the question is: should i completely omit freedom, scalability, security etc.?
[19:41] pieterh sustrik: i'm going to propose a new first paragraph
[19:42] pieterh based on economics and standards
[19:42] pieterh simple argumentation but rock solid
[19:42] pieterh see if you like it, 5 mins...
[19:42] sustrik ok
[20:05] moorooboorai Hi guys: although the make for erlzmq works - the make docs command crashes erlang. Error message: erl -noshell -run edoc_run application "'zmq'" '"."' \
[20:05] moorooboorai '[{dir, "../doc"}, {def, [{version, "0.1"}]}]' -run init stop
[20:05] moorooboorai {"init terminating in do_boot",{undef,[{edoc_run,application,[["'zmq'","\".\"","[{dir, \"../doc\"}, {def, [{version, \"0.1\"}]}]"]]},{init,start_it,1},{init,start_em,1}]}}
[20:06] sustrik what platform you are on?
[20:06] moorooboorai Ubuntu 10.04 LTS
[20:07] moorooboorai (Desktop, not server.)
[20:07] sustrik hm, dunno
[20:07] moorooboorai :-))
[20:08] moorooboorai That's okay. Will work it out somehow and bring fixes back - if any.
[20:08] sustrik that would be great
[20:08] sustrik thanks
[20:09] travlr pieterh: off topi -- wikidot.. how do i hide the edit button and other "wiki" options at the bottom of a page from vistors.. i.e. for admin only
[20:09] travlr i can't find this info anywhere
[20:09] pieterh travlr: in your site manager, permissions, uncheck the "Show page options" option for that category
[20:09] travlr thanks
[20:09] pieterh np
[20:10] moorooboorai (Travlr must be concerned because of my 'wild' adjustments... ;-) )
[20:10] pieterh sustrik: i've found a major flaw in the argument
[20:10] sustrik ?
[20:11] pieterh true competition is possible only with freedom of clients to choose suppliers
[20:11] pieterh i.e. standards for interoperability
[20:11] sustrik yes, and?
[20:11] pieterh a natural monopoly emerges from such a standard, when it's free and open
[20:11] pieterh 0mq is not a standard
[20:11] sustrik not yet
[20:11] pieterh a product cannot become a standard
[20:12] sustrik that's why i talk of 'scalability layer'
[20:12] sustrik i'll explain that later on
[20:12] pieterh if you want to make this argument solid you need to add "wire level protocol standards for service interop" to your 20 year vision
[20:12] pieterh like i've been complaining for years :-)
[20:12] sustrik yes, it'll be mentioned
[20:13] sustrik i agree, it should be there, but we need to make semantics clear first; once the semantics are clear, drafting a WLP is trivial
[20:13] pieterh if you want to understand the importance of free and open standards to this process please read http://www.digistan.org/text:rationale
[20:13] pieterh no, it is not trivial
[20:14] pieterh sorry, that is really wrong :-)
[20:14] pieterh especially if you try to work backwards from code
[20:15] pieterh i think we need to urgently create a standards workgroup
[20:15] sustrik too early
[20:15] sustrik we need a semantics wiorkgroup
[20:15] sustrik what is identity?
[20:15] sustrik how does it interact with different patterns?
[20:15] sustrik etc.
[20:15] pieterh semantics is part of standards
[20:15] sustrik yes
[20:15] pieterh but without a deliberate statement "we want to deliver standards"
[20:15] sustrik but it precedes WLP
[20:16] sustrik here it is"
[20:16] sustrik "We want to deliver standards"
[20:16] pieterh then the argument "we know how to enable competition between service providers" fails
[20:16] pieterh yes, put that on the web
[20:16] pieterh write that on the main page
[20:16] pieterh it's sufficient
[20:16] sustrik sure, do so
[20:16] pieterh but it must be a stated goal of the project
[20:16] sustrik i'm going to discuss that in part 3 of the article
[20:17] pieterh :-) you, please, since otherwise it's pieter injecting his "OMG lets standardize everything" ideology :-)
[20:17] pieterh ok, almost done with the intro
[20:17] sustrik lemme see
[20:17] pieterh hang on...
[20:17] pieterh if you are bored plz go read that digistan article
[20:18] sustrik reading an article on stratification of plant seeds right now :)
[20:18] sustrik but i read your rationale before
[20:19] sustrik i have nothing against standards
[20:19] sustrik the only thing i am saying is "do it properly"
[20:19] sustrik AMQP trauma, you know
[20:19] pieterh you should always learn from trauma
[20:20] pieterh AMQP was a brilliant concept badly done
[20:21] pieterh the part that was done correctly was the careful "consensus on the standard and running code" that preceded the madness
[20:21] pieterh this is also how IP was built
[20:21] sustrik sure i know
[20:21] pieterh not by people writing code
[20:21] pieterh sorry but that's the truth: RFCs came before code
[20:21] pieterh they also came out of code but everything before the RFC was thrown away
[20:22] sustrik seems reasonable
[20:23] pieterh next week in bratislava i'd like to take a few hours to sketch a draft 0mq WLP covering everything we do
[20:23] sustrik np
[20:23] pieterh gr8
[20:27] pieterh sustrik: ok, see http://gist.github.com/521665
[20:27] pieterh it is kind of pedantic but the argumentation is correct IMO, yours to play with as you like
[20:29] sustrik twenty orders of magnitude?
[20:29] sustrik 100000000000000000000x?
[20:30] pieterh yes, i think so
[20:30] sustrik good god
[20:30] pieterh look, it costs me 10 euro to send 500MB by phone
[20:30] pieterh and that is CHEAP for Belgium
[20:30] sustrik that's what i call a meargin
[20:30] sustrik margin
[20:30] pieterh if you count SMS as your unit of transfer
[20:30] pieterh it's 0.10 euro for 160 bytes
[20:31] pieterh that is actually the basis for the comparison
[20:33] pieterh 100TB for $200/month - http://www.100tb.com/
[20:33] sustrik :)
[20:33] sustrik the text is nice
[20:34] pieterh ah, ok
[20:34] sustrik but it's extremely focused on market issues
[20:34] pieterh yes, its very digistan
[20:34] sustrik are kernel devs interested in that?
[20:34] pieterh i suspect more than you'd think, anyhow this is the printed paper
[20:34] pieterh smart people like to understand the argumentation
[20:35] sustrik what i mean, isn't it worth of stressing some other aspects of the thing as well?
[20:35] sustrik freedom
[20:35] sustrik security
[20:35] sustrik scalability
[20:35] sustrik ?
[20:35] pieterh i think they are all emergent properties of free and open standards that enable pure competition
[20:35] pieterh :-)
[20:35] pieterh there was no charter for freedom in RFCxxx
[20:36] sustrik nope, if you have a free and open standard for centralised services
[20:36] sustrik it won't provide any of it
[20:36] pieterh HTTP vs IP, perhaps
[20:37] pieterh SOAP is a free and open standard for centralized services
[20:37] sustrik yes
[20:37] pieterh it allows centralized services to compete with each other
[20:37] sustrik is it scalable?
[20:37] pieterh HTTP is scalable, yes
[20:37] pieterh Ajax, websockets, etc. scales
[20:38] sustrik i mean what happens on the "server" side if there's too much requests?
[20:38] sustrik it'll fail
[20:38] pieterh oh, it scales
[20:38] pieterh this is a solved problem
[20:38] sustrik ?
[20:38] pieterh you throw money at it
[20:38] sustrik ah, that way
[20:38] pieterh caches, distributed static file servers, larger machines, clusters, etc.
[20:38] dos000 pieterh, i am going through the guide .. very amazing work btw ...
[20:38] sustrik sure
[20:39] pieterh dos000: glad you like it...:)
[20:39] sustrik anyway, doesn't matter
[20:39] pieterh sustrik: so scalability is not the magic ingredient
[20:39] pieterh and in theory HTTP does enable distributed services
[20:39] pieterh that was the idea behind web services, really
[20:39] sustrik yes, sure, even SOAP
[20:40] dos000 pieterh, how can you that guaranties state based processing of incoming sockets .. so a socket is only processed following a state machine
[20:40] pieterh why did it fail?
[20:40] sustrik the problem that the "scalability layer" on top of it is not defined
[20:40] pieterh dos000: you mean in 0MQ or in your app?
[20:40] sustrik things like: if the service is unavailable, what should I do?
[20:40] dos000 pieterh, in the figure 6 example worker threads will share the load ..
[20:40] sustrik SOAP doesn't address the problem
[20:40] dos000 pieterh, in my app
[20:41] pieterh dos000: ah, i have no answer for that, yet
[20:41] dos000 ah!
[20:41] pieterh it is actually not possible today
[20:41] pieterh you have to delegate the management of peer connections to 0MQ
[20:41] pieterh you can manage sockets individually using poll() but it will not give you state per remote peer
[20:41] pieterh this is by design
[20:42] pieterh sustrik: it looks like an answer searching for a problem
[20:42] pieterh "I have this scalability layer, now I need to explain why you need it"
[20:42] pieterh "But I don't have a scalability problem!"
[20:42] dos000 i have to process these requests sequentially so i have to keep track of the state of each socket in my protocol
[20:42] pieterh "Ah, but if you didn't rely on Google, you would have!"
[20:42] sustrik well, yes
[20:42] sustrik and?
[20:42] pieterh dos000: if you are trying to track individual sockets, you're probably doing it wrong
[20:43] pieterh sustrik: there has to be another hook
[20:43] sustrik there is lot of them
[20:43] sustrik what i am asking whether economics is the right hook for kernel devs
[20:43] dos000 pieterh, the protocol requires that requests coming from the same client be treated in sequence
[20:44] pieterh dos000: sorry, a moment...
[20:44] dos000 pieterh, ok
[20:44] pieterh sustrik: http already offers scalable, free, distributed services
[20:45] pieterh you're right that the scaling happens inside, not outside the wall
[20:45] pieterh that is perhaps the key difference
[20:45] pieterh this is what you mean by 0MQ in the Internet fabric
[20:45] sustrik yes
[20:45] pieterh your switch does pubsub and request-reply
[20:45] sustrik yes
[20:46] sustrik but that's too complex
[20:46] pieterh this is the hook, then
[20:46] pieterh right, but it's the critical path
[20:46] sustrik waht i meant was whether it's not worth of putting some more emphasis on freedom or so
[20:46] pieterh you want to appeal to emotions?
[20:46] pieterh perhaps
[20:46] sustrik right now the argument is 95% economics and 5% freedom
[20:46] sustrik rather a more balanced view
[20:46] pieterh freedom and economics are not different things
[20:47] sustrik i am just commenting on your text
[20:47] sustrik it's seems to focused on economic aspect
[20:47] sustrik "which is primarily a vehicle for delivering Google services (and ads) had 1.9% of the US smartphone market by Q2. In 2010, 12 months later, it had 17% of the US market"
[20:47] pieterh ok, so you want some more familiar keywords ;-)
[20:47] pieterh yes, linux is dominating
[20:48] pieterh but it's not quite as pretty as you thought it might be
[20:48] sustrik i mean why should that kind of info even be there
[20:48] sustrik it's for kernel devs, not businessmen
[20:48] pieterh that is to demonstrate the utter power involved
[20:49] pieterh a new kind of power on a scale you can't really comprehend
[20:49] sustrik ok, too late today, let's continue tomorrow
[20:49] pieterh sure
[20:49] sustrik good night
[20:49] pieterh dos000: we can continue if you like
[20:49] pieterh g'night martin!
[20:50] dos000 pieterh, ok ..
[20:50] dos000 so i did this java nio before
[20:50] pieterh right, so i probably have the answer for you
[20:50] pieterh speaking of nio
[20:50] pieterh which is one of the reactors on 0MQ Labs
[20:51] dos000 the only one way i could track the state using multiple threads was to remove the scket from the polling thread
[20:51] pieterh a quick question: how much have you already used 0MQ?
[20:52] dos000 pieterh, no actual code yet
[20:52] pieterh ok, so what you need to do is spend a day just writing different simple 0MQ cases
[20:53] pieterh before trying to design anything using 0MQ
[20:53] dos000 ok .. i buy the argument
[20:53] dos000 good point i mean
[20:53] pieterh this is something I need to explain in the Guide
[20:54] dos000 can i ask how the reactor is going to handle this ou it will go over my head ?
[20:54] pieterh i've not used the reactors
[20:54] dos000 i can also wait for the guide
[20:55] pieterh so my ignorant understanding is: the reactor is like poll turned inside out, it will invoke your code based on events arriving on sockets
[20:55] dos000 and it will do it on multiple threads ? or it will be a thread per request ?
[20:55] dos000 thread per connection i mean
[20:57] dos000 for the reactor pattern the ace lib explains it very well .. but maybe zmq is not following the way ace/tao did it
[20:59] pieterh dos000: sorry, was on another thread
[21:00] pieterh threads, requests, sockets...
[21:00] pieterh most of what you think you know about this won't map to 0MQ
[21:00] dos000 ah
[21:01] dos000 how expensive is it to add or remove a socket to/from the poll ?
[21:01] pieterh there should be a sign above the entrance saying "Warning: leave your understanding of threads, requests, and sockets at the door"
[21:01] pieterh pollsets are just structures
[21:02] dos000 ok .. thats what i hoped
[21:02] dos000 so please review this algo rithm with me
[21:02] pieterh dos000: seriously... what you are going to explain to me is broken :-)
[21:02] dos000 please ..
[21:02] pieterh :-) shoot
[21:03] dos000 as soon a activity comes on a socket the polling thread removes the socket from the pollset
[21:03] dos000 this guaranties that only one thread will always hadle a socket at a time.
[21:03] dos000 after the worker thread is done with the socket it puts it back in the pollset
[21:03] pieterh can i stop you now?
[21:03] dos000 sure .
[21:04] pieterh so you are thinking in terms of managing, yourself, the sockets that speak to remote peers
[21:04] pieterh you are trying to map your knowledge of how to build a multithreaded server onto the 0MQ example
[21:04] pieterh right?
[21:05] dos000 yes correct
[21:05] pieterh here is the key piece of information your mind is refusing to accept, because it goes against everything you know
[21:05] pieterh there is no connection
[21:05] pieterh or rather, you cannot see, work with, open, close, attach state to the connection
[21:06] pieterh the socket is NOT the connection
[21:06] dos000 ah
[21:06] dos000 unless you do your own poll correct ?
[21:06] pieterh no
[21:06] pieterh no unless
[21:06] dos000 ok .. then i am greatly mistaken
[21:06] pieterh there is no unless here, it's fundamental to 0MQ
[21:06] pieterh the socket is a smart application that manages ALL the connections for you
[21:07] pieterh you can talk to the socket
[21:07] pieterh and poll lets you talk to multiple sockets at once
[21:07] pieterh but you cannot talk to the socket's connections, that is private, hidden
[21:07] dos000 ok .. got you ..
[21:07] pieterh this abstraction is what lets you build properly scalable applications
[21:07] pieterh at first you're like... "... wtf?"
[21:08] pieterh and after you've tried it for a while, you're... "... dude!"
[21:08] dos000 i am not at the ... duude yet !
[21:08] pieterh it'll come
[21:08] pieterh :-)
[21:08] pieterh i'm going to capture this little discussion for the Guide, it's been very useful, thanks
[21:09] dos000 thanks to you man
[21:09] pieterh np duude
[21:09] dos000 should be daman
[21:10] dos000 u-da-man
[21:10] man nah, sustrik is daman
[21:37] travlr huh. what just happened?
[21:38] dos000 when you see these it means ... they changed something in the mattrix
[21:38] travlr odd
[21:38] dos000 all my channels dropped
[21:39] travlr weird again..
[21:47] pieterh travlr: ?
[21:49] travlr pieterh: yes sir
[21:49] pieterh what's weird?
[21:49] pieterh just curious...
[21:49] travlr oh, the irc server bugged out and dropped a bunch of users and then joined them again
[21:50] pieterh ah, right, and i thought everyone except me went for a quick beer... :-)
[21:50] travlr congrats on the upcoming new family member btw
[21:51] pieterh hey, thanks
[21:59] AndrewBC travlr, that happens when physical servers get desynchronized or lose connection between each other in the irc server set
[22:12] travlr cool, thanks andrew
[22:38] cremes they need to replace their buggy networking code with 0mq! ;)
[22:38] mato :)

Web site design and content is copyright © 2010-2011 Peter Alexander, ProDataLab. Site content licensed under the Creative Commons Attribution-Share Alike 3.0 License. ØMQ is copyright © Copyright © 2007-2011 iMatix Corporation and Contributors. Contact us for professional support. ØMQ is free software licensed under the LGPL. ØMQ, ZeroMQ, and 0MQ are trademarks of iMatix Corporation. Terms of Use ? Privacy Policy