IRC Log

Wednesday August 11, 2010

[Time] NameMessage
[09:19] mato sustrik: quick C++ best practice question
[09:20] mato sustrik: exceptions in constructors are fine, no? it's just exceptions in destructors that should be avoided
[09:25] sustrik mato: kind of
[09:26] sustrik but it's complex
[09:26] sustrik better to not throw exceptions from constructors
[09:26] mato hmm
[09:27] mato then I would need an extra "start" method that does anything interesting, which seems superfluous
[09:27] sustrik you can have an exception in constructor
[09:28] sustrik but then you should be aware that stack unwind can start with half-constructed object
[09:28] sustrik it you don't do anything strange, it's mostly ok
[09:29] sustrik if there are virtual functions involved it become a bit more tricky
[09:29] sustrik etc.
[09:29] mato ok, i'll try the naive way... not doing anything too strange i think
[09:29] mato no virtual functions
[09:29] mato no inheritance
[09:29] mato no nothing
[09:29] sustrik then it's ok
[09:29] mato thanks
[11:44] Zao Exceptions in ctors are handy to indicate that the invariant of the class cannot be ensured.
[11:45] Zao It sure beats leaving zombie objects that fail to do things on first use.
[12:04] sustrik yup, but can get messy once you start getting 'pure virtual function called' errors etc.
[12:09] Zao I can't quite see how any such behaviour would arise, unless you've built something incorrectly.
[12:10] CIA-20 zeromq2: 03Martin Sustrik 07wip-shutdown * r45bac29 10/ (51 files): I/O object hierarchy implemented - http://bit.ly/cpsGJy
[12:10] Zao As with everything else, know the rules and pray that your implementation is not broken :D
[12:15] sustrik mato: you there?
[12:28] mato sustrik: kind of
[12:28] sustrik how do i push my commit to the branch?
[12:30] mato sustrik: ?
[12:30] mato sustrik: AFAICS you already did
[12:30] mato sustrik: at least CIA-20 said so :-)
[12:31] sustrik a-ha
[12:31] mato sustrik: git push origin on that branch should do the trick if your branches are configured normally
[12:31] sustrik git complained so i though it was unsuccessgul
[12:31] pieterh sustrik: here is how you merge one branch into master
[12:31] pieterh # Switch to the issue branch
[12:31] pieterh git checkout -b 123_nice_feature
[12:31] pieterh # Switch to the master branch and get latest version
[12:31] pieterh git checkout master
[12:31] pieterh git pull
[12:31] pieterh # Merge the issue branch into the current (master) branch
[12:31] pieterh git merge 123_nice_feature
[12:31] pieterh # Push new master back to repository
[12:31] pieterh git push origin master
[12:31] mato sustrik: what did it complain about?
[12:32] sustrik random stuff
[12:32] sustrik let me see
[12:32] mato sustrik: it probably said something like "you didn't tell me that i should push to this remote branch by default, but i'll do it anyway"
[12:32] pieterh you used "git push" without arguments?
[12:33] mato pieterh: he's trying to push to a remote branch from a local branch which may or may not have been configured as a tracking branch
[12:34] pieterh hmm
[12:34] sustrik failed to push some refs to ...
[12:34] mato pieterh: he is also not trying trying to do anything like the $random_procedure you just pasted :-)
[12:34] sustrik to preven from losing history
[12:34] pieterh mato: sure
[12:34] sustrik non-fast-forward updates were rejected
[12:35] sustrik merge the remote changes before pushing again
[12:35] mato sustrik: ah
[12:35] mato sustrik: yeah, that's normal... remember i committed changes to that branch
[12:35] mato sustrik: do a git pull origin
[12:35] sustrik i did
[12:35] mato so, what's the problem?
[12:36] sustrik same error
[12:36] mato sustrik: when doing what?
[12:36] mato sustrik: what exact command ... ?
[12:36] sustrik git push origin
[12:36] mato huh? and "git pull origin" did nothing?
[12:37] sustrik it completed without error
[12:37] sustrik following push failed again
[12:37] sustrik with the same error
[12:37] mato ok, can you paste the entire error please? not re-type it?
[12:38] sustrik it's on the notebook
[12:38] sustrik let me send it to you
[12:38] mato sustrik: pastebin.com or similar is your friend
[12:39] pieterh mato: $random_procedure is simply a way of resolving merge issues locally
[12:39] mato pieterh: he is trying to do something else
[12:40] sustrik done
[12:40] pieterh mato: ack
[12:41] mato sustrik: oh, interesting, it seems to be complaining about something on your local 'master' branch
[12:41] mato sustrik: irrelevant
[12:41] mato sustrik: if you do "git push origin wip-shutdown" it should not complain
[12:41] sustrik ok
[12:41] mato sustrik: same goes for pull
[12:42] mato sustrik: you can always tell it to push/pull only a specific branch
[12:42] sustrik ack
[12:42] mato sustrik: it's possible we left your local 'master' on that machine in some weird state, shouldn't matter
[12:43] mato sustrik: at least not until you want to do something with 'master'
[12:43] mato sustrik: which you don't right now
[12:43] sustrik ok
[13:08] sustrik mato: how come git fetch doesn't result in me seeing the wip branch?
[13:10] sustrik gh
[13:10] sustrik aargh
[13:10] mato sustrik: ?
[13:10] mato sustrik: who's "me"?
[13:10] sustrik martin sustrik
[13:10] mato sustrik: you're trying to pull the branch on another machine?
[13:10] sustrik yes
[13:11] sustrik ghhh
[13:11] mato sustrik: /me never quite understood how fetch works
[13:11] sustrik so far i'm here:
[13:11] sustrik * (no branch)
[13:11] sustrik master
[13:11] mato go back to master
[13:11] sustrik what should i do to clone the branch?
[13:12] mato sustrik: what does "git branch -av" say?
[13:12] sustrik * (no branch) 45bac29 I/O object hierarchy implemented
[13:12] sustrik master a12f446 Modified zmq_tcp(7) to emphasize wildcard interfaces
[13:12] sustrik remotes/origin/HEAD -> origin/master
[13:12] sustrik remotes/origin/master a12f446 Modified zmq_tcp(7) to emphasize wildcard interfaces
[13:12] sustrik remotes/origin/wip-shutdown 45bac29 I/O object hierarchy implemented
[13:12] mato huh?
[13:12] sustrik just tell me how to get to the source code
[13:12] sustrik from the scratch
[13:13] sustrik rm -rf zeromq2
[13:13] sustrik now what?
[13:13] mato nothing, if it's from scratch just clone and then do "git checkout wip-output"
[13:13] mato that should work
[13:14] mato it's when it's not from scratch that you have to sometimes do something funny and i can never remember what
[13:14] sustrik sustrik@istvan:~/zeromq2$ git checkout wip-shutdown
[13:14] sustrik error: pathspec 'wip-shutdown' did not match any file(s) known to git.
[13:15] mato sustrik: give me a sec
[13:17] mato sustrik: i don't understand
[13:17] mato [dezo:tmp]$ mkdir zzz
[13:17] mato [dezo:tmp]$ cd zzz
[13:17] mato [dezo:zzz]$ git clone git@github.com:zeromq/zeromq2.git
[13:17] mato ...
[13:17] mato [dezo:zzz]$ cd zeromq2/
[13:17] mato [dezo:zeromq2]$ git branch -av
[13:17] mato * master a12f446 Modified zmq_tcp(7) to emphasize wildcard interfaces remotes/origin/HEAD -> origin/master remotes/origin/master a12f446 Modified zmq_tcp(7) to emphasize wildcard interfaces remotes/origin/wip-shutdown 45bac29 I/O object hierarchy implemented
[13:17] mato [dezo:zeromq2]$ git checkout wip-shutdown
[13:17] mato Branch wip-shutdown set up to track remote branch wip-shutdown from origin.
[13:17] mato [dezo:zeromq2]$ git branch master
[13:17] mato Switched to a new branch 'wip-shutdown'
[13:17] mato * wip-shutdown
[13:17] mato sustrik: you are doing something different?
[13:18] sustrik sustrik@istvan:~$ git clone git@github.com:zeromq/zeromq2.git
[13:18] sustrik Initialized empty Git repository in /home/sustrik/zeromq2/.git/
[13:18] sustrik remote: Counting objects: 5764, done.
[13:18] sustrik remote: Compressing objects: 100% (1489/1489), done.
[13:18] sustrik remote: Total 5764 (delta 4169), reused 5654 (delta 4083)
[13:18] sustrik sustrik@istvan:~$ cd zeromq2/
[13:18] sustrik sustrik@istvan:~/zeromq2$
[13:18] sustrik sustrik@istvan:~/zeromq2$
[13:18] sustrik sustrik@istvan:~/zeromq2$ git checkout wip-shutdown
[13:18] sustrik error: pathspec 'wip-shutdown' did not match any file(s) known to git.
[13:19] mato ?!?!
[13:19] sustrik !
[13:19] sustrik 8oO
[13:20] sustrik http://www2.ivcc.edu/rambo/eng1001/munch.htm
[13:20] mato sustrik: try "git checkout -b wip-shutdown origin/wip-shutdown" instead
[13:21] sustrik ok, this one works
[13:21] mato seems like that is the canonical way
[13:21] mato no idea why my 'git' is being smart and just doing the right thing :)
[13:22] sustrik ok, never mind
[13:35] mato arrrrgh
[13:36] mato why is it that there are 1001 ways to do anything in C++, of which only ways 1,17,354,771 and 997 actually work !?
[13:37] mato of course and of those only the subset of (17, 771) is any good :-)
[13:38] pieterh mato: the only person who understands 354 is the guy who wrote that part of the template
[13:39] drbobbeaty I can here knowing I'd get good help on ZeroMQ, but this is great comic relief. I feel your pain, mato... but it still makes me giggle.
[13:39] pieterh I think this carries on in the tradition of Algol-68, but it only had 15 ways of doing anything due to bitsize limitations
[13:42] mato :-)
[13:43] pieterh luckily the grand tradition of "just how complex can we make this before people realize we're trolling them" of language construction is dying out...
[13:43] pieterh it is nice to use 0MQ in Python :-)
[13:43] mato yes yes
[13:52] CIA-20 jzmq: 03Gonzalo Diethelm 07master * r21d7a43 10/ (src/Socket.cpp src/org/zeromq/ZMQ.java):
[13:52] CIA-20 jzmq: Added specific methods for each of the sockopt cases.
[13:52] CIA-20 jzmq: Added support for all missing sockopt cases.
[13:52] CIA-20 jzmq: Added support for getting all sockopt cases. - http://bit.ly/akWvk8
[14:22] pieterh mato: quick question
[14:22] pieterh Gonzalo points out that the get/setsockopt man pages seem to be using the wrong int64 type for some options
[14:22] pieterh shall i fix those pages or is this deliberate?
[14:28] mato pieterh: not deliberate, problematic, old bug, requires mucking with most/all the language bindings
[14:28] mato pieterh: don't attempt to fix it
[14:28] sustrik pieterh: socket option types are a mess
[14:29] mato pieterh: if it annoys you, define all the proper types, put out an announcement and solicit replies from as many (ideally all) language binding maintainers that those are ok for them
[14:29] pieterh i won't but could you explain why the man pages specify the wrong sign for most cases?
[14:29] pieterh i don't intend to change the code but document it accurately
[14:29] mato pieterh: ah, just the man pages?
[14:29] pieterh yes
[14:29] mato pieterh: there was chaos for a while
[14:30] mato pieterh: the man pages probably try to be "correct", the code is not really "correct"
[14:30] mato pieterh: so yeah, just update the man pages to match the code
[14:30] mato pieterh: but in any case, it's a mess...
[14:30] mato pieterh: bunch of the options should just be e.g. "int"
[14:30] pieterh i don't think anyone minds what types are actually used
[14:30] mato pieterh: impossible to fix w/o going thru a massive procedure as i wrote above
[14:31] pieterh perhaps for bindings
[14:31] pieterh i'll fix the docs then, thanks
[14:31] pieterh also i assume getsockopt will fetch ZMQ_SWAP?
[14:31] mato hmm, yes, it should, read the code
[14:31] pieterh options.cpp does that
[14:31] pieterh ok, i'll add that as well
[14:31] pieterh thx, over and out
[15:00] CIA-20 zeromq2: 03Pieter Hintjens 07master * re74d350 10/ (doc/zmq_getsockopt.txt doc/zmq_setsockopt.txt): Fixed (un)signed type errors in get/setsockopt manual - http://bit.ly/cX70vO
[15:07] CIA-20 zeromq2: 03Martin Sustrik 07wip-shutdown * r6e9744f 10/ (6 files): engine termination on disconnect fixed - http://bit.ly/cDpnTx
[15:08] CIA-20 zeromq2: 03Pieter Hintjens 0746_device_robustness * r5be54b9 10/ src/xrep.cpp :
[15:08] CIA-20 zeromq2: 46 - Devices vulnerable to invalid messages
[15:08] CIA-20 zeromq2: http://github.com/zeromq/zeromq2/issues#issue/46
[15:08] CIA-20 zeromq2: Invalid messages are now discarded silently, instead of causing an assertion
[15:08] CIA-20 zeromq2: failure. - http://bit.ly/c8H1WC
[15:10] pieterh mato: you asked for review of all code changes
[15:10] mato pieterh: yes...
[15:10] pieterh it's a two-line change, no hurry though, I'm waiting for confirmation from Benjamin that it works
[15:11] mato pieterh: thx for putting it on a branch
[15:11] pieterh does pushing topic branches to the repository work as a way of publishing proposed changes?
[15:11] pieterh i would delete the branch once issue is closed
[15:11] mato pieterh: it does for you since you have commit access
[15:12] pieterh yes, for any committer
[15:12] mato yes, more or less
[15:13] pieterh ok, cause i don't want to create chaos in the git
[15:13] mato pieterh: although given that sustrik has not yet been taught to work with branches, you're probably better off just emailing patches...
[15:13] mato pieterh: note that emailing patches has the added advantage of anyone on the list being able to review your code...
[15:14] pieterh mato: i already emailed the patch to the list this morning
[15:14] mato pieterh: so... ?
[15:14] pieterh so i need closure on my own workflow, can't leave random branches lying around
[15:14] pieterh working in too many places, on too many boxes
[15:15] pieterh and would like to use branches, in any case
[15:15] mato pieterh: sure
[15:15] pieterh ok, great
[15:15] mato pieterh: of course, the canonical way to do this is to create your own fork of zeromq
[15:15] mato pieterh: push to that, etc etc
[15:15] pieterh yes, of course but then it becomes complex for anyone wanting to try work in progress
[15:15] pieterh this way it's all in the single place
[15:16] mato hey, it's a distributed system
[15:16] mato :)
[15:16] mato the point is there is no "single place", really
[15:16] mato anyway, i don't care either way right now
[15:16] pieterh i know, mato... recall it was me who proposed we used git for 0mq in the first place
[15:16] pieterh :-)
[15:16] mato so?
[15:16] pieterh it's only about reducing complexity
[15:17] mato yes, sure
[15:17] mato but you had to announce the work to the mailing list anyway
[15:17] mato (the patch)
[15:17] pieterh yes but...
[15:17] pieterh that ends up using the mailing list as database
[15:17] mato :-)
[15:17] pieterh we know that is suboptimal
[15:18] pieterh "where is patch for change XYZ?"
[15:18] mato sure, whatever, i have no opinion right now
[15:19] mato yes :)
[15:19] pieterh the more the merrier, i say!
[15:19] mato i found it very refreshing to see a "i don't know" reply from you today or last night :-)
[15:19] mato much appreciated !
[15:19] pieterh sincerity is easy, once you learn how to fake it
[15:20] mato anyhow, gotta go to dinner
[15:20] mato cyl
[15:20] pieterh cyl mato
[15:20] guido_g hi all
[15:21] pieterh hi guido_g!
[15:22] guido_g the zmq_deviced mentioned in THE GUIDE (tm), where does it come from?
[15:22] pieterh guido_g: its a work in progress
[15:22] guido_g ahhh ok
[15:22] pieterh i should mention that in the user guide
[15:23] pieterh my intention was to finished the device daemon earlier but it got sidetracked
[15:23] pieterh there is a man page describing how it will work, on master
[15:23] guido_g ok
[15:23] pieterh it's a mix of Jon Dyte's zmqd and the existing devices
[15:24] pieterh but i figured it would be more useful in C, for people who want to reuse the code for their own devices
[15:24] pieterh also i am still 21 years away from learning C++... :-)
[15:24] guido_g that's clear, i was just confused becaue of the missing program
[15:24] pieterh i'll add a note to the User Guide, confusion is not good ...
[15:25] mato pieterh: see, the zmq_deviced you're writing should (even if in progress) be sitting on a nice branch somewhere, whether in your private repo or on zeromq2 is moot right now...
[15:25] mato pieterh: anyhow, me is really gone ...
[15:26] pieterh mato: yes, yes, with an issue etc. will do, thanks
[15:26] mato pieterh: with the manpage too, which should not be on master :-)
[15:26] mato pieterh: just being pedantic ...
[15:26] pieterh ur right
[15:26] pieterh pedantic is perfect
[15:27] pieterh this new process will be really helpful
[15:27] guido_g obviously my first idea that tools like git should make things easier is wrong ,)
[15:39] guido_g hmm... the REP/REP typo seems to have survived the changes...
[15:40] guido_g http://www.zeromq.org/docs:user-guide is current, isn't it?
[15:41] pieterh guido_g: no, it's not current
[15:42] pieterh tools like git and issue trackers are more work superficially
[15:43] pieterh i'll post an update later today but i'm writing offline
[15:43] guido_g no problem
[15:44] travlr hi pieter. maybe issue tracker should be for convenience rather than hard protocol.
[15:45] pieterh travlr: for sure... it's always an effort/benefit tradeoff
[15:45] pieterh using an issue tracker systematically makes it much easier for others to know what's going on and participate
[15:47] pieterh i think there is empirical evidence that projects that don't use issue trackers systematically are less successful
[15:47] pieterh but since we're a community of volunteers, it's always optional
[15:48] travlr yup, this is all true too. I think the (pragmatic) protocol you all decide upon will evently resolve itself upon experimentation.
[15:48] guido_g pieterh: and there is postgresql :)
[15:48] guido_g no issue tracker and such
[15:48] pieterh unfortunately if i have strong opinions on this it's because experience tells me people do not always like the inconvenience of making life easier for others
[15:49] pieterh this goes from how we write code, how we communicate what we're working on, to how we document our processes
[15:49] pieterh its always extra effort upfront
[15:49] guido_g right
[15:49] travlr pieterh: i could not agree more
[15:49] guido_g most code-workers don't think in these categories (/me included)
[15:50] pieterh most programmers (including me) focus on the results
[15:50] pieterh and they need others to remind them of due process
[15:50] guido_g *sigh* how true
[15:50] travlr being a self taught cs guy.. i always think in terms of the guy trying to pick up the torch ;)
[15:50] pieterh i don't think this will emerge organically, it does need deliberate work
[15:51] pieterh no-one likes other peoples' processes
[15:51] pieterh anyhow, next week i'll work through this with Martin and Mato and we'll come to a clear and simple way of working
[15:52] travlr +1
[15:52] pieterh i hope that will make it easier for everyone who tracks the project
[15:52] pieterh travlr: yes, documenting the torch is key
[15:53] travlr yes the torch..lol.
[15:59] sustrik mikko: are you there?
[16:01] sustrik btw: i am not sure that adding stuff that you would like to do into issue tracker is a good idea
[16:01] sustrik i've done that in the past
[16:01] sustrik and the tracker just become polluted with old and abandoned projects
[16:01] pieterh sustrik: more for work in progress rather than ideas
[16:02] pieterh you're right that issue trackers rapidly get polluted otherwise
[16:02] sustrik work in progress is the same thing
[16:02] mikko sustrik: ye
[16:02] sustrik if everyone starts putting his work in progress into bug tracker we'll get a mess
[16:03] sustrik working on something doesn't necessarily mean it'll get merged in
[16:03] sustrik mikko: how was the talk?
[16:03] guido_g isn't that the reason behind categories for tracker entries?
[16:03] pieterh sustrik: i don't disagree but then we need somewhere for people to publish their wips
[16:03] mikko sustrik: http://twitter.com/harrieverveer/status/20818282466
[16:03] travlr re: issue tracker entries could be qualified into categories like "bug | feature request | todo | fixed"
[16:03] pieterh if i want to publish a branch to the git, i need a issue number
[16:04] mikko sustrik: i think that summarises it well
[16:04] sustrik :)
[16:04] guido_g so you *need* an issue for a feature?
[16:04] pieterh if i want my code reviewed, i need to publish to the git (principle of least surprise)
[16:04] sustrik what's BS?
[16:04] mikko sustrik: bullshit
[16:04] pieterh lol
[16:04] sustrik great, looks it was successful
[16:04] sustrik you are in finland?
[16:05] pieterh mikko: did you use slides?
[16:05] mikko sustrik: nope, i am from finland
[16:05] sustrik guido_g: i would say no
[16:05] mikko sustrik: living in london and visiting netherlands at the moment
[16:05] mikko pieterh: yeah
[16:06] mikko pieterh: not very good slides
[16:06] sustrik implementing a feature is your private matter
[16:06] mikko pieterh: did them in the plane on the way here. im gonna do a few iterations on them at some point
[16:06] sustrik pushing it to the trunk is a different thing
[16:06] pieterh i'd be interested in how people responded, what they understood, did not, etc.
[16:07] pieterh sustrik: there is probably a balance between not announcing any wip, and trying to record all of it
[16:07] guido_g sustrik: that's what i was goiung for, if something needs to be commited to the official repo, it has to have an issue attached
[16:08] pieterh and for instance the wip you have now, has no issue, no tracked discussion
[16:08] pieterh right?
[16:08] sustrik should it?
[16:08] sustrik it's my wip
[16:08] pieterh so... :-)
[16:08] pieterh why is it in the main git?
[16:08] sustrik once i'm going to push it to the trunk
[16:08] sustrik that's the time for the discussion
[16:08] pieterh the point of upfront issues is to have discussion upfront as well
[16:08] sustrik because mato put it there
[16:09] sustrik i was developing on my repo originally
[16:09] pieterh yes, because that's where people expect it... :-)
[16:09] pieterh principle of least surprise
[16:09] sustrik i still don't see why anyone should be interested in my wip
[16:09] sustrik i mean while it is not ready
[16:09] pieterh look...
[16:09] pieterh it is a big bugfix
[16:10] sustrik nope, it's a change of semantics
[16:10] pieterh that's a point of view
[16:10] pieterh it fixes the bug that messages are dropped if you exit too early
[16:11] sustrik that's a documented feature afaik
[16:11] pieterh you're kind of justifying working outside of any formal process
[16:11] pieterh which i have no issue with
[16:11] sustrik the process beging once you share
[16:11] pieterh but it makes it harder for others to participate
[16:11] pieterh yes
[16:11] pieterh the process begins once you share
[16:12] sustrik rightr
[16:12] pieterh which is why sharing analysis of the problem starts the process
[16:12] pieterh the earlier you share, the better it works, empirically
[16:12] pieterh i think that's it
[16:13] sustrik so you propose to have feature discussions in issue tracker
[16:13] sustrik rather than mailing list
[16:13] pieterh yes, or somewhere
[16:13] pieterh IMO the mailing list and IRC are excellent for some aspects
[16:13] pieterh and the issue tracker is too limited for real discussions
[16:13] pieterh ...
[16:13] sustrik yes, and feature discussion in transient in its nature
[16:14] pieterh yes
[16:14] pieterh but not so transient it always gets lost
[16:14] sustrik so mailing list seems the best option
[16:14] pieterh email is information, not knowledge
[16:14] pieterh there's a substantive difference
[16:14] pieterh "find all discussion on the problem with devices crashing if you send them bad messages"
[16:14] pieterh -> issue 46
[16:14] sustrik yes, that's a bug
[16:15] pieterh not my point, sorry
[16:15] sustrik i am rather speaking of "random chat on how to implement ipv6 support"
[16:15] pieterh finding knowledge on an issue (bug, misfeature, improvement, api) is crucial to solving it accurately
[16:15] pieterh exactly
[16:15] pieterh where is the ipv6 discussion page?
[16:15] pieterh i was looking for it the other day
[16:15] sustrik no
[16:15] pieterh i could not find it
[16:15] pieterh therefore i did not participate
[16:16] sustrik search mail archives for the random chat on ipv6
[16:16] pieterh i'm not going to spend 2 hours searching old mail discussions
[16:16] pieterh a waste of my time, i have real work to do, as mato likes to say
[16:16] pieterh this is the problem
[16:16] sustrik unless there's a code backing the discussion it is almost by definition random and transient
[16:17] sustrik no point in storing it at a wiki page or so
[16:17] pieterh why are you against recording knowledge in a predictable way?
[16:17] sustrik it's not a knowledge
[16:17] sustrik it's people ranting about features
[16:17] pieterh "why 0MQ does not support IPv6" is knowledge
[16:17] sustrik sure
[16:17] pieterh i know it is
[16:17] sustrik FAQ?
[16:18] sustrik ok, it's kind of pointless
[16:18] sustrik back to coding!
[16:19] travlr why not, as a reference see what tried and true projects are doing for these workflows... like the linux kernel for instance
[16:20] sustrik travlr +1
[16:24] dos000 travlr, what do they do ?
[16:26] travlr dos000_: can't say specifically, but there are plenty of examples of growing/successful large project workflows out there.
[16:27] pieterh linux kernel tracker only tracks bugs
[16:28] pieterh other links projects track bugs, feature requests, patches, e.g. http://sourceforge.net/tracker/?group_id=429
[16:28] travlr i would also mention something like KDE but they are only transitioning to git now
[16:28] pieterh i think sustrik is right that the git issue tracker should hold bugs only
[16:28] travlr and feature requests?
[16:29] pieterh but i also think we need a place to discuss feature requests
[16:29] dos000 pieterh, my point is what do people use beside forums,maillist tocatch these random bits of info
[16:29] pieterh that is not the email list
[16:29] sustrik i was just trying to say that 90% of feature discussion will be trash anyway
[16:29] pieterh sustrik: 90% of all discussion is trash :-)
[16:29] pieterh sturgeon's law
[16:30] dos000 is this irc logged to begin with ?
[16:30] travlr if someone in the community has a feature request, it should be recorded in the tracker... imo
[16:30] pieterh dos000_: probably
[16:30] dos000 i find a lot of info just glancing over here from other people questions
[16:30] sustrik note zmqircd participant
[16:30] sustrik someone is spying on us
[16:30] pieterh travlr: at the least, just to answer "no, because"
[16:30] sustrik hm
[16:30] sustrik not even speaking of CIA-20
[16:31] sustrik in short, there's a log somewhere
[16:31] pieterh i'm willing to put up and manage a secondary tracker for feature discussion
[16:31] pieterh experimentally then
[16:31] dos000 but going to the mailing list is against the "least resistance path" unless i have a specific issue in my head to solve
[16:31] pieterh that will ensure the main issue tracker remains focussed
[16:31] pieterh dos000_: what would seem least surprising to you?
[16:32] sustrik you can try
[16:32] sustrik i just expect the feature tracker would look like this:
[16:32] sustrik "John Smith believes the IPv6 feature is not needed"
[16:32] sustrik "mato thinks that there should be new IPv6 transports (tcp6:// etc.)"
[16:32] sustrik "mato have changed his mind"
[16:33] pieterh yes, this is the kind of discussion you'd expect
[16:33] pieterh hmm
[16:33] pieterh for example, the thread on reliability
[16:33] pieterh that is a good example of a feature discussion
[16:33] pieterh it's a really useful thread
[16:33] sustrik what have you used it for?
[16:33] pieterh to read people's use cases and experience
[16:33] travlr the issue could simply point to the discussion on the mail list
[16:34] sustrik yes, use cases
[16:34] sustrik that's useful
[16:34] pieterh there is no mail list thread
[16:34] pieterh at best, loads of URIs into the archives
[16:34] pieterh ugh :-(
[16:34] pieterh use cases, sample code, etc. etc.
[16:35] pieterh look... what I'll do is start a section on the wiki for feature discussion
[16:35] pieterh i'll move the reliability page there to seed it
[16:35] sustrik sure, do so
[16:35] sustrik we'll see how it works
[16:35] pieterh sure
[16:35] travlr +1
[16:35] pieterh email is great for transient stuff
[16:35] pieterh but it is sand on the beach
[16:35] pieterh come back tomorrow, it's all gone
[16:36] pieterh ok
[16:36] pieterh sustrik: what i'll also do is when people discuss features on the list, try to capture this as new discussion threads on the wiki
[16:37] dos000 pieterh, what you mean ? i have not found a good medium to capture feature discussion . beside mailing lists. thats what i mean
[16:37] travlr pieterh: that makes sense to this mere and humble mortal
[16:37] sustrik i have no real opinion
[16:37] pieterh dos000_: look at http://www.zeromq.org/blog:requirements-for-reliability
[16:37] pieterh forget my initial text, it's rubbish, but the discussion that follows is pure gold
[16:37] sustrik the fact is that at some point someone arrives with an implementation of the feature
[16:38] dos000 pieterh, nice
[16:38] sustrik at that point we either merge it or not
[16:38] pieterh sustrik: yes, that's always an option
[16:38] pieterh and that certainly represents the mindset of those looking after the repository
[16:38] pieterh changeset -> review -> merge
[16:39] sustrik yes. but the feature discussion is kind of orphaned
[16:39] dos000 pieterh, this is based on what ? is the site on full wiki ?
[16:39] pieterh dos000_: the site is using wikidot.com
[16:39] travlr sustrik: you have a good point.. thinking also about long term maintainence
[16:39] travlr kiss
[16:39] travlr k.i.s.s
[16:40] pieterh yes, it's orphaned, i agree
[16:40] sustrik what i suspect is that we'll end up with feature discussion completely unrelated to actual code
[16:40] pieterh sustrik: feature discussion should really focus on external aspects anyhow
[16:40] pieterh such as use cases, and apis
[16:40] sustrik right
[16:41] sustrik use case should be captured
[16:41] pieterh i like the notion that how that discussion works is orthogonal to how stuff gets into master
[16:41] dos000 pieterh, the search actually works !
[16:41] pieterh dos000_: yes :-)
[16:42] pieterh it is actually a very powerful wiki engine
[16:42] travlr pieterh: how about a tag cloud too on the wiki
[16:43] pieterh its possible but tag clouds tend to be pretty but useless
[16:43] pieterh search and categorization work better, by experience
[16:43] travlr sure
[16:44] pieterh if you want to learn about wikidot there is a very active community at http://community.wikidot.com
[16:44] pieterh many template site types at http://irongiant.wikidot.com
[16:44] pieterh here ends the commercial
[16:44] travlr the problem with wikidot info is the needed information is EVERYWHERE
[16:45] travlr can't find specific info easily
[16:45] pieterh :-) http://doc.wikidot.com
[16:45] dos000 pieterh, i will keep wikidot in my radar. we might need it inhouse
[16:46] pieterh dos000_: disclaimer, i was running wikidot.com last year as we built it up, so I'm not neutral
[16:46] travlr pieterh: i think this is the first time i have seen this page
[16:47] travlr _all_ info needs to be gathered into one location
[16:47] dos000 which template is 0mq using ?
[16:47] pieterh dos000_: customisable CSS, see http://themes.wikidot.com
[16:47] pieterh travlr: there is much too much information for one place
[16:47] travlr links then
[16:47] travlr something
[16:48] pieterh http://dashboard.wikidot.com/helmuti-pdorf:_start
[16:48] travlr i have a hard time finding what i need
[16:49] pieterh travlr: usually, just ask the community and they'll point you to it... it is a huge project with no accurate map
[16:49] travlr i would rather be able to find it myself
[16:50] pieterh i did make an index, it is no longer published but: http://blog.wikidot.com/main:index
[16:51] travlr pieterh: with all do respect to the wikidot folks.. a better job needs to be done for access to this information
[16:52] pieterh travlr: for sure... wikidot is messy, its a good example of what happens when things just grow organically
[16:53] pieterh zeromq.org systematically tries to be a map of every available resource
[16:54] travlr pieterh: i agree and understand... but someone like me just does not have time to find all the ins and outs as wikidot appears to be a wonderful platform
[16:55] pieterh travlr: mostly with wikidot, it kind of works because each time you get more sophisticated you discover a new set of resources for you
[16:56] travlr cool. i have to spend more time with it.. i just get frustrated with any project, that does not provide the answer to a question i may have, easily
[16:57] travlr any project... thats one of my pet peeves
[16:57] pieterh there is a global search option, from the www.wikidot.com page
[16:57] travlr thats for all wikidot websites.. isn't it?
[16:57] travlr not how to use wikidot
[16:57] pieterh all websites you can access
[16:57] pieterh yes, well organized knowledge is kind of critical
[16:58] feroz Hello.
[16:58] pieterh :-) hi
[16:58] travlr in my dreams, i'd like to organize an effort of a doc template for all of the oss community to adopt
[16:59] feroz Wondering if anyone has made any benchmark on the performance of pubsub
[17:00] pieterh travel: one of the joys and strengths of the oss community is we will never agree totally on anything
[17:00] travlr yup... its just a dream
[17:00] pieterh feroz_: there is a performance test program for pubsub included with 0mq
[17:00] pieterh and there are afair whitepapers that report test results
[17:01] pieterh but the best is to make your own tests
[17:02] dos000 i was looking at rackspace last night for running tests on large sacale.
[17:02] pieterh see perf subdirectory, local_thr.cpp and remote_thr.cpp are a pubsub test case for throughput
[17:02] dos000 a 16g machine will cost $.98 for one hour
[17:02] feroz pieterh: okay, i'll lookup for the whitepapers. I just wanted to read what has already been done before doing my own. thanks
[17:03] dos000 thats a full machine btw with16g of ram
[17:04] dos000 and if you are running tests there is no extra money for internall traffic nor for storage
[17:05] pieterh dos000_: for 0MQ, 20MB of ram will do... the real issues are network infrastructure
[17:05] dos000 pieterh, i am running cassadra behind 0mq.
[17:06] dos000 but you are right i cant decide about the switches and routers between machines
[17:07] dos000 i cant go and buy those stuff on my own .. the idea is to test this on the cloud. since it will run on the cloud
[17:08] dos000 the point is getting a machine is soo cheap that you try this with large scale
[17:09] dos000 i am worried about the context switching if you are running the code java binding for zmq
[17:11] sustrik dos000_: if you are going to run on a VM, the context switching is out of your control
[17:12] dos000 i meant between c and java
[17:12] sustrik afaik there's no context switching there
[17:12] sustrik c functions are called in context of the java thread
[17:12] dos000 not in the kernel sense yes
[17:13] dos000 when traffic comes it comes to zmq then i have to read the bytes in java
[17:14] dos000 the protocol handler is done in java
[17:14] sustrik you are worried about the cost of copy?
[17:15] dos000 i think i can pass a pointer . but the fact that the c code has to call jvm byte code has to incur some delay i assume
[17:15] sustrik maybe
[17:16] sustrik but JNI looks pretty straighjtforward
[17:16] dos000 i would like to be proven (very) wrong
[17:16] sustrik try checking on some Java forum
[17:16] dos000 i will actually do the actual tests at some time
[17:17] dos000 this will give clear answer as to if i need to solve this or a non issue
[17:17] sustrik ack
[17:31] dos000 where is the irc deamon loggin this channel ?
[17:32] dos000 i need to go back to discussions i had a while ago
[17:34] travlr dos000_: re: irc log... i'm doing it soon.. i got real busy lately.
[17:37] travlr if anyone has an archive of the logs on their computer, let me know because i have to fill a couple of holes in my collection.
[18:58] sustrik dos000_: what are you asking about?
[18:58] sustrik it's not clear
[18:59] dos000 someone told me here that if i use zmq_poll i have to use it for all sockets.
[18:59] dos000 which was weird
[19:01] sustrik it's just about combining standard and 0mq sockets
[19:01] sustrik the only way to poll for both in a single go is zmq_poll
[19:02] sustrik however, you are free to use whatever polling mechanism if all you are interested in are standard sockets
[19:03] dos000 i have this : incoming traffic --> dispatcher --> 0mq subscribers
[19:03] dos000 i want to use poll on the left of the dispatcher .. not on the right
[19:03] sustrik what's on the left?
[19:03] sustrik standard sockets?
[19:03] sustrik if so, use any mechanism you like
[19:03] dos000 nop zmq subscribers
[19:04] sustrik ?
[19:04] dos000 so the dispatcher is publishing to subscribers
[19:04] sustrik via 0mq i suppose
[19:04] sustrik ok
[19:04] dos000 yes indeed
[19:04] sustrik what's on the left side?
[19:05] dos000 incoming network traffic. disoatcher is listener
[19:05] sustrik standard sockets, right?
[19:05] dos000 yes. i am using zmq_poll on the left
[19:06] sustrik if 0mq sockets are not involved
[19:06] dos000 just to read traffic off the network card
[19:06] sustrik then you can use select/pol etc.
[19:06] sustrik whatever you like
[19:06] dos000 ah!
[19:07] sustrik use zmq_poll only if there are 0mq sockets you need to poll for
[19:07] dos000 so on the right side if i want to use zmq_poll then i have to do so for all zmq socket in the dispatcher ?
[19:08] sustrik ?
[19:08] sustrik it's easy
[19:08] Chris I want to use zeromq to talk to Nasdaq and receive NLS protocol messages.
[19:08] sustrik is there a 0mq socket in the pollset?
[19:08] sustrik if so, use zmq_poll
[19:08] sustrik otherwise use whatever you like
[19:08] dos000 ok i got that.
[19:08] dos000 thanks a lot
[19:09] Chris The NLS spec uses SoupTCP which is an easy protocol that sits on top of TCP
[19:09] sustrik i am afraid nasdaq doesn't use 0mq
[19:09] sustrik :)
[19:10] Chris That's what I wonder whether this is the right forum. Can I build a protocol with sockets using zeromq?
[19:10] sustrik 0mq has it's own message framing
[19:10] sustrik basically size+data
[19:11] sustrik presumably, nasdaq wouldn't understand that
[19:11] sustrik you'll have to build a bridge
[19:11] sustrik that would speak SoupTCP on one side and 0mq on another
[19:11] Chris No, I need to build a client that understands SoupTCP and am searching for a network API to help me.
[19:12] sustrik no SoupTCP support in 0mq
[19:12] sustrik sorry
[19:12] Chris I am actively looking at Boost ASIO and am also curios about zeromq
[19:12] CIA-20 zeromq2: 03Pieter Hintjens 07master * rde01737 10/ (doc/Makefile.am doc/zmq.txt doc/zmq_deviced.txt): Removed wip zmq_deviced from master - http://bit.ly/aZn9NX
[19:12] Chris How hard is it to build SoupTCP into zeromq?
[19:13] sustrik dunno, depends on how complex SoupTCP is
[19:13] sustrik you would have to write a parser etc.
[19:13] Chris It's a simple line-oriented protocol that is mostly 1-way (Nasdaq -> user) whereby stock pricing information is sent to the user.
[19:14] Chris Upstream heartbests are needed about once-per-second. The protocol needs to support 15mbps streaming with burst of 30-35mbps.
[19:15] sustrik you would have to implement whole the SoupTCP stuff
[19:15] sustrik it's up to you
[19:15] Chris Yes, they give the spec and it is up to me to build the sofftware to interact with the system.
[19:15] sustrik :)
[19:15] sustrik try to find out an existing implementation first
[19:15] sustrik you may save a lot of work that way
[19:16] Chris Well, I saw that iMatrix did stuff in the financial world so that is why I thought maybe the zeromq approach would work for this application too.
[19:16] dos000 Chris_____, do you have a parser now ? and in which lang ?
[19:17] Chris No parser but I plan to use C++
[19:17] dos000 there is no open implementation already out there ?
[19:17] guido_g http://cargocult.ca/ccc_doc/classCargoCult_1_1SoupTCP.html
[19:18] Chris I did Google but it seems like only proprietary stuff which includes other streams.
[19:19] Chris Dooo.
[19:19] Chris Thanks Guido_g.
[19:19] guido_g this google thing is incredible...
[19:19] dos000 cargocult is canadian (from the domain) thats gotta be good !
[19:20] pieterh Chris_____: what I'd do to start is make a SoupTCP->0MQ bridge
[19:21] guido_g souptcp includes authentication, so it seems not to fit into the current Ømq model
[19:21] pieterh read SoupTCP at one side (with authentication etc.) and push out the other side
[19:21] Chris That's correct; part of the direction to Nasdaq includes a socket with login/password information and a response.
[19:21] pieterh I really wish I'd called our business "iMatrix" but that was a small graphics firm, in 1996
[19:23] sjampoo pieterh, seriously i've read iMatix zillions and it wasn't until I tried to google it that I noticed there was no 'r' in there.
[19:23] sjampoo really weird
[19:23] pieterh Chris_____: what message sizes?
[19:23] pieterh sjampoo: this is how we parse words
[19:23] pieterh :-)
[19:23] sjampoo i knw thts wh u cn undrstnd me
[19:23] pieterh wtf?
[19:23] sjampoo hehe
[19:23] dos000 howabout s : incoming traffic --> dispatcher publish raw packets to zmq req/reply --> distributed req/reply to parse suoptcp in any lang
[19:24] Chris Sorry, I see now that the company is iMatix. :)
[19:24] Chris My typeface is small but my face is now red.
[19:25] pieterh it's a common mistake...
[19:26] pieterh Chris_____: do you need a design for a bridge?
[19:27] Chris I do need knowledge to make the solution; if I can leverage off of other designs, that'd be great.
[19:27] sjampoo I think he would prefer an implementation
[19:28] Chris Is this right place to solicit jobs? If not, accept my apologies and I won't bring it up again.
[19:29] Chris Anyway, I will study Guido's URL and continue to work with zeromq. Thanks for the discussion thus far.
[19:31] Chris Message size in SoupTCP are around 100 bytes usually.
[19:31] pieterh Chris____: here's a design: http://www.zeromq.org/docs:how-to-bridge-to-souptcp
[19:32] pieterh When you say "solicit jobs" what do you mean precisely?
[19:32] dos000 pieterh, wow!
[19:32] Chris It is basically a realtime connection to obtain last price.
[19:32] Chris I will try to do it myself but could pay someone(s) for help.
[19:32] pieterh Chris:
[19:33] dos000 pieterh, what do you use for the images ?
[19:33] pieterh contact me and we'll make you an offer
[19:33] pieterh iMatix funds the project this way
[19:33] pieterh dos000_: ditaa
[19:33] pieterh it is really sweet
[19:34] pieterh Use the contact link on http://www.zeromq.org
[19:34] Chris You mean go to iMatix website and fill out the contact info?
[19:35] pieterh Chris____: right
[19:35] Chris OK; thanks. All for now.
[19:35] pieterh so bursts of 350,000 messages/second, that is really not a problem
[19:36] pieterh a bridge will add about 15-30usec of latency
[19:36] Chris Seems like it could work.
[19:36] dos000 pieterh, if this is pub/sub how do you channel responses bacck from subscribers ?
[19:37] pieterh dos000_: you don't... the bridge does the heartbeating and the 0MQ doesn't use it
[19:37] dos000 pieterh, unless this is only a read only
[19:37] pieterh it's market data, so strictly one-way
[19:37] dos000 ah!
[19:37] dos000 i assumed it was two way .. dumb me
[19:37] pieterh the key is low latency, so probably you'd use a multicast bus
[19:38] dos000 ah!
[19:38] pieterh this is actually the original main use case for 0MQ
[19:39] dos000 if you had tow way what would you do ?
[19:39] pieterh tow way? sorry... can't parse...
[19:39] pieterh oh no, i'm not a master in 0MQ, only in more general aspects of messaging
[19:39] dos000 pieterh, i mean if the suptcp protocol required a response for exemple
[19:40] pieterh well, it doesn't :-)
[19:40] dos000 my protocol does
[19:40] pieterh but if it did, you'd use a separate pattern to collect responses
[19:40] pieterh it has been observed by people who know 0MQ better than me that...
[19:40] dos000 ok ..
[19:40] pieterh - break down your use cases into individual patterns
[19:41] pieterh - solve each pattern using the appropriate 0MQ style
[19:41] pieterh - creates harmony and scalability
[19:41] pieterh so whereas you might mix control and data on one tcp socket
[19:41] pieterh you might well use two 0MQ sockets
[19:41] dos000 ah huh ..
[19:42] pieterh the reason is that policies for different patterns are different
[19:42] pieterh and when you get the policy *right* you can scale it easily
[19:42] dos000 ok ..
[19:42] pieterh if you mix policies in one connection it can't be scaled (or only with great cost)
[19:43] dos000 mmmmm digesting
[19:43] pieterh policy includes things like "do we send replies and if so, how do we route them", and "do we sent asynchronously or synchronously"
[19:43] dos000 ok ..
[19:43] pieterh i can give a concrete example
[19:43] pieterh which illustrates this nicely IMO
[19:43] dos000 please ..
[19:43] pieterh in OpenAMQ we had a performance problem
[19:44] pieterh where we could only pump 100,000 messages/second through a connection
[19:44] pieterh less, actually, 60k or so
[19:44] pieterh AMQP connections have a mix of policies
[19:44] pieterh request-reply commands plus streaming data
[19:44] pieterh but all mixed up
[19:44] pieterh we separated those into two connections
[19:44] pieterh one for commands, one for streaming data
[19:45] pieterh we were able to get 300,000 message/second finally
[19:45] pieterh plus we moved queues off the broker, making it a lot more stable
[19:46] pieterh http://wiki.amqp.org/spec:6
[19:46] pieterh it was significantly simpler once we moved to one policy per connection
[19:46] pieterh and simpler meant we could optimize and make it faster
[19:47] pieterh does this help?
[19:47] dos000 ok ..
[19:47] dos000 so far so good.
[19:47] dos000 so on the way back i would put data on another queue
[19:48] pieterh send it via another socket
[19:48] dos000 the question is when it comes to the bridge then the bridge has to use a hash to figure where is the object corelation
[19:49] dos000 i mean to figure which socket to send it back
[19:49] pieterh yes, that's probably a good way
[19:49] dos000 nice
[19:49] pieterh are you prototyping it?
[19:49] pieterh in python or something?
[19:50] dos000 not yet .. this stuff is still in head
[19:50] dos000 its a mix of java and c/c++
[19:50] dos000 bridge is definitely c/++
[19:50] dos000 subscribers are java (for now)
[19:52] dos000 very interesting indeed
[19:54] dos000 pieterh, the only problem is the has will have to be protected for multithreaded use
[19:54] dos000 the hash table
[19:54] pieterh why multithreaded?
[19:55] pieterh I'd use a single thread for routing
[19:55] dos000 i have say 300 threads wating for messages from subscribers
[19:55] dos000 ok
[19:55] pieterh furthermore, a single thread can collect replies from 300 subscribers
[19:55] pieterh this is what 0MQ does for you, handle the connections transparently
[19:56] dos000 ok that was my next question ... btw i apprciate all this time you are spending here
[19:56] pieterh shoot
[19:57] dos000 in the zmq_poll how many sockets until you need a second thread in zmq_poll
[19:57] pieterh no limit as such
[19:57] dos000 ah
[19:57] pieterh in fact you would never have multiple threads polling the same sockets
[19:58] dos000 so one thread doing zmq_poll is enough
[19:58] pieterh yes
[19:58] dos000 cool
[19:58] pieterh you can do a *lot* of work in one thread
[19:58] dos000 thanks a lot pieterh
[19:59] pieterh i'm only three lessons ahead of you, dos000_ :-)
[20:08] dos000 pieterh, do you have the src for that picture http://www.zeromq.org/docs:how-to-bridge-to-souptcp ?
[20:09] pieterh I'll add it to the page along with the perl script that does the transformation
[20:09] dos000 cool
[20:09] pieterh you need to get the ditaa jar
[20:09] dos000 yes i am planning to keep this jar with my projects
[20:14] pieterh dos000_: i've added the links to that page
[20:14] dos000 thansk a lot
[20:14] pieterh np
[22:14] CIA-20 jzmq: 03Sam Aarons 07master * r30712eb 10/ perf/local_thr.java : Update perf test to reflect changes to setsockopt. - http://bit.ly/cdi0QU

Web site design and content is copyright © 2010-2011 Peter Alexander, ProDataLab. Site content licensed under the Creative Commons Attribution-Share Alike 3.0 License. ØMQ is copyright © Copyright © 2007-2011 iMatix Corporation and Contributors. Contact us for professional support. ØMQ is free software licensed under the LGPL. ØMQ, ZeroMQ, and 0MQ are trademarks of iMatix Corporation. Terms of Use ? Privacy Policy