Thursday March 24, 2011

[Time] NameMessage
[00:05] mikko pieterh: did you push the latest make dist fixes?
[00:12] Guthur jeckstein, any success getting a minimal test case?
[00:14] jeckstein no, its going to take me a little while to do that, i just compiled and loaded the newest clrzmq2 (2.1.0) and am testing it on my application now
[00:18] Guthur ok, cool
[00:37] jaten hey guys, I'm getting a core dump in /lib/ in the head of the git repos for zeromq, running the first server.cpp example in the Guide. This is on ubuntu 10.10 with /lib/ from libc 2.12.1-0ubuntu10.2. It doesn't happen with with the earlier 10.04 with libc 2.11.1-0ubuntu7.8 or 7.7, which has /lib/ version 2.11.1. Is this a known issue? If not where would I post for assistance/bug report?
[00:43] Guthur jaten, you should probably raise a ticket on github, and maybe fire a description on to the mailing list
[01:51] jaten Guthur: okay, thanks. I'll do that.
[06:16] ferrouswheel is there any support for message priorities or QoS? saw various discussions on the mailing list, but not sure if it eventuated into anything?
[06:26] sustrik ad message priorities: no, you have to use different message feeds to get different priorities
[06:26] sustrik ad QoS: it can be set on network level
[06:27] sustrik
[06:30] ferrouswheel sustrik, thanks - that's what i suspected
[06:39] ferrouswheel i want to use zeromq for an (initially) inprocess rpc, but would like to decrease the priority of clients that make more requests. I can see various ways to do this, but am curious if a particular design makes more logical sense than others when using zeromq...
[06:46] sustrik ferrouswheel: create several REP sockets, one for each priority
[06:46] sustrik read requests from high priority sockets first
[06:47] sustrik alternatively, you can add priority scheduling to 0mq core directly, which is something that's on the roadmap already
[06:50] ferrouswheel sustrik, okay - the async server (fig 47) in the guide also seems appropriate... deciding on which worker to send a client to, depending on the request rate
[06:50] ferrouswheel but i think i'll leave 0mq core modifications until i've actually used and understand 0mq better ;-)
[06:50] sustrik ok
[08:28] ianbarber fully confused by git. i pulled in changes from the upstream, but it seems to have committed it as a commit, so now if I do a pull req it includes those
[08:28] ianbarber bunch of zguide examples and stuff
[08:42] Steve-o hence it's called "git"
[08:42] Steve-o I think it's git-rebase for the equivalent of svn update
[08:43] ianbarber ah cool, will have a look
[09:55] private_meta I've got a tcp connection, my client is stopping, but it won't quit the program, it hangs at "return" in main in mailbox_t::recv(), any idea why?
[09:56] private_meta It hangs there at the termination of the context, although the connection should have been set not to linger (zmq_linger with a value of 0)
[10:01] pieterh private_meta: you have a socket open somewhere
[10:02] pieterh you have to close every single socket and you have to first set linger to zero on any socket which may still be doing some work (sending, or connecting)
[10:03] pieterh this is the rather painful way 2.1 works but it can be fixed in the language binding
[10:03] private_meta pieterh: linger is on zero on EVERY socket that is open, and how can I close the sockets in C++ apart from calling their destructor (which I'm already doing)?
[10:04] pieterh I see two possibilities... either you have forgotten a socket somewhere, or the C++ binding is buggy
[10:05] private_meta let me try something
[10:05] pieterh private_meta: I've just checked the C++ binding code and there is an explicit close method
[10:06] pieterh which the destructor just wraps
[10:06] private_meta Ah ok... the socket was a pointer in there, and destructor of a pointer isn't called implicitly
[10:06] pieterh right
[10:06] private_meta only if it's a non-pointer object (which I use normally)
[10:07] pieterh in zapi, I'm delingering/closing sockets automatically when the context is destroyed
[10:07] pieterh it makes life so much simpler
[10:08] private_meta zapi?
[10:08] pieterh the new high-level C API
[10:08] pieterh
[10:09] private_meta Well, if the socket object were destroyed that would have been the case, but it wasn't
[10:10] pieterh Sure, you still need to go set linger to zero on sockets
[10:11] private_meta I did some other stuff for a week so I'm still at translating the guide examples
[10:11] pieterh into C++?
[10:11] private_meta yeah
[10:11] pieterh was it you who asked about modifying the C++ API?
[10:12] private_meta Well, I asked about using pointers in the C++ binding
[10:12] private_meta and the zmq::version
[10:12] pieterh yeah, and that led to thinking about splitting the C++ binding into its own project
[10:12] private_meta Yes
[10:13] private_meta As far as I remember, it did
[10:13] pieterh you already translated zhelpers, zmsg, etc. right?
[10:13] private_meta Yes
[10:13] pieterh so... I have a small suggestion
[10:13] pieterh let's make zapi++
[10:13] private_meta I didN't get to continue until now, so I'm at ppworker/ppqueue now
[10:13] pieterh zapi basically refactors the zhelpers/zmsg APIs into something neater
[10:15] private_meta Sounds ok, but I'd recommend not using pointers in function parameters
[10:15] pieterh indeed
[10:16] pieterh so the thing here is that I'm going to migrate the C examples onto zapi
[10:16] pieterh maybe not all, but the more complex ones, in any case
[10:17] pieterh since you know the zhelpers/zmsg API well, would you take a look at zapi?
[10:18] private_meta sure
[10:18] pieterh it'd be really useful to get your design of a C++ binding at the same level
[10:18] private_meta but right now I need to fix this (somewhat) small ppworker problem I have, it doesn't get a connection
[10:19] pieterh if you need a hand, shout, I'm happy to help debug code
[10:19] pieterh paranoid pirates :-)
[10:20] private_meta Apparently you dislike curly brackets in example code
[10:20] pieterh I use them only where needed
[10:20] pieterh code minimalism is kind of a style with me
[10:20] private_meta Every programming teacher tells you to not do that ;)
[10:20] pieterh that's why they're teachers
[10:20] pieterh not programmers
[10:20] private_meta It destroys readability
[10:21] pieterh there's only one way to learn how to program
[10:21] pieterh to quote Zed Shaw, by "programming, motherfucker"
[10:21] private_meta Oh, I tend to agree with them, and many programming teachers we have here had huge programming projects already
[10:21] pieterh so you mean for example an if statement with only one line, no braces?
[10:22] private_meta You add one line to an if statement, tend to forget you didn't have brackets
[10:22] pieterh sure
[10:23] pieterh then this is one of my bad habits, removing braces when they're not needed
[10:23] private_meta Code shouldn't be hard to read, especially if adding brackets doesn't change the compiled results
[10:23] pieterh readability comes only from consistency, adding or removing brackets doesn't affect it IMO
[10:23] pieterh however, if you want, go through all my examples and add brackets
[10:23] private_meta Consistency also means you use brackets, so use them everywhere :)
[10:23] pieterh my only rule is 100% consistency, really
[10:24] pieterh I consistently don't use them when I can avoid them
[10:24] pieterh it creates extra lines that do nothing, grouping a single line is pointless IMO
[10:24] private_meta sure, it seems pointless if you like minimalism
[10:24] pieterh indeed
[10:24] pieterh which I really do
[10:25] pieterh functional minimalism, of course, so using clear names, blank space, etc.
[10:25] pieterh but it is probably just a bad habit... :-)
[10:25] private_meta I mean, using "cond ? statem1 : statem2" would be minimalistic
[10:25] pieterh yes, but unreadable in practice
[10:25] private_meta still you should only do it in very special cases, 'cause mostly it's unreadable
[10:26] pieterh I do that in some cases when it's obvious
[10:26] private_meta yes
[10:26] pieterh the test over time is whether you can easily reuse code
[10:26] pieterh I was able to put together 3K lines of code in zapi, fully tested and documented, in 1 day
[10:26] private_meta by the way, is it your style to put the opening bracket on functions in the next line, but the opening brackt of ifs and elses on the same?
[10:27] pieterh yes, this is historic, we reused the GNU style for functions but our own style for statements
[10:27] private_meta yeah, you used that argument already with a simple text editor and printf debugging, but that still didn't make me use that
[10:27] pieterh I debug zapi using printfs and valgrind
[10:27] pieterh it's extraordinarily effective
[10:28] private_meta not for me :)
[10:28] pieterh I'm actually shocked at being able to write masses of C code and then hammer it into shape with brute force, and it works as designed
[10:29] pieterh trick is to start with the API, and tests, and then implement & test small pieces at a time
[10:29] pieterh presumably C++ eliminates a lot of the pointer errors we make in C
[10:30] pieterh anyhow, if zapi++ takes off, you can impose your style on that :-)
[10:30] private_meta hmm
[10:31] private_meta I've got a weird error in ppqueue that doesn't let lpclient connect
[10:31] pieterh can you paste the code somewhere?
[10:31] private_meta sec
[10:34] private_meta Well, I don't quite see where the error is exactly... If I use the c version of ppqueue and ppworker, lpclient works fine
[10:34] private_meta
[10:34] pieterh you can put multiple files into one gist BTW
[10:35] private_meta files, yeah, but I wasn't logged in at my virtual machine in gist
[10:35] private_meta so I didn't have my files here
[10:35] pieterh so ppqueue doesn't poll the client at all, unless there are workers, that's the first thing
[10:36] pieterh code looks nice, very neat...
[10:37] private_meta My test setup is starting ppqueue, 1 ppworker, 1 lpclient
[10:37] pieterh have you already traced activity by putting in msg.dump() all over the place?
[10:37] private_meta ppworker uses heartbeats
[10:37] private_meta but lpclient can't connect
[10:37] pieterh 'can't connect' means 'does not get a reply within X seconds'
[10:37] private_meta sec
[10:37] pieterh so you need to trace activity in the queue
[10:38] private_meta I: connecting to server...
[10:38] private_meta I: connecting to server...
[10:38] private_meta W: no response from server, retrying...
[10:38] private_meta I: connecting to server...
[10:38] pieterh sure
[10:38] pieterh that can mean anything
[10:38] pieterh it's not really a connection failure
[10:38] Guthur that could be seem behaviour I experience with teh default timeout
[10:38] Guthur It was too aggressive in it's timeout
[10:39] Guthur relaxing the intervals helped
[10:39] pieterh Guthur: good memory...!
[10:39] private_meta if I'm using ppworker and ppqueue in the c version, lpclient doesn't receive timeouts or anything
[10:39] pieterh I fixed this in the C code, hang on, let me check...
[10:39] Guthur pieterh: it had me scratching my head long enough that I have deep grove to remind me, hehe
[10:40] Guthur groove*
[10:40] private_meta by the way, using the c++ client takes way more time than the c client
[10:41] pieterh Guthur: it was in the ppworker when it simulated a CPU overload, that was always more than the client's full timeout
[10:41] private_meta meaning, using c queue and worker normally, then alternating between lpclient for c and c++, the c version is way faster
[10:41] pieterh private_meta: weird, that's not expected
[10:42] pieterh sounds like a timer problem of some kind
[10:42] pieterh there should be no measurable (let alone visible) difference between the two
[10:43] pieterh private_meta: so please add msg.dump on everything the broker recvs and sends, that's step 1
[10:45] private_meta k, receiving is always done in the zmsg constructor, so I'll add the dump there. should I paste the output of ppqueue?
[10:45] pieterh sounds good
[10:45] pieterh that will show us what the client is sending
[10:46] private_meta Should I also dump messages in worker and client or just queue for now?
[10:46] pieterh then we also want to see what ppqueue is sending
[10:46] pieterh start with queue
[10:46] private_meta k
[10:46] pieterh presumably there's a problem between queue and worker
[10:46] pieterh so next, worker
[10:51] private_meta argh
[10:51] private_meta how do I redirect output AND stderror to file?
[10:51] pieterh myprog > err.log 2>&1
[10:52] private_meta "Aborted"
[10:53] private_meta ah ok now
[10:53] private_meta argh
[10:54] pieterh sounds positive... :-)
[10:54] pieterh or else you're at the dentist... "open wider, please!"
[10:54] ianbarber pieterh: sent a pull req for the zguide, noticed that it was notifying sustrik though, which seems probably incorrect
[10:55] private_meta no, that didn't redirect everything
[10:55] pieterh ianbarber: weird, but presumably he's a watcher of that repo?
[10:55] private_meta ok, screw it, I'll just copy from console if that doesn't work
[10:57] private_meta pieterh:
[10:57] private_meta pieterh: that's the log from starting ppworker till disconnecting (killing) ppworker
[10:58] private_meta in the middle, I started lpclient and it terminated by itself
[10:58] pieterh ianbarber: got them, thanks!
[10:58] pieterh private_meta: ok, reading that...
[10:59] pieterh private_meta: I assume it should say 'after recv' not 'before recv'...
[10:59] private_meta ah yes
[10:59] pieterh so you have a bug in the worker
[10:59] pieterh try running your queue, your client, my worker
[11:00] pieterh I'll let you figure out what the error is, it's clear from the trace
[11:00] pieterh :-)
[11:00] private_meta Doesn't work
[11:00] private_meta same error
[11:01] pieterh that's unpossible! :-)
[11:01] pieterh ok, try my worker, my client, your queue...
[11:02] pieterh hang on, that's just the same, sorry...
[11:02] pieterh your worker, my queue, your client...
[11:03] pieterh the error shows on lines 32-41
[11:03] private_meta should be the same, let me check (i think i tried that already)
[11:03] private_meta No, doesn't work
[11:04] private_meta sec, gotta fetch a delivery, bb in 5 minutes or so
[11:04] pieterh then you have a problem in BOTH your worker and your queue, perhaps because it's in the msg class
[11:04] pieterh cyl, then
[11:08] private_meta i hope this time the tablet pc is alright
[11:08] private_meta pieterh: ok, then what might be the problem in the zmsg class? any idea?
[11:08] pieterh private_meta: look at lines 32-41 of your trace
[11:08] pieterh the worker sends back (or the queue wrongly receives) the worker address twice
[11:09] pieterh when it should be worker address, then original client address
[11:09] pieterh thus the reply to the client gets sent back to the worker instead
[11:09] private_meta hmm
[11:09] pieterh you see the wrong addresses, right?
[11:10] private_meta yes
[11:10] pieterh this is kind of standard when using envelopes, they are tricky to get right
[11:11] pieterh you can now add printfs to msg where it's recving and sending frames
[11:11] private_meta So this might be a problem the selftest didn't catch?
[11:11] pieterh for sure
[11:12] pieterh hmm, there is a unit test for this but it's not double-checking every frame
[11:12] pieterh could improve that
[11:13] private_meta ah goddammit, i need to fetch a usb dvd drive again...
[11:13] pieterh :-)
[11:13] private_meta well, hp guys didn't think it would be a good idea to install a 64bit operating system on their latest notebooks
[11:13] pieterh 32 bits should be enough for anyone
[11:13] pieterh maybe it's a subtle hint to install Ubuntu...
[11:14] private_meta No, I can't use Linux in everyday life, creates too many problems
[11:14] private_meta (Murphies Law and everything)
[11:14] pieterh my sympathies...
[11:15] private_meta Everytime I tried to use something on Linux for personal use, it failed in a way that wasn't fixable
[11:15] pieterh weird, I've been using Linux for 10+ years and it's irreplaceable for me
[11:15] private_meta I've been using Linux for that long as well, but I can't use it as a primary system
[11:16] private_meta apart from that, as a gamer I need Windows anyway
[11:16] pieterh 10+ years for primary personal use, since early 1990's for work
[11:16] pieterh ah, games
[11:16] private_meta Well, some like it, others don't, I tend to be someone fond of games :)
[11:17] private_meta But it's not like playing games is the only reason ;)
[11:17] pieterh are any game engines using 0MQ yet, I wonder...
[11:18] pieterh "shoots 4M bullets per second!!!"
[11:27] mikko- mato
[11:27] mikko- there?
[11:27] mikko- pieterh: you remember ben from zeromq london meetup?
[11:27] pieterh sure
[11:27] mikko- he works in gaming industry
[11:28] pieterh right, he was "putting 0MQ everywhere"
[11:28] mikko- you could pry from him on which games he works on
[11:28] pieterh nah, I don't really care :-)
[11:28] ianbarber his latest, total war shogun 2 is out now! i'm sure he'd support your purchasing :)
[11:29] pieterh I used to write games but don't play them
[11:29] Guthur I really liked the total war series
[11:29] Guthur don't game much anymore though
[11:30] ianbarber shogun 2 is actually quite a lot of fun. i'm all mac everywhere now though, and they're windows only though
[11:30] Guthur no time these days unfortunately
[11:31] ianbarber i think it would be very beneficial for engines though, everyone has to deal with many cores these days
[11:32] ianbarber x360 is what, 6? ps3 7. and most decent desktops are at least 4
[11:32] pieterh I recall people asking about this ages ago, when 0MQ was 0.something
[11:45] pieterh mikko-_: you got a second?
[11:48] mikko- pieterh: yes
[11:49] pieterh I'd like to start on a new phase of the zmq packaging
[11:49] pieterh that is, start including other parts of the community into "ZeroMQ/x.y"
[11:49] pieterh 2.1 is libzmq and libpgm
[11:50] pieterh I'm thinking 2.2 will be libzmq, libpgm, libzapi, and maybe one or two other language bindings
[11:51] pieterh any thoughts on that? libzapi uses autotools, should be simple
[11:53] sustrik given that it's exactly 1 day old, it sound a bit premature to include it into stable
[11:53] pieterh sustrik: that's why I said 2.2...
[11:53] sustrik ah, ok
[11:53] pieterh :-) indeed, it's not ready for stable
[11:54] pieterh it strikes me that libpgm integration doesn't have to be a special case
[11:55] pieterh sustrik: btw, I used inproc pipes to create a lockfree object, as you do inside core
[11:55] pieterh it's tricky because the object is actually the context wrapper, but it's really neat otherwise
[11:58] mikko- hmm
[11:58] mikko- pieter: you mean into same package?
[11:59] pieterh yes, I mean expand the scope of the download
[11:59] mikko- distros like if you keep things separate
[11:59] pieterh different issue
[11:59] pieterh distros can repackage individual libraries
[11:59] pieterh users like getting one package
[11:59] pieterh imagine having to download & build libpgm separately
[11:59] mikko- thats what i would normally expect to do
[12:00] mikko- :)
[12:00] mikko- install dependencies before the software
[12:00] pieterh yes, but you're a developer not a sysadmin
[12:00] mikko- from sysadmin pov i would package rpm and install the packages that are defined in "BuildRequires" stanza
[12:00] mikko- assuming everything is available as rpm
[12:00] pieterh we can still publish minimal packages for distros
[12:01] mikko- do you want these to be built together as well?
[12:01] mikko- or just packaged together?
[12:01] pieterh well, there are two main reasons for packaging together
[12:01] pieterh a. convenience for installing
[12:01] pieterh b. tested as a single unit
[12:02] pieterh so for b, yes, we'd also build/test the collection
[12:02] pieterh as we would do for libpgm, it's exactly the same
[12:02] pieterh otherwise 0MQ cannot scale
[12:03] mikko- where do you draw the line what should be packaged?
[12:03] mikko- should all language bindings be bundled?
[12:03] pieterh well, I don't draw a line as such
[12:03] mikko- what about devices?
[12:03] pieterh what I'd do is do this step by step and stop when it's no longer worth going further
[12:03] pieterh imagine there's a separate project with overall tests
[12:03] pieterh I'd probably package that
[12:04] pieterh I'd probably package all available documentation
[12:04] pieterh but gradually, otherwise it's indigestable
[12:05] pieterh this also makes it easier to e.g. make a Windows kit, it's the same set of packages with a MSI frontend
[12:06] pieterh so the release repositories are where this happens, presumably using submodules
[12:07] pieterh this also gives project developers something to aim at
[12:07] Guthur I have to admit that I would take a similar point of view as mikko
[12:07] pieterh many people will, for sure
[12:07] Guthur I have a similar decision to make with clrzmq2
[12:08] pieterh if you actually look at the process of getting 0MQ working with language XYZ, it's often a real pain
[12:08] Guthur prepackaging libzmq.dll would allow the binding to seamlessly support 32/64 bit environment
[12:08] pieterh indeed
[12:08] Guthur but I would rather have the users get the libzmq straight from the 'golden' source
[12:09] pieterh If I download 0MQ for Windows I'd expect to get clrzmq2 included
[12:09] Guthur so bug fixes etc can be propagated
[12:09] pieterh well, as an option
[12:09] pieterh but mostly, it's asking users to do extra work
[12:09] pieterh that always eliminates a large chunk of users
[12:10] Guthur yeah, I'm aware I maybe be falling into the worse is better mind set
[12:10] pieterh people who don't (yet) have an emotional connection to a project simply won't tolerate pain to start using it
[12:11] pieterh it has to be really, really simple to "get" 0MQ
[12:11] pieterh after that, it can become more and more complex, that's fine
[12:11] pieterh but the first taste must not be complex or difficult
[12:12] pieterh IMHO
[12:12] Guthur sure, someone is actually looking to contribute something to clrzmq2 to improve the 32/64 bit issue
[12:12] Guthur that issue would even more simply removed by prepackaging
[12:13] pieterh so let's look at making a Windows package?
[12:13] pieterh libzmq+clrzmq2+libzapi
[12:13] pieterh +libpgm
[12:14] Guthur I'll put looking at a WiX project for it on my ToDo list
[12:14] pieterh WiX = windows installer?
[12:14] Guthur WiX is a windows installer declarative language
[12:14] pieterh nice
[12:15] Guthur yeah only down side is it uses XML based grammar
[12:15] pieterh XML grammars can be neat if done right but usually they're not
[12:15] Guthur the worlds irrational love of XML pains me
[12:16] pieterh Well, I've written maybe 100k lines of declarative XML and it works fine if you're minimalist
[12:16] Guthur there is some many attributes and elements that it quickly becomes messy
[12:16] Guthur ...for WiX
[12:17] pieterh for example
[12:17] pieterh anyhow, WiX...
[12:17] pieterh if you decide to start a packaging project, let's make a separate repo for that
[12:17] pieterh not inside clrzmq2
[12:17] Guthur on the other hand I like sexp, which a lot of people don't like
[12:18] Guthur pieterh: ok
[12:30] mikko- pieter
[12:30] mikko- the doc generation is broken
[12:30] mikko- in mnay ways
[12:31] pieterh which project?
[12:31] mikko- :)
[12:31] pieterh rfc?
[12:31] mikko- zeromq
[12:31] mikko- the wikidot api thingy
[12:31] pieterh hang on a sec...
[12:31] mikko- 1) the call to wikidot fails with "Unauthorised"
[12:31] mikko- 2) it prints out your API key in the error message
[12:31] pieterh yes, I pinged you about that, you missed that
[12:31] pieterh Wikidot upgraded their API for security reasons
[12:32] mikko- i miss so many things using this web irc client
[12:32] pieterh you need to be a member of the rfc site to publish to it, that's one thing
[12:32] pieterh afair I did make you member, let me check
[12:32] pieterh s/rfc/api/
[12:33] mikko- it uses pieterh user
[12:33] pieterh you should be using your own API key by now :-)
[12:33] pieterh you were authorized ages ago
[12:33] mikko- oh, ok
[12:33] pieterh mine changed, old one is no longer valid
[12:33] mikko- will check that this evening
[12:33] mikko- good
[12:34] pieterh what's your wikidot user id?
[12:34] pieterh i need to make you member of the site
[12:34] pieterh mkoppanen, I assume, you're invited
[12:34] pieterh you were already invited, mikko :-)
[12:35] pieterh accept the invite, grab your own API key, fix your environment, and it'll work
[12:36] pieterh meanwhile I'll update that site for 2.1.3
[12:53] Guthur pieterh: actually in relation to clrzmq2 packaging, there is something called NuGet
[12:53] Guthur which I think is something similar to Maven
[12:53] pieterh Guthur tries to confuse pieterh by throwing random words at him
[12:53] Guthur similiar to ANT with dependency repo management
[12:53] private_meta This is great
[12:54] Guthur hehe I was just trying to confuse you more with that last one
[12:54] private_meta I have VirtualBox Virtual Machines running on Windows
[12:54] pieterh private_meta: VirtualBox is really useful, indeed
[12:54] private_meta As soon as I pull up the Microsoft VirtualPC Settings dialog, every VirtualBox Machine crashes
[12:54] Guthur private_meta: I go the other way round VM windows on Linux, heeh
[12:54] pieterh oh, lol
[12:54] pieterh VirtualPC has the VirtualBox killer option enabled!
[12:54] private_meta great, isn't it?
[12:55] Guthur Virtual Box is pretty neat
[12:55] pieterh it is, indeed... I used it to install a Windows iTunes to reformat my old iPod so I could use that under Ubuntu...
[12:55] private_meta It has it's disadvantages, but there is only so much choice you have if you go the gratis way
[12:55] Guthur oracle is not renowned for their generosity
[12:55] private_meta Not really, no
[12:56] pieterh Guthur: the GPL doesn't depend on generosity...
[12:56] private_meta That's not the problem imho
[12:57] private_meta Oracle owns the majority of the developers working on VirtualBox
[12:57] private_meta That means, whatever they want in, they can put in
[12:57] private_meta Same with Java or OpenOffice
[12:57] private_meta Now, as seen with OpenOffice, the only way to go against that would be a fork
[12:57] private_meta and that's never an ideal solution
[12:57] pieterh I doubt Oracle is worse than, say, IBM, who have lots of people working on Apache projects
[12:58] pieterh Oracle will happily do what IBM does, use free software to hurt competitors
[12:59] private_meta The good/bad thing about doubt is that it's not the same thing as proof
[13:00] pieterh well, Sun weren't particularly great with FOSS either
[13:00] pieterh shrug, there is no certainty in life, and that's probably a good thing
[13:01] private_meta Well, seeing it in a different way, when you analyze evolution, and you think about mutation, it's not always the "better" solution that survives
[13:01] private_meta it's the stronger one...
[13:01] private_meta Oracle might be stronger, doesn't mean it's better for everyone
[13:01] pieterh private_meta: actually, by the very definition of selection, it is always the better one... :-)
[13:02] pieterh but business is not evolutionary, really
[13:02] private_meta in some way it is
[13:02] pieterh well, actually yes, it is
[13:02] pieterh and large firms are like large predators
[13:02] private_meta it's just a redefinition of "better"
[13:02] pieterh yes, precisely
[13:03] pieterh large firms can be extraordinarily greedy, damaging, etc. but they are also slow, stupid, cautious
[13:03] Guthur 0MQ is better, so we win, hehe
[13:03] pieterh 100 genius brains, sitting in a corporate office, cannot compete with 100 happy brains, organized freely
[13:04] pieterh we're better, therefore 0MQ is better
[13:04] stimpie if we find out why my application gets slow when using inproc to communicate between java threads
[13:05] pieterh stimpie: how are you measuring it?
[13:06] stimpie systems with network: p-> b ->s where b uses a thread to receivce tcp and an inproc socket to sent to another thread
[13:06] stimpie when b is a single thread I get 200k m/s
[13:07] stimpie adding a dispatcher to another thread drops speed to 70k m/s
[13:07] stimpie I would expect the performance penalty in througput would be almost zero
[13:08] pieterh stimpie: how many cores are you running on?
[13:08] pieterh private_meta:
[13:08] stimpie 8 available 3 are used
[13:09] pieterh do you see a CPU usage increase?
[13:09] pieterh there are two possible causes, IMO
[13:09] pieterh a. actually doing extra work, e.g. recoding messages
[13:10] pieterh b. round-tripping, i.e. doing req/rep on each message
[13:12] stimpie extra work is minimal, all sockets are XREP
[13:12] pieterh no acknowledgments?
[13:12] stimpie none
[13:12] stimpie iam sending and receiving on a single socket
[13:13] stimpie could that cause some race condition?
[13:13] pieterh hmm, interesting. So do you see CPU usage or memory usage going up?
[13:14] pieterh presumably messages queues will back up and memory use will increase
[13:14] pieterh sounds like something in the Java binding... I've not played much with that
[13:15] sustrik if you are sending and receiving on the same socket from different threads, that can result in race condition
[13:17] stimpie cpu usage is ~230% memory usages appears to keep increasing
[13:17] pieterh stimpie: are you using the same socket from more than one thread?
[13:18] stimpie one sockets sends to multiple threads
[13:18] stimpie each thread has his own socket
[13:18] pieterh something is doing a *lot* of work... how large are these messages?
[13:19] stimpie message size is about 40 bytes, I will try using 2 sockets, one for sending one for receiving
[13:19] pieterh that won't help IMO
[13:19] pieterh stimpie: even your single thread case is abnormally slow
[13:19] pieterh 200K per second is not what you'd expect
[13:20] pieterh best approach IMO would be to verify Java binding performance with a simple test case
[13:21] pieterh two threads, send messages from A to B over inproc
[13:21] stimpie oke let me write something, just a sec
[13:22] pieterh problem can be in 3 layers, libzmq, jzmq, or your code
[13:22] pieterh this test should tell us what layer is guilty
[14:02] stimpie while(true) send(); //is not nice on inproc
[14:02] stimpie duration: 1961.39 ms, messages:1000000, speed: 509842.88 m/s
[14:02] stimpie xrep inproc->xrep inproc
[14:07] stimpie ps. this is on my 'slow' laptop
[14:13] stimpie my test for 2way messages appears to deadlock, see
[14:14] stimpie aha my mistake
[14:16] stimpie performance on with a 2 way ping-pong
[14:16] stimpie duration: 3871.77 ms, messages:100000, speed: 25827.98 m/s
[14:21] sustrik what's wrong with that?
[14:22] stimpie nothing
[14:22] stimpie 2 threads both sending and receiving 1M messages (not interleaved)
[14:22] stimpie duration: 531.86 ms, messages:100000, speed: 188020.73 m/s
[14:22] sustrik is it 25k rountrips/sec
[14:22] sustrik ?
[14:22] stimpie yes roundtrips
[14:22] sustrik that's 40us per roundrip
[14:22] sustrik i.e. 20us single-way
[14:22] pieterh stimpie: you said you weren't doing round trips in your application
[14:22] sustrik that sounds reasonable, given that JNI and java is on the stack
[14:23] stimpie pieterh, its just a test case
[14:24] pieterh sure, but we want a test case that simulates the basic flow you're using
[14:24] stimpie so far everything appears fine with jzmq+zmq
[14:24] pieterh i.e. xrep to xrep over inproc, one-way
[14:24] pieterh hang on, you did that, 509K/sec...
[14:25] pieterh and 188k send/recv...
[14:25] pieterh IMO there's something with the Java threading
[14:25] stimpie the send/recv should be double its only measured on 1 side
[14:28] stimpie I do believe my setup with a client -> dispatcher -> thread -> dispatcher -> client should not use one socket to connect thread and dispatcher
[14:29] stimpie probably I should run each dispatcher in his own thread
[14:30] stimpie that should give a flow of ~500K m/s
[14:32] pieterh looks like that's the jzmq limit per thread, on your 'slow' machine
[14:38] mato mikko-_: hi
[14:39] mato actually, more a question for pieterh
[14:39] pieterh mato: shoot
[14:39] mato pieterh: how soon until 2.1.4? i can either arm-twist the debian build to work with 2.1.3 or wait for 2.1.4 (re the openpgm thread)
[14:40] pieterh I can make 2.1.4 this weekend, I'd like to wait a day or two for more feedback on use
[14:40] mato ok, fine
[14:40] pieterh if you are blocked by this, I can make it sooner, or give you a preview
[14:40] pieterh but you need an official package, don't you
[14:41] mato pretty much
[14:41] mato don't worry about it, it can wait till the weekend
[14:41] pieterh ok, that'll work then
[14:41] mato but, i need to test the fixes. so, if i want "what will become 2.1.4", that is in zeromq2-1.git ?
[14:42] pieterh yes, the 2-1 git is the to-be-released 2.1.4
[14:42] pieterh master branch, no surprises
[14:42] mato good,m so i can do make dist myself from that to test things
[14:42] pieterh exactly
[14:42] pieterh I noticed dist also makes zip files, that's nice
[14:42] pieterh I'd like to do that in zapi and zfl too...
[14:43] mato just copy what libzmq does
[14:43] pieterh yup, will do
[14:47] stimpie my 'faster' machine gets the following perfomance on inproc duration: 12681.44 ms, messages:10000000, speed: 788554.25 m/s
[14:48] stimpie considering I got over 2M with tcp its still quite slow
[14:57] private_meta 788km/sec... that's almost twice as fast as the earths rotational speed!
[14:57] private_meta oh wait
[14:57] private_meta way more
[14:57] private_meta spinning out of control
[14:58] stimpie it would save nasa some money ;-)
[15:02] sustrik stimpie: iirc JNI locks a mutex for each call
[15:02] sustrik which could account for limiting number of calls per sec to below 1M
[15:05] SurfaceTension Does anybody have an approximate object code size for ARM Thumb?
[15:12] private_meta hmpf... it's difficult to find the problem with the paranoid pirate/zmsg problem
[15:13] mato pieterh: ok, 2.1.4 git fixes my problem but still has a couple of problems with openpgm integration, have reported those on the ML
[15:15] sustrik mato: also notice that pre-compiled devices were removed
[15:15] mato sustrik: yes, i have already noticed
[15:15] mato sustrik: thx
[15:15] sustrik which means some functionality will actually disappear after installing new package
[15:15] mato yes, i know
[15:15] mato not up to me
[15:16] sustrik no idea what process debian uses for that kind of thing
[15:16] mato actually, it won't disappear in debian
[15:16] sustrik heh
[15:16] mato you'll just get the 2.0.x devices staying installed if you installed them in the 1st place
[15:16] sustrik ok
[15:17] mato i.e. after upgrading to 2.1.x you will get libzmq0 (2.0.10) zeromq-bin (2.0.10) libzmq1 (2.1.x) and libzmq-dev (2.1.x) installed
[15:18] mato which may confuse some people, in which case i will cheerfully pass on said people to the upstream maintainer :-)
[15:19] mato hmm.
[15:20] mato sustrik: i'm just thinking, about those devices
[15:20] mato sustrik: d'you think i should force their removal?
[15:20] sustrik mato: no idea
[15:21] sustrik i would say the minor upgrade shouldn't break backward compatibility
[15:21] sustrik but it's too late for that
[15:22] sustrik hm, presumably, leaving the old devices there would be better
[15:22] sustrik at least the things won't stop working after upgrade
[15:22] mato yes, except that they won't talk to 2.1 applications anyway
[15:22] mato (wire protocol change)
[15:23] mato i'll ask the debian people i'm working with
[15:23] sustrik ah, the last debian release was 2.0, right?
[15:23] mato yup.
[15:23] mikko- mato: i will look at the two issues you mentioned
[15:23] sustrik then delete the devices
[15:23] sustrik missing executable is pretty obvious
[15:24] sustrik incompatible wire format is hard to detect
[15:24] mato sustrik: precisely
[15:24] mato sustrik: ok i'll see if there's a way to do that
[15:24] mato mikko-_: thanks!
[15:25] mikko- mato: the linking issue im not sure how that will work
[15:25] mikko- might not have time today as i got leaving drinks tonight
[15:25] mato mikko-_: well, i used to do it by building openpgm with -fvisibility=hidden
[15:26] mato mikko-_: we should be able to do that with the current system, no?
[15:26] mikko- mato: i thought that openpgm builds with visibility=hidden
[15:27] mikko- but i'll get into this at some point today or tomorrow
[15:27] mato mikko-_: IIRC what it does is use __attribute__ internally in the code
[15:27] mato mikko-_: but what we need is for it to hide the API symbols as well
[15:28] mato mikko-_: and I managed to override that by forcing it to build with -fvisibility=hidden
[15:28] mato mikko-_: back in the day...
[15:29] mikko- roughly what happens at the moment:
[15:29] mikko- zeromq build invokes openpgm build and they build a libtool convenience library
[15:30] mikko- which contains the built objects
[15:30] mikko- called
[15:30] mikko-
[15:30] mikko- and that libtool archive is added with libzmq_la_LIBADD
[15:31] mato ignoring the libtool bit, that is essentially a static library archive, right?
[15:31] mikko- essentially yes
[15:32] mato and AC_CONIFG_SUBDIRS(...) is what invokes the OpenPGM configure?
[15:33] mikko- it's collection of the built objects
[15:33] mikko- yes
[15:34] mato ok, problem is i see no way to pass e.g. CFLAGS to AC_CONFIG_SUBDIRS
[15:34] mikko- mato: if you look at
[15:34] mato or options, or anything
[15:34] mikko- they are detecting visibility
[15:35] mikko- CFLAGS are passed down from parent
[15:35] mikko- CFLAGS='-xyz' ./configure in zeromq build passes those down to openpgm build
[15:35] mato mikko-_: yes, but openpgm deals with visibility differently than zmq
[15:36] mato mikko-_: zmq uses the "*everything is hidden" approach
[15:36] mikko- CFLAGS should be safe to modify even within zeromq build as that should not be used anywhere
[15:36] mikko- CXXFLAGS and CPPFLAGS for zeromq
[15:36] mikko- as the source is c++
[15:36] sustrik mato: btw, how do i remove an outdated topic branch from github?
[15:36] mato pgm uses default visibility, and defines PGM_GNUC_INTERNAL to use __attribute__ ((visibility("hidden"))
[15:37] mato which means if the default is set to hidden with -fvisibility=hidden
[15:37] mato then the pgm api symbols will disappear from the library
[15:37] mato sustrik: git push origin :<branch name> i think
[15:38] sustrik the colon deletes the branch?
[15:38] sustrik strange
[15:38] sustrik thanks
[15:38] mato yeah, it pushes an empty ref to it or something
[15:38] mato deleting the remote branch via push is not the same as deleting a local branch...
[15:39] sustrik ok, np
[15:42] mato mikko-_: in theory all that needs to be done is CFLAGS=-fvisibility=hidden for the PGM build
[15:42] mato mikko-_: let me know if you can't make it work
[15:42] mato i have to go, am on a train and battery dying
[15:42] mikko- mato: i will test as soon as i get a chance
[15:42] mikko- wil let you know
[15:44] mato ok, cyl
[15:58] CIA-22 zeromq2: 03Martin Sustrik 07pre30 * rbc4a1ce 10/ (8 files in 4 dirs):
[15:58] CIA-22 zeromq2: ZMQ_HWM split into ZMQ_SNDHWM and ZMQ_RCVHWM
[15:58] CIA-22 zeromq2: These new options allow to control the maximum size of the
[15:58] CIA-22 zeromq2: inbound and outbound message pipe separately.
[15:58] CIA-22 zeromq2: Signed-off-by: Martin Sustrik <> (+17 more commits...) -
[17:11] pieterh sustrik: random question
[17:11] sustrik yes?
[17:11] pieterh you're working both on master and on the pre30 branch?
[17:11] sustrik it's a topic branch
[17:11] pieterh is that a yes or a no?
[17:11] sustrik it's like a transaction
[17:12] sustrik not to break the master
[17:12] pieterh e.g. ZMQ_HWM split, is on master or on pre30
[17:12] pieterh ?
[17:12] sustrik the work is done on a branch
[17:12] sustrik then it can be pulled to master in one go
[17:12] sustrik yes
[17:13] sustrik on pre30
[17:13] pieterh I'll rephrase: are you still making improvements to the master branch provisionally marked as 2.2.0?
[17:13] pieterh or is that frozen?
[17:13] sustrik 2.2 is frozen
[17:13] sustrik tou are free to use it
[17:13] sustrik you
[17:13] pieterh yes, that was my idea, to pull specific changes into that
[17:13] sustrik note that first patch to be pulled to master is version bump to 3.0
[17:14] pieterh hmm
[17:14] pieterh ok, let me create a 2.2 release git, then we can do the switch for 3.0
[17:14] pieterh there's not much point using a topic branch for the 3.0 stuff IMO
[17:15] ptrb mato: you around?
[17:15] pieterh I'd like to cherry pick some changes into 2.2, such as the HWM change
[17:15] sustrik sure, do so
[17:15] pieterh ack, going for it now
[17:16] sustrik i would wait for reviews at least
[17:16] sustrik otherwise you risk having conflictins with actual master
[17:16] pieterh how so?
[17:17] sustrik well, say someone complains about something in pre30
[17:17] sustrik it will be fixed
[17:17] sustrik and the fixed patch will be pulled to master
[17:17] sustrik then you have an incompatible version in 2.2
[17:17] sustrik and you have to reconcile the problem by hand
[17:18] pieterh sure... as long as you make atomic commits, it'll be doable IMO
[17:18] sustrik ok, it's up to you
[17:18] pieterh at some point the structure of 3.0 will change too far
[17:18] pieterh but we're not going to make big changes in 2.2, just tweaks
[17:19] pieterh we may not even need the 2.2 release but I'd rather be ready than have to try to make it later
[17:19] sustrik are you aware that the patch in question is backward incompatible?
[17:19] sustrik actually all the patches in pre30 are
[17:19] pieterh yes, indeed
[17:20] sustrik and that it breaks all the language bindings?
[17:20] pieterh my immediate reaction is, this patch adds value, is worth it
[17:20] sustrik ok
[17:20] pieterh I think each time we've asked binding authors this question, they've answered the same
[17:21] pieterh something like, "we'd rather get new useful functionality now, even if it means fixing up stuff"
[17:21] pieterh ok
[17:22] cremes as a bindings author, i agree with that quote
[17:22] pieterh cremes: thanks, I don't like to speak for others
[17:24] sustrik cremes: it's up to you obviously
[17:24] sustrik my point was that if bacward incompatible changes are to happen anytime
[17:24] sustrik the bindings end up as a mess of ifdefs
[17:24] cremes since the library version is exposed via the zmq_* api, it's pretty easy to turn things on/off at runtime
[17:25] cremes i agree that with too many minor differences things could get ugly
[17:25] cremes as a bindings author, i reserve the right to require minimum versions of the library
[17:25] cremes so that i can keep those ifdefs to a manageable size
[17:26] cremes btw, this weekend i plan to do a small test using cxxtest for writing unit tests for yqueue
[17:27] sustrik great
[17:27] cremes i'm going to put that into a fork of the current master; i'll fork it under my username chuckremes
[17:27] cremes and point you guys to it when i get something working
[17:27] cremes i want to just stick with ruby but i have an itch that can only be scratched by c++ apparently... ;)
[17:28] sustrik i think the tests unit tests have to be incorporated into the build system somehow ultimately
[17:28] pieterh sustrik: in zfl & zapi we do this with 'make check', which runs a single main which runs all unit tests
[17:29] cremes sustrik: that will be part of my experiment
[17:29] sustrik yes, something like tha
[17:29] cremes the cxxtest readme has examples on how to include that in the build process
[17:29] sustrik that
[17:29] pieterh sustrik: ok, 2.2 is safely forked off, we can now play with the core repo...
[17:30] pieterh no hurry but I'd suggest we make and announce the rename when you're comfortable with it
[17:31] sustrik have you done the test?
[17:31] sustrik does rename retain watchers
[17:31] sustrik ?
[17:32] sustrik do the forks map nicely to the renamed repo?
[17:32] pieterh let me run some tests in my own profile, then
[17:32] pieterh (good questions)
[17:32] sustrik ok
[17:33] sustrik if you need a watcher, ping me
[17:34] pieterh ok, can you watch ?
[17:34] pieterh and also fork it
[17:36] sustrik pieterh: done
[17:36] pieterh ok, I've also made a couple of forks...
[17:36] pieterh now renaming it...
[17:37] pieterh you watched it as 'zeromq'?
[17:37] sustrik no idea
[17:37] pieterh oh snap, sorry
[17:37] pieterh I forked it, and then renamed the fork... :-)
[17:39] pieterh so it works, the repo is now called
[17:39] pieterh watchers and forks shaken but still attached
[17:40] sustrik the 'network' graph seems to be broken
[17:40] sustrik i don't see you there
[17:41] pieterh it seemed broken before the rename, too
[17:41] pieterh only showed a single entry
[17:41] sustrik that's ok as there was only a single entry
[17:41] sustrik what i'm seeing
[17:41] sustrik is that the fist commit you've made is under 'sustrik'
[17:42] pieterh what repo?
[17:42] sustrik sustrik/test
[17:42] pieterh on my box the whole network viewer is buggy, shows no commits at all
[17:42] pieterh but the commit list is accurate
[17:43] sustrik yes, commits look ok
[17:43] pieterh I've no idea what this means, a github bug?
[17:43] sustrik no idea
[17:44] pieterh well, I'd suggest we go ahead with the rename, if the network appears broken afterwards we ask github to fix it
[17:44] pieterh but let's wait a day or two until the new name is obvious
[17:44] pieterh libzmq is my preference right now
[17:45] pieterh but it's your project, your call :-)
[17:45] sustrik i'll give it a thought
[17:45] sustrik zeromq otoh would require less renaming
[17:45] sustrik ie. stable repos would stay as they are now
[17:46] pieterh indeed, it's more consistent with that
[17:46] pieterh but if we start to make the release repos into distribution containers, that changes
[17:46] sustrik yes, i know
[17:52] klestes hullo, is anyone actively awake ?
[17:53] pieterh klestes: passively awake, here...
[17:53] klestes that'll do :)
[17:53] pieterh what's up?
[17:53] klestes sorry about my first patch. I could have sworn I read somewhere that a 2.2 version wasn't being pursued.
[17:54] pieterh we changed our minds...
[17:54] klestes D'oh!
[17:54] klestes oh well, back to the hacking boarrd =)
[17:55] klestes uhm, would minor spelling corrections to header files be a worthwile contribution ?
[17:57] pieterh depends how perfectionist the maintainer is, they may just find it annoying
[17:57] klestes reasonable.
[17:57] klestes also, on to the matter of LOAD BALANCING.
[17:58] klestes now, suppose I have a client talking to a server, a PUSH/PULL kinda relationship.
[17:58] klestes once I have these two chatting, I introduce a second client PULLing from the server as well.
[17:59] klestes my question is, upon introduction of the second client, will the load be balanced again ?
[18:03] klestes (oooh! FF6 is out !)
[18:03] klestes (sorry, bug in my FPU, thats FF4)
[18:08] sustrik patches to the spelling are acceptable
[18:09] sustrik the load already assigned to the first client won't be re-balanced
[18:09] sustrik but new messages will be
[18:10] klestes ok, thats what I had thought the proper behavior should be.
[18:10] klestes where in the source is the load balancing mechanism ?
[18:14] cremes sustrik: running a test using jemalloc and LD_PRELOAD on linux
[18:14] cremes results look promising so far
[18:23] pieterh sustrik: actually there are some nice improvements in the 2.2 branch over 2.1...
[18:23] pieterh e.g. checking &logging of duplicate identities...
[18:23] pieterh I'll check them all
[18:25] cremes pieterh: i have an open ticket on that duplicate identity commit; i get crashes with it
[18:26] pieterh cremes: ah, right...
[18:26] pieterh well, 2.2 is not going to be stable for a while, that's OK
[18:26] cremes true; hopefully i'll be able to come up with a small repro
[18:27] Guthur is 2.2 available?
[18:27] Guthur is it the pgm stuff?
[18:28] pieterh Guthur: not yet, soon enough
[18:28] pieterh the pgm stuff is already in 2.1
[18:28] klestes can you not get 2.2 with git ?
[18:28] sustrik still missing in master :(
[18:28] pieterh sustrik: what's missing in master?
[18:28] pieterh ah, yes
[18:29] sustrik jemalloc: let me know whether it helps
[18:29] pieterh sustrik, mikko is still working on making a patch, maybe he's stuck, I'll ask him... I know he's real busy right now
[18:29] pieterh I'm applying those openpgm changes to 2.2, I'll see if it works
[18:29] pieterh,, libpgm tarball, that's it...
[18:38] Guthur would it be possible to expose the subscription mechanism on SUB sockets, so that people could maybe add other models to replace the prefix one
[18:39] Guthur this is in relation to 3.0
[18:47] pieterh Guthur: it's a neat idea but probably quite difficult to make happen
[18:56] sustrik Guthur: can be doable on top of 0mq, with XPUB/XSUB sockets
[18:56] sustrik when those are finished :)
[18:58] pieterh sustrik: upstream arbitrary subscription keys?
[19:02] sustrik possibly
[19:02] sustrik we'll see
[19:02] sustrik it's not done yet
[19:02] pieterh could be fun to turn xpub sockets into mini exchanges
[19:03] pieterh sustrik: if you want a patch for the openpgm stuff, I can make it, have tested that on 2.2
[19:04] pieterh but I think you want the real author's patch... sigh
[19:04] pieterh well, it's LGPL
[19:04] pieterh already signed off
[19:04] pieterh this whole free software concept sometimes confuses me... :-)
[19:29] Guthur cool, I look forward to seeing XSUB and XPUB
[19:29] Guthur will they need to be renamed after they have been released for a while, hehe
[19:31] cremes Guthur: yes, they are going to be renamed SUCK and BLOW
[19:31] cremes no one raised any objection on the mailing list, so that's going into 2.1.4
[19:31] cremes :)
[19:41] jond cremes: what's the conclusion on jemalloc
[19:42] cremes jond: superior to glibc malloc on linux for the way i use 0mq
[19:43] cremes the memory footprint still grows, but now the 'fragmentation leak' is slower
[19:43] jond cremes: does jemalloc just replace malloc/free et al or the C++ operator's as well
[19:44] cremes jond: it's my understanding that under the covers C++ #new, et al. are calling malloc/free
[19:44] cremes so jemalloc replaces them too
[19:44] pieterh cremes: sometimes you're evil...
[19:44] cremes pieterh: ah, you like the new pub/sub names then?
[19:45] pieterh I think the old names were fine (PUB/SUB)
[19:45] jond cremes: ok ; think I asked the tcmalloc people a while back and technically new etc don't have to use malloc free ....
[19:46] cremes jond: that's true; from comments i have read while researching this stuff, no one was able to find a case where c++ did *not*
[19:46] jond pieterh: HOOKER and PIMP?
[19:46] cremes use malloc/free
[19:46] pieterh HOOKER would be XSUB then?
[19:46] cremes jond: ooh, good replacement names for PUSH/PULL
[19:47] cremes :)
[19:47] pieterh where does JOHN fit in? that's my question
[19:47] cremes pieterh: PAIR -> JOHN
[19:47] pieterh surely PAIR-> FABIO
[19:47] cremes surely
[19:48] pieterh let's start a Naming Committee
[19:48] pieterh with its own repository
[19:48] cremes heh
[19:49] jond cremes: c++ new et al; that's what I've read too, but there's probably some obscure games console compiler somewhere
[19:49] cremes jond: probably
[19:50] cremes but since i don't use it, i don't care ;)
[19:51] jond so you just LD_PRELOAD with jemalloc on the path and that's it? I'm sure google recommend recompile with tcmalloc though LD_PRELOAD can work
[19:52] cremes yes
[19:52] cremes LD_PRELOAD=/usr/local/lib/ a.out
[19:53] jond cremes does it improve ruby performance at all apart from 0mq aspect?
[19:53] cremes jond: if it does, i can't see it... there may be a small % difference but i can't tell
[19:54] jond cremes: I'll go and have a look at jemalloc now
[19:55] cremes jond: when i ran sustrik's leak repro (ticket 174) the resident memory would jump to 25MB and stay there
[19:55] cremes with jemalloc it would drop back down to 15MB
[19:55] cremes so while it isn't perfect, it *was* able to help out a bit
[19:58] jond cremes: i may give jemalloc a test against an app at work, non zeromq though sadly
[20:04] cremes jond: please share your results (0mq or non-0mq) with me; i'd love to hear your results
[20:16] jond pieterh: you ever thought of putting a 'jobs' section on the website?
[20:16] pieterh jond: yes
[20:17] pieterh but rather, I'd like to make a 'marketplace'
[20:17] jond pieterh: yes better idea
[20:18] pieterh the idea would be that proven contributors can promote their business
[20:18] pieterh this would happen on the main website (not the community site) and would be run by iMatix
[20:19] pieterh 'run' = vetted, more than anything else
[20:20] sustrik btw, i've created a zeromq group on linked-in long time ago
[20:21] sustrik i don't use it much
[20:21] sustrik i don't even recall why i've created the group
[20:21] sustrik someone asked me to, i suppose
[20:21] pieterh the idea of promoting partners able to help in specific areas, or specific regions, seems win-win
[20:21] pieterh :-)
[20:21] sustrik intersting
[20:21] sustrik 28 members
[20:22] sustrik looks like some people are actually using linkedin
[20:22] pieterh some people do seem to like collecting contacts, indeed...
[20:22] Moltar I'm trying to limit the outstanding messages in python using setsockopt(zmq.HWM, 1000) (for example) and it seems that this has no effect. Any suggestions to what I might be doing wrong?
[20:24] hexa Hi I just went tought the guide.. and wow :) .. but it got me many ideas one of which is to use it to do some worker/task management inprocess .. but I'm struggling to find a way to pass for exemple function pointers to a generic worker thread.. I checked google protobufs .. things like that but i'm not sure about it when it comes to pointers etc.. any suggestions ideas on this ?
[20:25] pieterh hexa: the way I send pointers between threads is printf ("%p", pointer) and sscanf (message, "%p", &pointer)
[20:25] sustrik Moltar: the messages are presumably queued in TCP buffers
[20:25] pieterh seems to work
[20:26] pieterh hexa: there's an example in the zapi, zctx class, somewhere in
[20:26] hexa pieterh, heheh guess that would work.. but I would like something a bit more versatile...
[20:27] pieterh well, just shove the pointer in a message of the right size
[20:27] Moltar sustrik: I'm using unix-domain socket with zSocket.bind('ipc:///tmp/sendalias'), so probably queued in unix socket buffer
[20:27] Moltar sustrik: But I have quite a lot of data to push to my worker processes, and I don't think having gigabytes of data in this buffer is going to be a good idea.
[20:28] hexa pieterh, heh I guess I could just do void* func , void* marshalling_func, void* data , pieterh thx for the link I hadn't see that yet
[20:33] jond sustrik: what's the status of subscription forwarding in master?
[20:50] SurfaceTension still nobody with a (rough) number for the binary code size on arm thumb?
[21:11] sustrik Moltar: you can set the buffer size using ZMQ_SNDBUF and ZMQ_RCVBUF options
[21:11] sustrik jond: no progress
[21:11] sustrik SurfaceTension: try asking on the mailing list
[21:12] Moltar sustrik: Thanks, I'll give that a go
[21:12] sustrik there are few people using 0mq on ARM iirc
[21:18] jond sustrik: does that mean it's all in master but incomplete or on that branch?
[21:19] sustrik it's incomplete and located in sub-forward branch
[21:20] jond sustrik: ok, thanks. I'm abandoning xrep you see from my proof of concept
[21:56] SurfaceTension sustrik: will do, thanks
[22:47] Guthur thrift provides a means for RPC, which I suppose could be used instead passing func ptrs
[22:47] Guthur but hexa has left...