[Time] Name | Message |
[00:05] mikko
|
pieterh: did you push the latest make dist fixes?
|
[00:12] Guthur
|
jeckstein, any success getting a minimal test case?
|
[00:14] jeckstein
|
no, its going to take me a little while to do that, i just compiled and loaded the newest clrzmq2 (2.1.0) and am testing it on my application now
|
[00:18] Guthur
|
ok, cool
|
[00:37] jaten
|
hey guys, I'm getting a core dump in /lib/libpthread.so in the head of the git repos for zeromq, running the first server.cpp example in the Guide. This is on ubuntu 10.10 with /lib/libpthread.so.0 from libc 2.12.1-0ubuntu10.2. It doesn't happen with with the earlier 10.04 with libc 2.11.1-0ubuntu7.8 or 7.7, which has /lib/libpthread.so.0 version 2.11.1. Is this a known issue? If not where would I post for assistance/bug report?
|
[00:43] Guthur
|
jaten, you should probably raise a ticket on github, and maybe fire a description on to the mailing list
|
[01:51] jaten
|
Guthur: okay, thanks. I'll do that.
|
[06:16] ferrouswheel
|
is there any support for message priorities or QoS? saw various discussions on the mailing list, but not sure if it eventuated into anything?
|
[06:26] sustrik
|
ad message priorities: no, you have to use different message feeds to get different priorities
|
[06:26] sustrik
|
ad QoS: it can be set on network level
|
[06:27] sustrik
|
http://www.zeromq.org/whitepapers:traffic-monitoring-v20
|
[06:30] ferrouswheel
|
sustrik, thanks - that's what i suspected
|
[06:39] ferrouswheel
|
i want to use zeromq for an (initially) inprocess rpc, but would like to decrease the priority of clients that make more requests. I can see various ways to do this, but am curious if a particular design makes more logical sense than others when using zeromq...
|
[06:46] sustrik
|
ferrouswheel: create several REP sockets, one for each priority
|
[06:46] sustrik
|
read requests from high priority sockets first
|
[06:47] sustrik
|
alternatively, you can add priority scheduling to 0mq core directly, which is something that's on the roadmap already
|
[06:50] ferrouswheel
|
sustrik, okay - the async server (fig 47) in the guide also seems appropriate... deciding on which worker to send a client to, depending on the request rate
|
[06:50] ferrouswheel
|
but i think i'll leave 0mq core modifications until i've actually used and understand 0mq better ;-)
|
[06:50] sustrik
|
ok
|
[08:28] ianbarber
|
fully confused by git. i pulled in changes from the upstream, but it seems to have committed it as a commit, so now if I do a pull req it includes those
|
[08:28] ianbarber
|
bunch of zguide examples and stuff
|
[08:42] Steve-o
|
hence it's called "git"
|
[08:42] Steve-o
|
I think it's git-rebase for the equivalent of svn update
|
[08:43] ianbarber
|
ah cool, will have a look
|
[09:55] private_meta
|
I've got a tcp connection, my client is stopping, but it won't quit the program, it hangs at "return" in main in mailbox_t::recv(), any idea why?
|
[09:56] private_meta
|
It hangs there at the termination of the context, although the connection should have been set not to linger (zmq_linger with a value of 0)
|
[10:01] pieterh
|
private_meta: you have a socket open somewhere
|
[10:02] pieterh
|
you have to close every single socket and you have to first set linger to zero on any socket which may still be doing some work (sending, or connecting)
|
[10:03] pieterh
|
this is the rather painful way 2.1 works but it can be fixed in the language binding
|
[10:03] private_meta
|
pieterh: linger is on zero on EVERY socket that is open, and how can I close the sockets in C++ apart from calling their destructor (which I'm already doing)?
|
[10:04] pieterh
|
I see two possibilities... either you have forgotten a socket somewhere, or the C++ binding is buggy
|
[10:05] private_meta
|
let me try something
|
[10:05] pieterh
|
private_meta: I've just checked the C++ binding code and there is an explicit close method
|
[10:06] pieterh
|
which the destructor just wraps
|
[10:06] private_meta
|
Ah ok... the socket was a pointer in there, and destructor of a pointer isn't called implicitly
|
[10:06] pieterh
|
right
|
[10:06] private_meta
|
only if it's a non-pointer object (which I use normally)
|
[10:07] pieterh
|
in zapi, I'm delingering/closing sockets automatically when the context is destroyed
|
[10:07] pieterh
|
it makes life so much simpler
|
[10:08] private_meta
|
zapi?
|
[10:08] pieterh
|
the new high-level C API
|
[10:08] pieterh
|
http://zero.mq/c
|
[10:09] private_meta
|
Well, if the socket object were destroyed that would have been the case, but it wasn't
|
[10:10] pieterh
|
Sure, you still need to go set linger to zero on sockets
|
[10:11] private_meta
|
I did some other stuff for a week so I'm still at translating the guide examples
|
[10:11] pieterh
|
into C++?
|
[10:11] private_meta
|
yeah
|
[10:11] pieterh
|
was it you who asked about modifying the C++ API?
|
[10:12] private_meta
|
Well, I asked about using pointers in the C++ binding
|
[10:12] private_meta
|
and the zmq::version
|
[10:12] pieterh
|
yeah, and that led to thinking about splitting the C++ binding into its own project
|
[10:12] private_meta
|
Yes
|
[10:13] private_meta
|
As far as I remember, it did
|
[10:13] pieterh
|
you already translated zhelpers, zmsg, etc. right?
|
[10:13] private_meta
|
Yes
|
[10:13] pieterh
|
so... I have a small suggestion
|
[10:13] pieterh
|
let's make zapi++
|
[10:13] private_meta
|
I didN't get to continue until now, so I'm at ppworker/ppqueue now
|
[10:13] pieterh
|
zapi basically refactors the zhelpers/zmsg APIs into something neater
|
[10:15] private_meta
|
Sounds ok, but I'd recommend not using pointers in function parameters
|
[10:15] pieterh
|
indeed
|
[10:16] pieterh
|
so the thing here is that I'm going to migrate the C examples onto zapi
|
[10:16] pieterh
|
maybe not all, but the more complex ones, in any case
|
[10:17] pieterh
|
since you know the zhelpers/zmsg API well, would you take a look at zapi?
|
[10:18] private_meta
|
sure
|
[10:18] pieterh
|
it'd be really useful to get your design of a C++ binding at the same level
|
[10:18] private_meta
|
but right now I need to fix this (somewhat) small ppworker problem I have, it doesn't get a connection
|
[10:19] pieterh
|
if you need a hand, shout, I'm happy to help debug code
|
[10:19] pieterh
|
paranoid pirates :-)
|
[10:20] private_meta
|
Apparently you dislike curly brackets in example code
|
[10:20] pieterh
|
I use them only where needed
|
[10:20] pieterh
|
code minimalism is kind of a style with me
|
[10:20] private_meta
|
Every programming teacher tells you to not do that ;)
|
[10:20] pieterh
|
that's why they're teachers
|
[10:20] pieterh
|
not programmers
|
[10:20] private_meta
|
It destroys readability
|
[10:21] pieterh
|
there's only one way to learn how to program
|
[10:21] pieterh
|
to quote Zed Shaw, by "programming, motherfucker"
|
[10:21] private_meta
|
Oh, I tend to agree with them, and many programming teachers we have here had huge programming projects already
|
[10:21] pieterh
|
so you mean for example an if statement with only one line, no braces?
|
[10:22] private_meta
|
You add one line to an if statement, tend to forget you didn't have brackets
|
[10:22] pieterh
|
sure
|
[10:23] pieterh
|
then this is one of my bad habits, removing braces when they're not needed
|
[10:23] private_meta
|
Code shouldn't be hard to read, especially if adding brackets doesn't change the compiled results
|
[10:23] pieterh
|
readability comes only from consistency, adding or removing brackets doesn't affect it IMO
|
[10:23] pieterh
|
however, if you want, go through all my examples and add brackets
|
[10:23] private_meta
|
Consistency also means you use brackets, so use them everywhere :)
|
[10:23] pieterh
|
my only rule is 100% consistency, really
|
[10:24] pieterh
|
I consistently don't use them when I can avoid them
|
[10:24] pieterh
|
it creates extra lines that do nothing, grouping a single line is pointless IMO
|
[10:24] private_meta
|
sure, it seems pointless if you like minimalism
|
[10:24] pieterh
|
indeed
|
[10:24] pieterh
|
which I really do
|
[10:25] pieterh
|
functional minimalism, of course, so using clear names, blank space, etc.
|
[10:25] pieterh
|
but it is probably just a bad habit... :-)
|
[10:25] private_meta
|
I mean, using "cond ? statem1 : statem2" would be minimalistic
|
[10:25] pieterh
|
yes, but unreadable in practice
|
[10:25] private_meta
|
still you should only do it in very special cases, 'cause mostly it's unreadable
|
[10:26] pieterh
|
I do that in some cases when it's obvious
|
[10:26] private_meta
|
yes
|
[10:26] pieterh
|
the test over time is whether you can easily reuse code
|
[10:26] pieterh
|
I was able to put together 3K lines of code in zapi, fully tested and documented, in 1 day
|
[10:26] private_meta
|
by the way, is it your style to put the opening bracket on functions in the next line, but the opening brackt of ifs and elses on the same?
|
[10:27] pieterh
|
yes, this is historic, we reused the GNU style for functions but our own style for statements
|
[10:27] private_meta
|
yeah, you used that argument already with a simple text editor and printf debugging, but that still didn't make me use that
|
[10:27] pieterh
|
I debug zapi using printfs and valgrind
|
[10:27] pieterh
|
it's extraordinarily effective
|
[10:28] private_meta
|
not for me :)
|
[10:28] pieterh
|
I'm actually shocked at being able to write masses of C code and then hammer it into shape with brute force, and it works as designed
|
[10:29] pieterh
|
trick is to start with the API, and tests, and then implement & test small pieces at a time
|
[10:29] pieterh
|
presumably C++ eliminates a lot of the pointer errors we make in C
|
[10:30] pieterh
|
anyhow, if zapi++ takes off, you can impose your style on that :-)
|
[10:30] private_meta
|
hmm
|
[10:31] private_meta
|
I've got a weird error in ppqueue that doesn't let lpclient connect
|
[10:31] pieterh
|
can you paste the code somewhere?
|
[10:31] private_meta
|
sec
|
[10:34] private_meta
|
Well, I don't quite see where the error is exactly... If I use the c version of ppqueue and ppworker, lpclient works fine
|
[10:34] private_meta
|
https://gist.github.com/5f6a4b0a41981a66d99f https://gist.github.com/0d7456f199b546ffe5e9 https://gist.github.com/253bc2038df7d9b3ba00
|
[10:34] pieterh
|
you can put multiple files into one gist BTW
|
[10:35] private_meta
|
files, yeah, but I wasn't logged in at my virtual machine in gist
|
[10:35] private_meta
|
so I didn't have my files here
|
[10:35] pieterh
|
so ppqueue doesn't poll the client at all, unless there are workers, that's the first thing
|
[10:36] pieterh
|
code looks nice, very neat...
|
[10:37] private_meta
|
My test setup is starting ppqueue, 1 ppworker, 1 lpclient
|
[10:37] pieterh
|
have you already traced activity by putting in msg.dump() all over the place?
|
[10:37] private_meta
|
ppworker uses heartbeats
|
[10:37] private_meta
|
but lpclient can't connect
|
[10:37] pieterh
|
'can't connect' means 'does not get a reply within X seconds'
|
[10:37] private_meta
|
sec
|
[10:37] pieterh
|
so you need to trace activity in the queue
|
[10:38] private_meta
|
I: connecting to server...
|
[10:38] private_meta
|
I: connecting to server...
|
[10:38] private_meta
|
W: no response from server, retrying...
|
[10:38] private_meta
|
I: connecting to server...
|
[10:38] pieterh
|
sure
|
[10:38] pieterh
|
that can mean anything
|
[10:38] pieterh
|
it's not really a connection failure
|
[10:38] Guthur
|
that could be seem behaviour I experience with teh default timeout
|
[10:38] Guthur
|
It was too aggressive in it's timeout
|
[10:39] Guthur
|
relaxing the intervals helped
|
[10:39] pieterh
|
Guthur: good memory...!
|
[10:39] private_meta
|
if I'm using ppworker and ppqueue in the c version, lpclient doesn't receive timeouts or anything
|
[10:39] pieterh
|
I fixed this in the C code, hang on, let me check...
|
[10:39] Guthur
|
pieterh: it had me scratching my head long enough that I have deep grove to remind me, hehe
|
[10:40] Guthur
|
groove*
|
[10:40] private_meta
|
by the way, using the c++ client takes way more time than the c client
|
[10:41] pieterh
|
Guthur: it was in the ppworker when it simulated a CPU overload, that was always more than the client's full timeout
|
[10:41] private_meta
|
meaning, using c queue and worker normally, then alternating between lpclient for c and c++, the c version is way faster
|
[10:41] pieterh
|
private_meta: weird, that's not expected
|
[10:42] pieterh
|
sounds like a timer problem of some kind
|
[10:42] pieterh
|
there should be no measurable (let alone visible) difference between the two
|
[10:43] pieterh
|
private_meta: so please add msg.dump on everything the broker recvs and sends, that's step 1
|
[10:45] private_meta
|
k, receiving is always done in the zmsg constructor, so I'll add the dump there. should I paste the output of ppqueue?
|
[10:45] pieterh
|
sounds good
|
[10:45] pieterh
|
that will show us what the client is sending
|
[10:46] private_meta
|
Should I also dump messages in worker and client or just queue for now?
|
[10:46] pieterh
|
then we also want to see what ppqueue is sending
|
[10:46] pieterh
|
start with queue
|
[10:46] private_meta
|
k
|
[10:46] pieterh
|
presumably there's a problem between queue and worker
|
[10:46] pieterh
|
so next, worker
|
[10:51] private_meta
|
argh
|
[10:51] private_meta
|
how do I redirect output AND stderror to file?
|
[10:51] pieterh
|
myprog > err.log 2>&1
|
[10:52] private_meta
|
"Aborted"
|
[10:53] private_meta
|
ah ok now
|
[10:53] private_meta
|
argh
|
[10:54] pieterh
|
sounds positive... :-)
|
[10:54] pieterh
|
or else you're at the dentist... "open wider, please!"
|
[10:54] ianbarber
|
pieterh: sent a pull req for the zguide, noticed that it was notifying sustrik though, which seems probably incorrect
|
[10:55] private_meta
|
no, that didn't redirect everything
|
[10:55] pieterh
|
ianbarber: weird, but presumably he's a watcher of that repo?
|
[10:55] private_meta
|
ok, screw it, I'll just copy from console if that doesn't work
|
[10:57] private_meta
|
pieterh: https://gist.github.com/2a9823bb9163202ec026
|
[10:57] private_meta
|
pieterh: that's the log from starting ppworker till disconnecting (killing) ppworker
|
[10:58] private_meta
|
in the middle, I started lpclient and it terminated by itself
|
[10:58] pieterh
|
ianbarber: got them, thanks!
|
[10:58] pieterh
|
private_meta: ok, reading that...
|
[10:59] pieterh
|
private_meta: I assume it should say 'after recv' not 'before recv'...
|
[10:59] private_meta
|
ah yes
|
[10:59] pieterh
|
so you have a bug in the worker
|
[10:59] pieterh
|
try running your queue, your client, my worker
|
[11:00] pieterh
|
I'll let you figure out what the error is, it's clear from the trace
|
[11:00] pieterh
|
:-)
|
[11:00] private_meta
|
Doesn't work
|
[11:00] private_meta
|
same error
|
[11:01] pieterh
|
that's unpossible! :-)
|
[11:01] pieterh
|
ok, try my worker, my client, your queue...
|
[11:02] pieterh
|
hang on, that's just the same, sorry...
|
[11:02] pieterh
|
your worker, my queue, your client...
|
[11:03] pieterh
|
the error shows on lines 32-41
|
[11:03] private_meta
|
should be the same, let me check (i think i tried that already)
|
[11:03] private_meta
|
No, doesn't work
|
[11:04] private_meta
|
sec, gotta fetch a delivery, bb in 5 minutes or so
|
[11:04] pieterh
|
then you have a problem in BOTH your worker and your queue, perhaps because it's in the msg class
|
[11:04] pieterh
|
cyl, then
|
[11:08] private_meta
|
i hope this time the tablet pc is alright
|
[11:08] private_meta
|
pieterh: ok, then what might be the problem in the zmsg class? any idea?
|
[11:08] pieterh
|
private_meta: look at lines 32-41 of your trace
|
[11:08] pieterh
|
the worker sends back (or the queue wrongly receives) the worker address twice
|
[11:09] pieterh
|
when it should be worker address, then original client address
|
[11:09] pieterh
|
thus the reply to the client gets sent back to the worker instead
|
[11:09] private_meta
|
hmm
|
[11:09] pieterh
|
you see the wrong addresses, right?
|
[11:10] private_meta
|
yes
|
[11:10] pieterh
|
this is kind of standard when using envelopes, they are tricky to get right
|
[11:11] pieterh
|
you can now add printfs to msg where it's recving and sending frames
|
[11:11] private_meta
|
So this might be a problem the selftest didn't catch?
|
[11:11] pieterh
|
for sure
|
[11:12] pieterh
|
hmm, there is a unit test for this but it's not double-checking every frame
|
[11:12] pieterh
|
could improve that
|
[11:13] private_meta
|
ah goddammit, i need to fetch a usb dvd drive again...
|
[11:13] pieterh
|
:-)
|
[11:13] private_meta
|
well, hp guys didn't think it would be a good idea to install a 64bit operating system on their latest notebooks
|
[11:13] pieterh
|
32 bits should be enough for anyone
|
[11:13] pieterh
|
maybe it's a subtle hint to install Ubuntu...
|
[11:14] private_meta
|
No, I can't use Linux in everyday life, creates too many problems
|
[11:14] private_meta
|
(Murphies Law and everything)
|
[11:14] pieterh
|
my sympathies...
|
[11:15] private_meta
|
Everytime I tried to use something on Linux for personal use, it failed in a way that wasn't fixable
|
[11:15] pieterh
|
weird, I've been using Linux for 10+ years and it's irreplaceable for me
|
[11:15] private_meta
|
I've been using Linux for that long as well, but I can't use it as a primary system
|
[11:16] private_meta
|
apart from that, as a gamer I need Windows anyway
|
[11:16] pieterh
|
10+ years for primary personal use, since early 1990's for work
|
[11:16] pieterh
|
ah, games
|
[11:16] private_meta
|
Well, some like it, others don't, I tend to be someone fond of games :)
|
[11:17] private_meta
|
But it's not like playing games is the only reason ;)
|
[11:17] pieterh
|
are any game engines using 0MQ yet, I wonder...
|
[11:18] pieterh
|
"shoots 4M bullets per second!!!"
|
[11:27] mikko-
|
mato
|
[11:27] mikko-
|
there?
|
[11:27] mikko-
|
pieterh: you remember ben from zeromq london meetup?
|
[11:27] pieterh
|
sure
|
[11:27] mikko-
|
he works in gaming industry
|
[11:28] pieterh
|
right, he was "putting 0MQ everywhere"
|
[11:28] mikko-
|
you could pry from him on which games he works on
|
[11:28] pieterh
|
nah, I don't really care :-)
|
[11:28] ianbarber
|
his latest, total war shogun 2 is out now! i'm sure he'd support your purchasing :)
|
[11:29] pieterh
|
I used to write games but don't play them
|
[11:29] Guthur
|
I really liked the total war series
|
[11:29] Guthur
|
don't game much anymore though
|
[11:30] ianbarber
|
shogun 2 is actually quite a lot of fun. i'm all mac everywhere now though, and they're windows only though
|
[11:30] Guthur
|
no time these days unfortunately
|
[11:31] ianbarber
|
i think it would be very beneficial for engines though, everyone has to deal with many cores these days
|
[11:32] ianbarber
|
x360 is what, 6? ps3 7. and most decent desktops are at least 4
|
[11:32] pieterh
|
I recall people asking about this ages ago, when 0MQ was 0.something
|
[11:45] pieterh
|
mikko-_: you got a second?
|
[11:48] mikko-
|
pieterh: yes
|
[11:49] pieterh
|
I'd like to start on a new phase of the zmq packaging
|
[11:49] pieterh
|
that is, start including other parts of the community into "ZeroMQ/x.y"
|
[11:49] pieterh
|
2.1 is libzmq and libpgm
|
[11:50] pieterh
|
I'm thinking 2.2 will be libzmq, libpgm, libzapi, and maybe one or two other language bindings
|
[11:51] pieterh
|
any thoughts on that? libzapi uses autotools, should be simple
|
[11:53] sustrik
|
given that it's exactly 1 day old, it sound a bit premature to include it into stable
|
[11:53] pieterh
|
sustrik: that's why I said 2.2...
|
[11:53] sustrik
|
ah, ok
|
[11:53] pieterh
|
:-) indeed, it's not ready for stable
|
[11:54] pieterh
|
it strikes me that libpgm integration doesn't have to be a special case
|
[11:55] pieterh
|
sustrik: btw, I used inproc pipes to create a lockfree object, as you do inside core
|
[11:55] pieterh
|
it's tricky because the object is actually the context wrapper, but it's really neat otherwise
|
[11:58] mikko-
|
hmm
|
[11:58] mikko-
|
pieter: you mean into same package?
|
[11:59] pieterh
|
yes, I mean expand the scope of the download
|
[11:59] mikko-
|
distros like if you keep things separate
|
[11:59] pieterh
|
different issue
|
[11:59] pieterh
|
distros can repackage individual libraries
|
[11:59] pieterh
|
users like getting one package
|
[11:59] pieterh
|
imagine having to download & build libpgm separately
|
[11:59] mikko-
|
thats what i would normally expect to do
|
[12:00] mikko-
|
:)
|
[12:00] mikko-
|
install dependencies before the software
|
[12:00] pieterh
|
yes, but you're a developer not a sysadmin
|
[12:00] mikko-
|
from sysadmin pov i would package rpm and install the packages that are defined in "BuildRequires" stanza
|
[12:00] mikko-
|
assuming everything is available as rpm
|
[12:00] pieterh
|
we can still publish minimal packages for distros
|
[12:01] mikko-
|
do you want these to be built together as well?
|
[12:01] mikko-
|
or just packaged together?
|
[12:01] pieterh
|
well, there are two main reasons for packaging together
|
[12:01] pieterh
|
a. convenience for installing
|
[12:01] pieterh
|
b. tested as a single unit
|
[12:02] pieterh
|
so for b, yes, we'd also build/test the collection
|
[12:02] pieterh
|
as we would do for libpgm, it's exactly the same
|
[12:02] pieterh
|
otherwise 0MQ cannot scale
|
[12:03] mikko-
|
where do you draw the line what should be packaged?
|
[12:03] mikko-
|
should all language bindings be bundled?
|
[12:03] pieterh
|
well, I don't draw a line as such
|
[12:03] mikko-
|
what about devices?
|
[12:03] pieterh
|
what I'd do is do this step by step and stop when it's no longer worth going further
|
[12:03] pieterh
|
imagine there's a separate project with overall tests
|
[12:03] pieterh
|
I'd probably package that
|
[12:04] pieterh
|
I'd probably package all available documentation
|
[12:04] pieterh
|
but gradually, otherwise it's indigestable
|
[12:05] pieterh
|
this also makes it easier to e.g. make a Windows kit, it's the same set of packages with a MSI frontend
|
[12:06] pieterh
|
so the release repositories are where this happens, presumably using submodules
|
[12:07] pieterh
|
this also gives project developers something to aim at
|
[12:07] Guthur
|
I have to admit that I would take a similar point of view as mikko
|
[12:07] pieterh
|
many people will, for sure
|
[12:07] Guthur
|
I have a similar decision to make with clrzmq2
|
[12:08] pieterh
|
if you actually look at the process of getting 0MQ working with language XYZ, it's often a real pain
|
[12:08] Guthur
|
prepackaging libzmq.dll would allow the binding to seamlessly support 32/64 bit environment
|
[12:08] pieterh
|
indeed
|
[12:08] Guthur
|
but I would rather have the users get the libzmq straight from the 'golden' source
|
[12:09] pieterh
|
If I download 0MQ for Windows I'd expect to get clrzmq2 included
|
[12:09] Guthur
|
so bug fixes etc can be propagated
|
[12:09] pieterh
|
well, as an option
|
[12:09] pieterh
|
but mostly, it's asking users to do extra work
|
[12:09] pieterh
|
that always eliminates a large chunk of users
|
[12:10] Guthur
|
yeah, I'm aware I maybe be falling into the worse is better mind set
|
[12:10] pieterh
|
people who don't (yet) have an emotional connection to a project simply won't tolerate pain to start using it
|
[12:11] pieterh
|
it has to be really, really simple to "get" 0MQ
|
[12:11] pieterh
|
after that, it can become more and more complex, that's fine
|
[12:11] pieterh
|
but the first taste must not be complex or difficult
|
[12:12] pieterh
|
IMHO
|
[12:12] Guthur
|
sure, someone is actually looking to contribute something to clrzmq2 to improve the 32/64 bit issue
|
[12:12] Guthur
|
that issue would even more simply removed by prepackaging
|
[12:13] pieterh
|
so let's look at making a Windows package?
|
[12:13] pieterh
|
libzmq+clrzmq2+libzapi
|
[12:13] pieterh
|
+libpgm
|
[12:14] Guthur
|
I'll put looking at a WiX project for it on my ToDo list
|
[12:14] pieterh
|
WiX = windows installer?
|
[12:14] Guthur
|
WiX is a windows installer declarative language
|
[12:14] pieterh
|
nice
|
[12:15] Guthur
|
yeah only down side is it uses XML based grammar
|
[12:15] pieterh
|
XML grammars can be neat if done right but usually they're not
|
[12:15] Guthur
|
the worlds irrational love of XML pains me
|
[12:16] pieterh
|
Well, I've written maybe 100k lines of declarative XML and it works fine if you're minimalist
|
[12:16] Guthur
|
there is some many attributes and elements that it quickly becomes messy
|
[12:16] Guthur
|
...for WiX
|
[12:17] pieterh
|
http://download.imatix.com/mop/beginning.html for example
|
[12:17] pieterh
|
anyhow, WiX...
|
[12:17] pieterh
|
if you decide to start a packaging project, let's make a separate repo for that
|
[12:17] pieterh
|
not inside clrzmq2
|
[12:17] Guthur
|
on the other hand I like sexp, which a lot of people don't like
|
[12:18] Guthur
|
pieterh: ok
|
[12:30] mikko-
|
pieter
|
[12:30] mikko-
|
the doc generation is broken
|
[12:30] mikko-
|
in mnay ways
|
[12:31] pieterh
|
which project?
|
[12:31] mikko-
|
:)
|
[12:31] pieterh
|
rfc?
|
[12:31] mikko-
|
zeromq
|
[12:31] mikko-
|
the wikidot api thingy
|
[12:31] pieterh
|
hang on a sec...
|
[12:31] mikko-
|
1) the call to wikidot fails with "Unauthorised"
|
[12:31] mikko-
|
2) it prints out your API key in the error message
|
[12:31] pieterh
|
yes, I pinged you about that, you missed that
|
[12:31] pieterh
|
Wikidot upgraded their API for security reasons
|
[12:32] mikko-
|
i miss so many things using this web irc client
|
[12:32] pieterh
|
you need to be a member of the rfc site to publish to it, that's one thing
|
[12:32] pieterh
|
afair I did make you member, let me check
|
[12:32] pieterh
|
s/rfc/api/
|
[12:33] mikko-
|
it uses pieterh user
|
[12:33] pieterh
|
you should be using your own API key by now :-)
|
[12:33] pieterh
|
you were authorized ages ago
|
[12:33] mikko-
|
oh, ok
|
[12:33] pieterh
|
mine changed, old one is no longer valid
|
[12:33] mikko-
|
will check that this evening
|
[12:33] mikko-
|
good
|
[12:34] pieterh
|
what's your wikidot user id?
|
[12:34] pieterh
|
i need to make you member of the site
|
[12:34] pieterh
|
mkoppanen, I assume, you're invited
|
[12:34] pieterh
|
you were already invited, mikko :-)
|
[12:35] pieterh
|
accept the invite, grab your own API key, fix your environment, and it'll work
|
[12:36] pieterh
|
meanwhile I'll update that site for 2.1.3
|
[12:53] Guthur
|
pieterh: actually in relation to clrzmq2 packaging, there is something called NuGet
|
[12:53] Guthur
|
which I think is something similar to Maven
|
[12:53] pieterh
|
Guthur tries to confuse pieterh by throwing random words at him
|
[12:53] Guthur
|
similiar to ANT with dependency repo management
|
[12:53] private_meta
|
This is great
|
[12:54] Guthur
|
hehe I was just trying to confuse you more with that last one
|
[12:54] private_meta
|
I have VirtualBox Virtual Machines running on Windows
|
[12:54] pieterh
|
private_meta: VirtualBox is really useful, indeed
|
[12:54] private_meta
|
As soon as I pull up the Microsoft VirtualPC Settings dialog, every VirtualBox Machine crashes
|
[12:54] Guthur
|
private_meta: I go the other way round VM windows on Linux, heeh
|
[12:54] pieterh
|
oh, lol
|
[12:54] pieterh
|
VirtualPC has the VirtualBox killer option enabled!
|
[12:54] private_meta
|
great, isn't it?
|
[12:55] Guthur
|
Virtual Box is pretty neat
|
[12:55] pieterh
|
it is, indeed... I used it to install a Windows iTunes to reformat my old iPod so I could use that under Ubuntu...
|
[12:55] private_meta
|
It has it's disadvantages, but there is only so much choice you have if you go the gratis way
|
[12:55] Guthur
|
oracle is not renowned for their generosity
|
[12:55] private_meta
|
Not really, no
|
[12:56] pieterh
|
Guthur: the GPL doesn't depend on generosity...
|
[12:56] private_meta
|
That's not the problem imho
|
[12:57] private_meta
|
Oracle owns the majority of the developers working on VirtualBox
|
[12:57] private_meta
|
That means, whatever they want in, they can put in
|
[12:57] private_meta
|
Same with Java or OpenOffice
|
[12:57] private_meta
|
Now, as seen with OpenOffice, the only way to go against that would be a fork
|
[12:57] private_meta
|
and that's never an ideal solution
|
[12:57] pieterh
|
I doubt Oracle is worse than, say, IBM, who have lots of people working on Apache projects
|
[12:58] pieterh
|
Oracle will happily do what IBM does, use free software to hurt competitors
|
[12:59] private_meta
|
The good/bad thing about doubt is that it's not the same thing as proof
|
[13:00] pieterh
|
well, Sun weren't particularly great with FOSS either
|
[13:00] pieterh
|
shrug, there is no certainty in life, and that's probably a good thing
|
[13:01] private_meta
|
Well, seeing it in a different way, when you analyze evolution, and you think about mutation, it's not always the "better" solution that survives
|
[13:01] private_meta
|
it's the stronger one...
|
[13:01] private_meta
|
Oracle might be stronger, doesn't mean it's better for everyone
|
[13:01] pieterh
|
private_meta: actually, by the very definition of selection, it is always the better one... :-)
|
[13:02] pieterh
|
but business is not evolutionary, really
|
[13:02] private_meta
|
in some way it is
|
[13:02] pieterh
|
well, actually yes, it is
|
[13:02] pieterh
|
and large firms are like large predators
|
[13:02] private_meta
|
it's just a redefinition of "better"
|
[13:02] pieterh
|
yes, precisely
|
[13:03] pieterh
|
large firms can be extraordinarily greedy, damaging, etc. but they are also slow, stupid, cautious
|
[13:03] Guthur
|
0MQ is better, so we win, hehe
|
[13:03] pieterh
|
100 genius brains, sitting in a corporate office, cannot compete with 100 happy brains, organized freely
|
[13:04] pieterh
|
we're better, therefore 0MQ is better
|
[13:04] stimpie
|
if we find out why my application gets slow when using inproc to communicate between java threads
|
[13:05] pieterh
|
stimpie: how are you measuring it?
|
[13:06] stimpie
|
systems with network: p-> b ->s where b uses a thread to receivce tcp and an inproc socket to sent to another thread
|
[13:06] stimpie
|
when b is a single thread I get 200k m/s
|
[13:07] stimpie
|
adding a dispatcher to another thread drops speed to 70k m/s
|
[13:07] stimpie
|
I would expect the performance penalty in througput would be almost zero
|
[13:08] pieterh
|
stimpie: how many cores are you running on?
|
[13:08] pieterh
|
private_meta: http://www.devilswiki.com/wiki:collective-intelligence
|
[13:08] stimpie
|
8 available 3 are used
|
[13:09] pieterh
|
do you see a CPU usage increase?
|
[13:09] pieterh
|
there are two possible causes, IMO
|
[13:09] pieterh
|
a. actually doing extra work, e.g. recoding messages
|
[13:10] pieterh
|
b. round-tripping, i.e. doing req/rep on each message
|
[13:12] stimpie
|
extra work is minimal, all sockets are XREP
|
[13:12] pieterh
|
no acknowledgments?
|
[13:12] stimpie
|
none
|
[13:12] stimpie
|
iam sending and receiving on a single socket
|
[13:13] stimpie
|
could that cause some race condition?
|
[13:13] pieterh
|
hmm, interesting. So do you see CPU usage or memory usage going up?
|
[13:14] pieterh
|
presumably messages queues will back up and memory use will increase
|
[13:14] pieterh
|
sounds like something in the Java binding... I've not played much with that
|
[13:15] sustrik
|
if you are sending and receiving on the same socket from different threads, that can result in race condition
|
[13:17] stimpie
|
cpu usage is ~230% memory usages appears to keep increasing
|
[13:17] pieterh
|
stimpie: are you using the same socket from more than one thread?
|
[13:18] stimpie
|
one sockets sends to multiple threads
|
[13:18] stimpie
|
each thread has his own socket
|
[13:18] pieterh
|
something is doing a *lot* of work... how large are these messages?
|
[13:19] stimpie
|
message size is about 40 bytes, I will try using 2 sockets, one for sending one for receiving
|
[13:19] pieterh
|
that won't help IMO
|
[13:19] pieterh
|
stimpie: even your single thread case is abnormally slow
|
[13:19] pieterh
|
200K per second is not what you'd expect
|
[13:20] pieterh
|
best approach IMO would be to verify Java binding performance with a simple test case
|
[13:21] pieterh
|
two threads, send messages from A to B over inproc
|
[13:21] stimpie
|
oke let me write something, just a sec
|
[13:22] pieterh
|
problem can be in 3 layers, libzmq, jzmq, or your code
|
[13:22] pieterh
|
this test should tell us what layer is guilty
|
[14:02] stimpie
|
while(true) send(); //is not nice on inproc
|
[14:02] stimpie
|
duration: 1961.39 ms, messages:1000000, speed: 509842.88 m/s
|
[14:02] stimpie
|
xrep inproc->xrep inproc
|
[14:07] stimpie
|
ps. this is on my 'slow' laptop
|
[14:13] stimpie
|
my test for 2way messages appears to deadlock, see http://pastebin.com/K20P6YhZ
|
[14:14] stimpie
|
aha my mistake
|
[14:16] stimpie
|
performance on with a 2 way ping-pong
|
[14:16] stimpie
|
duration: 3871.77 ms, messages:100000, speed: 25827.98 m/s
|
[14:21] sustrik
|
what's wrong with that?
|
[14:22] stimpie
|
nothing
|
[14:22] stimpie
|
2 threads both sending and receiving 1M messages (not interleaved)
|
[14:22] stimpie
|
duration: 531.86 ms, messages:100000, speed: 188020.73 m/s
|
[14:22] sustrik
|
is it 25k rountrips/sec
|
[14:22] sustrik
|
?
|
[14:22] stimpie
|
yes roundtrips
|
[14:22] sustrik
|
that's 40us per roundrip
|
[14:22] sustrik
|
i.e. 20us single-way
|
[14:22] pieterh
|
stimpie: you said you weren't doing round trips in your application
|
[14:22] sustrik
|
that sounds reasonable, given that JNI and java is on the stack
|
[14:23] stimpie
|
pieterh, its just a test case
|
[14:24] pieterh
|
sure, but we want a test case that simulates the basic flow you're using
|
[14:24] stimpie
|
so far everything appears fine with jzmq+zmq
|
[14:24] pieterh
|
i.e. xrep to xrep over inproc, one-way
|
[14:24] pieterh
|
hang on, you did that, 509K/sec...
|
[14:25] pieterh
|
and 188k send/recv...
|
[14:25] pieterh
|
IMO there's something with the Java threading
|
[14:25] stimpie
|
the send/recv should be double its only measured on 1 side
|
[14:28] stimpie
|
I do believe my setup with a client -> dispatcher -> thread -> dispatcher -> client should not use one socket to connect thread and dispatcher
|
[14:29] stimpie
|
probably I should run each dispatcher in his own thread
|
[14:30] stimpie
|
that should give a flow of ~500K m/s
|
[14:32] pieterh
|
looks like that's the jzmq limit per thread, on your 'slow' machine
|
[14:38] mato
|
mikko-_: hi
|
[14:39] mato
|
actually, more a question for pieterh
|
[14:39] pieterh
|
mato: shoot
|
[14:39] mato
|
pieterh: how soon until 2.1.4? i can either arm-twist the debian build to work with 2.1.3 or wait for 2.1.4 (re the openpgm thread)
|
[14:40] pieterh
|
I can make 2.1.4 this weekend, I'd like to wait a day or two for more feedback on use
|
[14:40] mato
|
ok, fine
|
[14:40] pieterh
|
if you are blocked by this, I can make it sooner, or give you a preview
|
[14:40] pieterh
|
but you need an official package, don't you
|
[14:41] mato
|
pretty much
|
[14:41] mato
|
don't worry about it, it can wait till the weekend
|
[14:41] pieterh
|
ok, that'll work then
|
[14:41] mato
|
but, i need to test the fixes. so, if i want "what will become 2.1.4", that is in zeromq2-1.git ?
|
[14:42] pieterh
|
yes, the 2-1 git is the to-be-released 2.1.4
|
[14:42] pieterh
|
master branch, no surprises
|
[14:42] mato
|
good,m so i can do make dist myself from that to test things
|
[14:42] pieterh
|
exactly
|
[14:42] pieterh
|
I noticed dist also makes zip files, that's nice
|
[14:42] pieterh
|
I'd like to do that in zapi and zfl too...
|
[14:43] mato
|
just copy what libzmq does
|
[14:43] pieterh
|
yup, will do
|
[14:47] stimpie
|
my 'faster' machine gets the following perfomance on inproc duration: 12681.44 ms, messages:10000000, speed: 788554.25 m/s
|
[14:48] stimpie
|
considering I got over 2M with tcp its still quite slow
|
[14:57] private_meta
|
788km/sec... that's almost twice as fast as the earths rotational speed!
|
[14:57] private_meta
|
oh wait
|
[14:57] private_meta
|
way more
|
[14:57] private_meta
|
spinning out of control
|
[14:58] stimpie
|
it would save nasa some money ;-)
|
[15:02] sustrik
|
stimpie: iirc JNI locks a mutex for each call
|
[15:02] sustrik
|
which could account for limiting number of calls per sec to below 1M
|
[15:05] SurfaceTension
|
Does anybody have an approximate object code size for ARM Thumb?
|
[15:12] private_meta
|
hmpf... it's difficult to find the problem with the paranoid pirate/zmsg problem
|
[15:13] mato
|
pieterh: ok, 2.1.4 git fixes my problem but still has a couple of problems with openpgm integration, have reported those on the ML
|
[15:15] sustrik
|
mato: also notice that pre-compiled devices were removed
|
[15:15] mato
|
sustrik: yes, i have already noticed
|
[15:15] mato
|
sustrik: thx
|
[15:15] sustrik
|
which means some functionality will actually disappear after installing new package
|
[15:15] mato
|
yes, i know
|
[15:15] mato
|
not up to me
|
[15:16] sustrik
|
no idea what process debian uses for that kind of thing
|
[15:16] mato
|
actually, it won't disappear in debian
|
[15:16] sustrik
|
heh
|
[15:16] mato
|
you'll just get the 2.0.x devices staying installed if you installed them in the 1st place
|
[15:16] sustrik
|
ok
|
[15:17] mato
|
i.e. after upgrading to 2.1.x you will get libzmq0 (2.0.10) zeromq-bin (2.0.10) libzmq1 (2.1.x) and libzmq-dev (2.1.x) installed
|
[15:18] mato
|
which may confuse some people, in which case i will cheerfully pass on said people to the upstream maintainer :-)
|
[15:19] mato
|
hmm.
|
[15:20] mato
|
sustrik: i'm just thinking, about those devices
|
[15:20] mato
|
sustrik: d'you think i should force their removal?
|
[15:20] sustrik
|
mato: no idea
|
[15:21] sustrik
|
i would say the minor upgrade shouldn't break backward compatibility
|
[15:21] sustrik
|
but it's too late for that
|
[15:22] sustrik
|
hm, presumably, leaving the old devices there would be better
|
[15:22] sustrik
|
at least the things won't stop working after upgrade
|
[15:22] mato
|
yes, except that they won't talk to 2.1 applications anyway
|
[15:22] mato
|
(wire protocol change)
|
[15:23] mato
|
i'll ask the debian people i'm working with
|
[15:23] sustrik
|
ah, the last debian release was 2.0, right?
|
[15:23] mato
|
yup.
|
[15:23] mikko-
|
mato: i will look at the two issues you mentioned
|
[15:23] sustrik
|
then delete the devices
|
[15:23] sustrik
|
missing executable is pretty obvious
|
[15:24] sustrik
|
incompatible wire format is hard to detect
|
[15:24] mato
|
sustrik: precisely
|
[15:24] mato
|
sustrik: ok i'll see if there's a way to do that
|
[15:24] mato
|
mikko-_: thanks!
|
[15:25] mikko-
|
mato: the linking issue im not sure how that will work
|
[15:25] mikko-
|
might not have time today as i got leaving drinks tonight
|
[15:25] mato
|
mikko-_: well, i used to do it by building openpgm with -fvisibility=hidden
|
[15:26] mato
|
mikko-_: we should be able to do that with the current system, no?
|
[15:26] mikko-
|
mato: i thought that openpgm builds with visibility=hidden
|
[15:27] mikko-
|
but i'll get into this at some point today or tomorrow
|
[15:27] mato
|
mikko-_: IIRC what it does is use __attribute__ internally in the code
|
[15:27] mato
|
mikko-_: but what we need is for it to hide the API symbols as well
|
[15:28] mato
|
mikko-_: and I managed to override that by forcing it to build with -fvisibility=hidden
|
[15:28] mato
|
mikko-_: back in the day...
|
[15:29] mikko-
|
roughly what happens at the moment:
|
[15:29] mikko-
|
zeromq build invokes openpgm build and they build a libtool convenience library
|
[15:30] mikko-
|
which contains the built objects
|
[15:30] mikko-
|
called libpgm_noinsta.la
|
[15:30] mikko-
|
libpgm_noinst.la
|
[15:30] mikko-
|
and that libtool archive is added with libzmq_la_LIBADD
|
[15:31] mato
|
ignoring the libtool bit, that libpgm_noinst.la is essentially a static library archive, right?
|
[15:31] mikko-
|
essentially yes
|
[15:32] mato
|
and AC_CONIFG_SUBDIRS(...) is what invokes the OpenPGM configure?
|
[15:33] mikko-
|
it's collection of the built objects
|
[15:33] mikko-
|
yes
|
[15:34] mato
|
ok, problem is i see no way to pass e.g. CFLAGS to AC_CONFIG_SUBDIRS
|
[15:34] mikko-
|
mato: if you look at http://openpgm.googlecode.com/svn/trunk/openpgm/pgm/configure.ac
|
[15:34] mato
|
or options, or anything
|
[15:34] mikko-
|
they are detecting visibility
|
[15:35] mikko-
|
CFLAGS are passed down from parent
|
[15:35] mikko-
|
CFLAGS='-xyz' ./configure in zeromq build passes those down to openpgm build
|
[15:35] mato
|
mikko-_: yes, but openpgm deals with visibility differently than zmq
|
[15:36] mato
|
mikko-_: zmq uses the "*everything is hidden" approach
|
[15:36] mikko-
|
CFLAGS should be safe to modify even within zeromq build as that should not be used anywhere
|
[15:36] mikko-
|
CXXFLAGS and CPPFLAGS for zeromq
|
[15:36] mikko-
|
as the source is c++
|
[15:36] sustrik
|
mato: btw, how do i remove an outdated topic branch from github?
|
[15:36] mato
|
pgm uses default visibility, and defines PGM_GNUC_INTERNAL to use __attribute__ ((visibility("hidden"))
|
[15:37] mato
|
which means if the default is set to hidden with -fvisibility=hidden
|
[15:37] mato
|
then the pgm api symbols will disappear from the library
|
[15:37] mato
|
sustrik: git push origin :<branch name> i think
|
[15:38] sustrik
|
the colon deletes the branch?
|
[15:38] sustrik
|
strange
|
[15:38] sustrik
|
thanks
|
[15:38] mato
|
yeah, it pushes an empty ref to it or something
|
[15:38] mato
|
deleting the remote branch via push is not the same as deleting a local branch...
|
[15:39] sustrik
|
ok, np
|
[15:42] mato
|
mikko-_: in theory all that needs to be done is CFLAGS=-fvisibility=hidden for the PGM build
|
[15:42] mato
|
mikko-_: let me know if you can't make it work
|
[15:42] mato
|
i have to go, am on a train and battery dying
|
[15:42] mikko-
|
mato: i will test as soon as i get a chance
|
[15:42] mikko-
|
wil let you know
|
[15:44] mato
|
ok, cyl
|
[15:58] CIA-22
|
zeromq2: 03Martin Sustrik 07pre30 * rbc4a1ce 10/ (8 files in 4 dirs):
|
[15:58] CIA-22
|
zeromq2: ZMQ_HWM split into ZMQ_SNDHWM and ZMQ_RCVHWM
|
[15:58] CIA-22
|
zeromq2: These new options allow to control the maximum size of the
|
[15:58] CIA-22
|
zeromq2: inbound and outbound message pipe separately.
|
[15:58] CIA-22
|
zeromq2: Signed-off-by: Martin Sustrik <sustrik@250bpm.com> (+17 more commits...) - http://bit.ly/gIsQa8
|
[17:11] pieterh
|
sustrik: random question
|
[17:11] sustrik
|
yes?
|
[17:11] pieterh
|
you're working both on master and on the pre30 branch?
|
[17:11] sustrik
|
it's a topic branch
|
[17:11] pieterh
|
is that a yes or a no?
|
[17:11] sustrik
|
it's like a transaction
|
[17:12] sustrik
|
not to break the master
|
[17:12] pieterh
|
e.g. ZMQ_HWM split, is on master or on pre30
|
[17:12] pieterh
|
?
|
[17:12] sustrik
|
the work is done on a branch
|
[17:12] sustrik
|
then it can be pulled to master in one go
|
[17:12] sustrik
|
yes
|
[17:13] sustrik
|
on pre30
|
[17:13] pieterh
|
I'll rephrase: are you still making improvements to the master branch provisionally marked as 2.2.0?
|
[17:13] pieterh
|
or is that frozen?
|
[17:13] sustrik
|
2.2 is frozen
|
[17:13] sustrik
|
tou are free to use it
|
[17:13] sustrik
|
you
|
[17:13] pieterh
|
yes, that was my idea, to pull specific changes into that
|
[17:13] sustrik
|
note that first patch to be pulled to master is version bump to 3.0
|
[17:14] pieterh
|
hmm
|
[17:14] pieterh
|
ok, let me create a 2.2 release git, then we can do the switch for 3.0
|
[17:14] pieterh
|
there's not much point using a topic branch for the 3.0 stuff IMO
|
[17:15] ptrb
|
mato: you around?
|
[17:15] pieterh
|
I'd like to cherry pick some changes into 2.2, such as the HWM change
|
[17:15] sustrik
|
sure, do so
|
[17:15] pieterh
|
ack, going for it now
|
[17:16] sustrik
|
i would wait for reviews at least
|
[17:16] sustrik
|
otherwise you risk having conflictins with actual master
|
[17:16] pieterh
|
how so?
|
[17:17] sustrik
|
well, say someone complains about something in pre30
|
[17:17] sustrik
|
it will be fixed
|
[17:17] sustrik
|
and the fixed patch will be pulled to master
|
[17:17] sustrik
|
then you have an incompatible version in 2.2
|
[17:17] sustrik
|
and you have to reconcile the problem by hand
|
[17:18] pieterh
|
sure... as long as you make atomic commits, it'll be doable IMO
|
[17:18] sustrik
|
ok, it's up to you
|
[17:18] pieterh
|
at some point the structure of 3.0 will change too far
|
[17:18] pieterh
|
but we're not going to make big changes in 2.2, just tweaks
|
[17:19] pieterh
|
we may not even need the 2.2 release but I'd rather be ready than have to try to make it later
|
[17:19] sustrik
|
are you aware that the patch in question is backward incompatible?
|
[17:19] sustrik
|
actually all the patches in pre30 are
|
[17:19] pieterh
|
yes, indeed
|
[17:20] sustrik
|
and that it breaks all the language bindings?
|
[17:20] pieterh
|
my immediate reaction is, this patch adds value, is worth it
|
[17:20] sustrik
|
ok
|
[17:20] pieterh
|
I think each time we've asked binding authors this question, they've answered the same
|
[17:21] pieterh
|
something like, "we'd rather get new useful functionality now, even if it means fixing up stuff"
|
[17:21] pieterh
|
ok
|
[17:22] cremes
|
as a bindings author, i agree with that quote
|
[17:22] pieterh
|
cremes: thanks, I don't like to speak for others
|
[17:24] sustrik
|
cremes: it's up to you obviously
|
[17:24] sustrik
|
my point was that if bacward incompatible changes are to happen anytime
|
[17:24] sustrik
|
the bindings end up as a mess of ifdefs
|
[17:24] cremes
|
since the library version is exposed via the zmq_* api, it's pretty easy to turn things on/off at runtime
|
[17:25] cremes
|
i agree that with too many minor differences things could get ugly
|
[17:25] cremes
|
as a bindings author, i reserve the right to require minimum versions of the library
|
[17:25] cremes
|
so that i can keep those ifdefs to a manageable size
|
[17:26] cremes
|
btw, this weekend i plan to do a small test using cxxtest for writing unit tests for yqueue
|
[17:27] sustrik
|
great
|
[17:27] cremes
|
i'm going to put that into a fork of the current master; i'll fork it under my username chuckremes
|
[17:27] cremes
|
and point you guys to it when i get something working
|
[17:27] cremes
|
i want to just stick with ruby but i have an itch that can only be scratched by c++ apparently... ;)
|
[17:28] sustrik
|
i think the tests unit tests have to be incorporated into the build system somehow ultimately
|
[17:28] pieterh
|
sustrik: in zfl & zapi we do this with 'make check', which runs a single main which runs all unit tests
|
[17:29] cremes
|
sustrik: that will be part of my experiment
|
[17:29] sustrik
|
yes, something like tha
|
[17:29] cremes
|
the cxxtest readme has examples on how to include that in the build process
|
[17:29] sustrik
|
that
|
[17:29] pieterh
|
sustrik: ok, 2.2 is safely forked off, we can now play with the core repo...
|
[17:30] pieterh
|
no hurry but I'd suggest we make and announce the rename when you're comfortable with it
|
[17:31] sustrik
|
have you done the test?
|
[17:31] sustrik
|
does rename retain watchers
|
[17:31] sustrik
|
?
|
[17:32] sustrik
|
do the forks map nicely to the renamed repo?
|
[17:32] pieterh
|
let me run some tests in my own profile, then
|
[17:32] pieterh
|
(good questions)
|
[17:32] sustrik
|
ok
|
[17:33] sustrik
|
if you need a watcher, ping me
|
[17:34] pieterh
|
ok, can you watch https://github.com/pieterh/test ?
|
[17:34] pieterh
|
and also fork it
|
[17:36] sustrik
|
pieterh: done
|
[17:36] pieterh
|
ok, I've also made a couple of forks...
|
[17:36] pieterh
|
now renaming it...
|
[17:37] pieterh
|
you watched it as 'zeromq'?
|
[17:37] sustrik
|
no idea
|
[17:37] pieterh
|
oh snap, sorry
|
[17:37] pieterh
|
I forked it, and then renamed the fork... :-)
|
[17:39] pieterh
|
so it works, the repo is now called https://github.com/pieterh/libtest
|
[17:39] pieterh
|
watchers and forks shaken but still attached
|
[17:40] sustrik
|
the 'network' graph seems to be broken
|
[17:40] sustrik
|
i don't see you there
|
[17:41] pieterh
|
it seemed broken before the rename, too
|
[17:41] pieterh
|
only showed a single entry
|
[17:41] sustrik
|
that's ok as there was only a single entry
|
[17:41] sustrik
|
what i'm seeing
|
[17:41] sustrik
|
is that the fist commit you've made is under 'sustrik'
|
[17:42] pieterh
|
what repo?
|
[17:42] sustrik
|
sustrik/test
|
[17:42] pieterh
|
on my box the whole network viewer is buggy, shows no commits at all
|
[17:42] pieterh
|
but the commit list is accurate
|
[17:43] sustrik
|
yes, commits look ok
|
[17:43] pieterh
|
I've no idea what this means, a github bug?
|
[17:43] sustrik
|
no idea
|
[17:44] pieterh
|
well, I'd suggest we go ahead with the rename, if the network appears broken afterwards we ask github to fix it
|
[17:44] pieterh
|
but let's wait a day or two until the new name is obvious
|
[17:44] pieterh
|
libzmq is my preference right now
|
[17:45] pieterh
|
but it's your project, your call :-)
|
[17:45] sustrik
|
i'll give it a thought
|
[17:45] sustrik
|
zeromq otoh would require less renaming
|
[17:45] sustrik
|
ie. stable repos would stay as they are now
|
[17:46] pieterh
|
indeed, it's more consistent with that
|
[17:46] pieterh
|
but if we start to make the release repos into distribution containers, that changes
|
[17:46] sustrik
|
yes, i know
|
[17:52] klestes
|
hullo, is anyone actively awake ?
|
[17:53] pieterh
|
klestes: passively awake, here...
|
[17:53] klestes
|
that'll do :)
|
[17:53] pieterh
|
what's up?
|
[17:53] klestes
|
sorry about my first patch. I could have sworn I read somewhere that a 2.2 version wasn't being pursued.
|
[17:54] pieterh
|
we changed our minds...
|
[17:54] klestes
|
D'oh!
|
[17:54] klestes
|
oh well, back to the hacking boarrd =)
|
[17:55] klestes
|
uhm, would minor spelling corrections to header files be a worthwile contribution ?
|
[17:57] pieterh
|
depends how perfectionist the maintainer is, they may just find it annoying
|
[17:57] klestes
|
reasonable.
|
[17:57] klestes
|
also, on to the matter of LOAD BALANCING.
|
[17:58] klestes
|
now, suppose I have a client talking to a server, a PUSH/PULL kinda relationship.
|
[17:58] klestes
|
once I have these two chatting, I introduce a second client PULLing from the server as well.
|
[17:59] klestes
|
my question is, upon introduction of the second client, will the load be balanced again ?
|
[18:03] klestes
|
(oooh! FF6 is out !)
|
[18:03] klestes
|
(sorry, bug in my FPU, thats FF4)
|
[18:08] sustrik
|
patches to the spelling are acceptable
|
[18:09] sustrik
|
the load already assigned to the first client won't be re-balanced
|
[18:09] sustrik
|
but new messages will be
|
[18:10] klestes
|
ok, thats what I had thought the proper behavior should be.
|
[18:10] klestes
|
where in the source is the load balancing mechanism ?
|
[18:14] cremes
|
sustrik: running a test using jemalloc and LD_PRELOAD on linux
|
[18:14] cremes
|
results look promising so far
|
[18:23] pieterh
|
sustrik: actually there are some nice improvements in the 2.2 branch over 2.1...
|
[18:23] pieterh
|
e.g. checking &logging of duplicate identities...
|
[18:23] pieterh
|
I'll check them all
|
[18:25] cremes
|
pieterh: i have an open ticket on that duplicate identity commit; i get crashes with it
|
[18:26] pieterh
|
cremes: ah, right...
|
[18:26] pieterh
|
well, 2.2 is not going to be stable for a while, that's OK
|
[18:26] cremes
|
true; hopefully i'll be able to come up with a small repro
|
[18:27] Guthur
|
is 2.2 available?
|
[18:27] Guthur
|
is it the pgm stuff?
|
[18:28] pieterh
|
Guthur: not yet, soon enough
|
[18:28] pieterh
|
the pgm stuff is already in 2.1
|
[18:28] klestes
|
can you not get 2.2 with git ?
|
[18:28] sustrik
|
still missing in master :(
|
[18:28] pieterh
|
sustrik: what's missing in master?
|
[18:28] pieterh
|
ah, yes
|
[18:29] sustrik
|
jemalloc: let me know whether it helps
|
[18:29] pieterh
|
sustrik, mikko is still working on making a patch, maybe he's stuck, I'll ask him... I know he's real busy right now
|
[18:29] pieterh
|
I'm applying those openpgm changes to 2.2, I'll see if it works
|
[18:29] pieterh
|
configure.in, Makefile.am, libpgm tarball, that's it...
|
[18:38] Guthur
|
would it be possible to expose the subscription mechanism on SUB sockets, so that people could maybe add other models to replace the prefix one
|
[18:39] Guthur
|
this is in relation to 3.0
|
[18:47] pieterh
|
Guthur: it's a neat idea but probably quite difficult to make happen
|
[18:56] sustrik
|
Guthur: can be doable on top of 0mq, with XPUB/XSUB sockets
|
[18:56] sustrik
|
when those are finished :)
|
[18:58] pieterh
|
sustrik: upstream arbitrary subscription keys?
|
[19:02] sustrik
|
possibly
|
[19:02] sustrik
|
we'll see
|
[19:02] sustrik
|
it's not done yet
|
[19:02] pieterh
|
could be fun to turn xpub sockets into mini exchanges
|
[19:03] pieterh
|
sustrik: if you want a patch for the openpgm stuff, I can make it, have tested that on 2.2
|
[19:04] pieterh
|
but I think you want the real author's patch... sigh
|
[19:04] pieterh
|
well, it's LGPL
|
[19:04] pieterh
|
already signed off
|
[19:04] pieterh
|
this whole free software concept sometimes confuses me... :-)
|
[19:29] Guthur
|
cool, I look forward to seeing XSUB and XPUB
|
[19:29] Guthur
|
will they need to be renamed after they have been released for a while, hehe
|
[19:31] cremes
|
Guthur: yes, they are going to be renamed SUCK and BLOW
|
[19:31] cremes
|
no one raised any objection on the mailing list, so that's going into 2.1.4
|
[19:31] cremes
|
:)
|
[19:41] jond
|
cremes: what's the conclusion on jemalloc
|
[19:42] cremes
|
jond: superior to glibc malloc on linux for the way i use 0mq
|
[19:43] cremes
|
the memory footprint still grows, but now the 'fragmentation leak' is slower
|
[19:43] jond
|
cremes: does jemalloc just replace malloc/free et al or the C++ operator's as well
|
[19:44] cremes
|
jond: it's my understanding that under the covers C++ #new, et al. are calling malloc/free
|
[19:44] cremes
|
so jemalloc replaces them too
|
[19:44] pieterh
|
cremes: sometimes you're evil...
|
[19:44] cremes
|
pieterh: ah, you like the new pub/sub names then?
|
[19:45] pieterh
|
I think the old names were fine (PUB/SUB)
|
[19:45] jond
|
cremes: ok ; think I asked the tcmalloc people a while back and technically new etc don't have to use malloc free ....
|
[19:46] cremes
|
jond: that's true; from comments i have read while researching this stuff, no one was able to find a case where c++ did *not*
|
[19:46] jond
|
pieterh: HOOKER and PIMP?
|
[19:46] cremes
|
use malloc/free
|
[19:46] pieterh
|
HOOKER would be XSUB then?
|
[19:46] cremes
|
jond: ooh, good replacement names for PUSH/PULL
|
[19:47] cremes
|
:)
|
[19:47] pieterh
|
where does JOHN fit in? that's my question
|
[19:47] cremes
|
pieterh: PAIR -> JOHN
|
[19:47] pieterh
|
surely PAIR-> FABIO
|
[19:47] cremes
|
surely
|
[19:48] pieterh
|
let's start a Naming Committee
|
[19:48] pieterh
|
with its own repository
|
[19:48] cremes
|
heh
|
[19:49] jond
|
cremes: c++ new et al; that's what I've read too, but there's probably some obscure games console compiler somewhere
|
[19:49] cremes
|
jond: probably
|
[19:50] cremes
|
but since i don't use it, i don't care ;)
|
[19:51] jond
|
so you just LD_PRELOAD with jemalloc on the path and that's it? I'm sure google recommend recompile with tcmalloc though LD_PRELOAD can work
|
[19:52] cremes
|
yes
|
[19:52] cremes
|
LD_PRELOAD=/usr/local/lib/libjemalloc.so a.out
|
[19:53] jond
|
cremes does it improve ruby performance at all apart from 0mq aspect?
|
[19:53] cremes
|
jond: if it does, i can't see it... there may be a small % difference but i can't tell
|
[19:54] jond
|
cremes: I'll go and have a look at jemalloc now
|
[19:55] cremes
|
jond: when i ran sustrik's leak repro (ticket 174) the resident memory would jump to 25MB and stay there
|
[19:55] cremes
|
with jemalloc it would drop back down to 15MB
|
[19:55] cremes
|
so while it isn't perfect, it *was* able to help out a bit
|
[19:58] jond
|
cremes: i may give jemalloc a test against an app at work, non zeromq though sadly
|
[20:04] cremes
|
jond: please share your results (0mq or non-0mq) with me; i'd love to hear your results
|
[20:16] jond
|
pieterh: you ever thought of putting a 'jobs' section on the website?
|
[20:16] pieterh
|
jond: yes
|
[20:17] pieterh
|
but rather, I'd like to make a 'marketplace'
|
[20:17] jond
|
pieterh: yes better idea
|
[20:18] pieterh
|
the idea would be that proven contributors can promote their business
|
[20:18] pieterh
|
this would happen on the main website (not the community site) and would be run by iMatix
|
[20:19] pieterh
|
'run' = vetted, more than anything else
|
[20:20] sustrik
|
btw, i've created a zeromq group on linked-in long time ago
|
[20:21] sustrik
|
i don't use it much
|
[20:21] sustrik
|
i don't even recall why i've created the group
|
[20:21] sustrik
|
someone asked me to, i suppose
|
[20:21] pieterh
|
the idea of promoting partners able to help in specific areas, or specific regions, seems win-win
|
[20:21] pieterh
|
:-)
|
[20:21] sustrik
|
intersting
|
[20:21] sustrik
|
28 members
|
[20:22] sustrik
|
looks like some people are actually using linkedin
|
[20:22] pieterh
|
some people do seem to like collecting contacts, indeed...
|
[20:22] Moltar
|
I'm trying to limit the outstanding messages in python using setsockopt(zmq.HWM, 1000) (for example) and it seems that this has no effect. Any suggestions to what I might be doing wrong?
|
[20:24] hexa
|
Hi I just went tought the guide.. and wow :) .. but it got me many ideas one of which is to use it to do some worker/task management inprocess .. but I'm struggling to find a way to pass for exemple function pointers to a generic worker thread.. I checked google protobufs .. things like that but i'm not sure about it when it comes to pointers etc.. any suggestions ideas on this ?
|
[20:25] pieterh
|
hexa: the way I send pointers between threads is printf ("%p", pointer) and sscanf (message, "%p", &pointer)
|
[20:25] sustrik
|
Moltar: the messages are presumably queued in TCP buffers
|
[20:25] pieterh
|
seems to work
|
[20:26] pieterh
|
hexa: there's an example in the zapi, zctx class, somewhere in http://zero.mq/c
|
[20:26] hexa
|
pieterh, heheh guess that would work.. but I would like something a bit more versatile...
|
[20:27] pieterh
|
well, just shove the pointer in a message of the right size
|
[20:27] Moltar
|
sustrik: I'm using unix-domain socket with zSocket.bind('ipc:///tmp/sendalias'), so probably queued in unix socket buffer
|
[20:27] Moltar
|
sustrik: But I have quite a lot of data to push to my worker processes, and I don't think having gigabytes of data in this buffer is going to be a good idea.
|
[20:28] hexa
|
pieterh, heh I guess I could just do void* func , void* marshalling_func, void* data , pieterh thx for the link I hadn't see that yet
|
[20:33] jond
|
sustrik: what's the status of subscription forwarding in master?
|
[20:50] SurfaceTension
|
still nobody with a (rough) number for the binary code size on arm thumb?
|
[21:11] sustrik
|
Moltar: you can set the buffer size using ZMQ_SNDBUF and ZMQ_RCVBUF options
|
[21:11] sustrik
|
jond: no progress
|
[21:11] sustrik
|
SurfaceTension: try asking on the mailing list
|
[21:12] Moltar
|
sustrik: Thanks, I'll give that a go
|
[21:12] sustrik
|
there are few people using 0mq on ARM iirc
|
[21:18] jond
|
sustrik: does that mean it's all in master but incomplete or on that branch?
|
[21:19] sustrik
|
it's incomplete and located in sub-forward branch
|
[21:20] jond
|
sustrik: ok, thanks. I'm abandoning xrep you see from my proof of concept
|
[21:56] SurfaceTension
|
sustrik: will do, thanks
|
[22:47] Guthur
|
thrift provides a means for RPC, which I suppose could be used instead passing func ptrs
|
[22:47] Guthur
|
but hexa has left...
|