[Time] Name | Message |
[01:47] sustrik
|
pieterh: there's a bug there
|
[01:47] sustrik
|
fix is on the mailing list
|
[01:47] sustrik
|
sent by jon dyte yesterdat
|
[01:47] sustrik
|
yesterday
|
[01:48] jhawk28
|
hello, does anyone know if the Java bindings can compile on the mac?
|
[01:49] jhawk28
|
I'm getting this error: http://pastebin.com/Vng9YCNn
|
[08:43] pieterh
|
sustrik: just saw that now, it didn't register...
|
[08:43] sustrik
|
ok
|
[09:42] shekispeaks
|
hi i am trying to install the Java Bindings on my mac
|
[09:42] shekispeaks
|
a
|
[09:42] shekispeaks
|
the ./configure always says JAVA_HOME not set
|
[09:42] shekispeaks
|
i already have it set
|
[09:42] shekispeaks
|
I am using java 1.6+
|
[09:48] sustrik
|
skekispeaks: try to locate jni.h on your box
|
[09:48] sustrik
|
where does it reside?
|
[09:49] shekispeaks
|
should i point JAVA_HOME to that directory or its parent?
|
[09:49] sustrik
|
its parent
|
[09:49] shekispeaks
|
"/System/Library/Frameworks/JavaVM.framework/Home/include
|
[09:50] sustrik
|
point it to Home
|
[09:50] shekispeaks
|
ya JAVA_HOME is already set to Home still the config always throws the same error
|
[09:50] sustrik
|
what's exactly the error?
|
[09:51] sustrik
|
there's this in the configure.in:
|
[09:51] sustrik
|
if test "x$JAVA_HOME" = "x"; then
|
[09:51] sustrik
|
AC_MSG_ERROR([the JAVA_HOME environment variable must be set to your JDK location.]);
|
[09:51] sustrik
|
fi
|
[09:52] sustrik
|
so if that's the message you are seeing
|
[09:52] sustrik
|
it looks like $JAVA_HOME resolves into empty string
|
[09:53] shekispeaks
|
abhishek:jzmq abhishekk$ echo $JAVA_HOME
|
[09:54] shekispeaks
|
"/System/Library/Frameworks/JavaVM.framework/Home"
|
[09:54] shekispeaks
|
this is the variable
|
[09:54] shekispeaks
|
and the next command is ./configure
|
[09:55] shekispeaks
|
it ends in configure: error: the JAVA_HOME environment variable must be set to your JDK location.
|
[09:56] shekispeaks
|
the variable is set
|
[10:10] pieterh
|
shekispeaks: try 'export JAVA_HOME'
|
[10:10] shekispeaks
|
i have exported it
|
[10:11] pieterh
|
I'd suggest: edit configure script to echo $JAVA_HOME or 'set > vars' before error message
|
[10:11] shekispeaks
|
abhishek:pyzmq abhishekk$ export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home
|
[10:11] shekispeaks
|
abhishek:pyzmq abhishekk$ echo $JAVA_HOME
|
[10:11] shekispeaks
|
"/System/Library/Frameworks/JavaVM.framework/Home"
|
[10:12] pieterh
|
shekispeaks: not sure you can do export like that, try 'set JAVA_HOME=...; export JAVA_HOME'
|
[10:14] pieterh
|
sustrik: is there any valid semantic in doing a blocking recv on a SUB socket with zero subscriptions?
|
[10:15] sustrik
|
yes
|
[10:15] sustrik
|
waiting for ETERM
|
[10:16] shekispeaks
|
both set export does not work
|
[10:16] shekispeaks
|
even though the variable is set in the shell
|
[10:16] shekispeaks
|
it is printing a null in the script
|
[10:17] sustrik
|
what about JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Home ./configure
|
[10:17] sustrik
|
JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Home" ./configure
|
[10:17] pieterh
|
shekispeaks: Darwin weirdness... are you using any non-standard shell?
|
[10:18] pieterh
|
sustrik: isn't that an abuse of the SUB semantics? If you want to wait for ETERM, surely there are more explicit ways
|
[10:18] pieterh
|
such as bind to private inproc endpoint and recv on it
|
[10:20] shekispeaks
|
ok i just set the variable in the config script and it ran
|
[10:21] sustrik
|
pieter: yes, you are right
|
[10:21] sustrik
|
waiting for ETRM using recv is silly
|
[10:21] pieterh
|
sustrik: i'd like to propose that blocking recv on SUB with no subscriptions is a semantic error -> assertion
|
[10:22] pieterh
|
not even worth returning an error code, an app that does this should fail quickly
|
[10:22] pieterh
|
let me throw this suggestion to the dev list...
|
[10:56] pieterh
|
sustrik: the XML parser in foreign, it's only for devices, right?
|
[11:05] sustrik
|
pieterh: yes, not used elsewhere so far
|
[11:05] pieterh
|
i'd like to use libconfig as a 'standard' way of reading config files
|
[11:05] sustrik
|
what's that?
|
[11:06] pieterh
|
its a pretty standard library for parsing config files
|
[11:06] pieterh
|
http://www.hyperrealm.com/libconfig/libconfig_manual.html
|
[11:06] pieterh
|
the 0MQ of config files
|
[11:06] pieterh
|
it would absolve us of having that foreign package
|
[11:07] pieterh
|
i want to document how to use that to configure a device process
|
[11:08] sustrik
|
for version 3.0?
|
[11:08] pieterh
|
advantage of libconfig is lots of language support and standard package on many linuxes
|
[11:08] pieterh
|
for version 2.1, I think
|
[11:08] pieterh
|
devices were not documented up to now
|
[11:08] sustrik
|
two problems i see:
|
[11:08] sustrik
|
1. one more dependency
|
[11:09] sustrik
|
what platforms it is supported on?
|
[11:09] sustrik
|
you have to check
|
[11:09] pieterh
|
1. yes, but only for main devices
|
[11:09] pieterh
|
2. pretty much everything
|
[11:09] sustrik
|
OpenVMS?
|
[11:09] sustrik
|
2. it breaks backward compatibility, so be careful
|
[11:09] pieterh
|
The library runs on modern POSIX-compilant systems, such as Linux, Solaris, and Mac OS X (Darwin), as well as on Microsoft Windows 2000/XP and later
|
[11:10] pieterh
|
2. we can leave the XML library in there until 3.0, for sure
|
[11:10] pieterh
|
but it's a long term maintenance cost
|
[11:10] sustrik
|
it was never changed since it was included
|
[11:10] pieterh
|
yes, it was
|
[11:11] pieterh
|
i made several security fixes
|
[11:11] sustrik
|
i had to revert your changes
|
[11:11] pieterh
|
ah
|
[11:11] sustrik
|
they broke win32 build
|
[11:11] pieterh
|
kind of proves my poibnt
|
[11:11] pieterh
|
*point
|
[11:11] pieterh
|
anyhow, i don't like xml config files (after using them for years!)
|
[11:11] sustrik
|
as i said, it's up to
|
[11:11] pieterh
|
any non-trivial 0MQ application will have dependencies
|
[11:12] sustrik
|
if you don't care about exotic platforms and possible dependency problems, just do ti
|
[11:12] pieterh
|
in fact... i'm thinking we don't need standard external device programs
|
[11:12] pieterh
|
just zmq_device(3) and decent docs on how to write your own
|
[11:12] pieterh
|
so I'd like to make a tutorial on how to write devices
|
[11:13] pieterh
|
(already have most of that)
|
[11:13] pieterh
|
in the guide
|
[11:13] sustrik
|
yes, it's convenience stuff
|
[11:13] sustrik
|
ok
|
[11:13] pieterh
|
ok, great
|
[11:14] sustrik
|
we can for example split pre-compiled into a separate project
|
[11:14] pieterh
|
i think it's important that 0MQ users understand the internals of devices rapidly
|
[11:14] pieterh
|
and write their own
|
[11:14] sustrik
|
leaving 0mq to be what it is supposed to be, i.e. library
|
[11:14] pieterh
|
indeed
|
[11:14] pieterh
|
that turns zmq_deviced(1) into a worked example
|
[11:14] pieterh
|
and template code that's trivial to reuse
|
[11:15] pieterh
|
we don't need a separate project at all ... if you build the Guide you get these examples running
|
[11:15] pieterh
|
it's all on github now
|
[11:15] pieterh
|
this simplifies the packaging
|
[11:15] sustrik
|
it's up to you
|
[11:15] sustrik
|
precompiled devices may be handy sometimes though
|
[11:15] pieterh
|
for me, when we can remove things, we should
|
[11:16] pieterh
|
always make it lighter
|
[11:16] pieterh
|
http://www.zeromq.org/docs:user-guide#toc24
|
[11:16] pieterh
|
it is 20 lines of C
|
[11:17] pieterh
|
for me, having precompiled devices actually became a barrier to understanding how they work
|
[11:18] sustrik
|
actually, yes
|
[11:18] sustrik
|
full blown devices can be stand alone projects
|
[11:18] pieterh
|
yes
|
[11:18] pieterh
|
i like this, it solves a number of things I was wondering about
|
[11:19] sustrik
|
hm, inproc devices may become an annoyance
|
[11:19] sustrik
|
rewriting same 20 lines of code over and over again
|
[11:20] pieterh
|
what's an inproc device?
|
[11:20] sustrik
|
zmq_device()
|
[11:20] sustrik
|
have a look at mutli-threaded server example
|
[11:20] pieterh
|
yes
|
[11:21] pieterh
|
what's the code we're rewriting over and over?
|
[11:21] pieterh
|
calling these inbuilt devices?
|
[11:21] sustrik
|
queue.cpp forwarder.cpp streamer.cpp
|
[11:21] pieterh
|
it's not clear to me
|
[11:22] pieterh
|
otoh it's useful to have properly engineered devices there
|
[11:22] pieterh
|
otoh, there's nothing special about these and why are they in the core?
|
[11:22] sustrik
|
exactly
|
[11:22] sustrik
|
it's clean vs. usable
|
[11:22] pieterh
|
i'd rather, for example, have a reactor in the core
|
[11:22] sustrik
|
a tradeoff
|
[11:22] pieterh
|
and then implement devices using that
|
[11:23] pieterh
|
a reactor is a generic tool
|
[11:23] sustrik
|
dunno, i have no real opinion, ask on the list
|
[11:24] pieterh
|
well, it works for now, i'd rather solve real problems
|
[11:24] sustrik
|
ok
|
[11:24] pieterh
|
if there really is an issue it'll become clear sooner or later
|
[11:25] sustrik
|
yes, solving real problems makes the design cleaner than theoretical messing with the design
|
[11:38] rbraley
|
http://www.cs.wustl.edu/~schmidt/PDF/proactor.pdf I'd prefer proactor to reactor
|
[11:53] pieterh
|
rbraley: excellent idea, I'm curious to see what you make...
|
[11:57] rbraley
|
I have an interview with Google tomorrow so I've been busy studying the best algorithms and data structures I could find. Haven't had much time to implement a proactor system for 0mq.
|
[11:57] rbraley
|
I would like to, however
|
[11:57] keffo
|
lol, everyone gets a call from google at some point :)
|
[11:58] rbraley
|
I have some pretty ambitious ideas using 0mq
|
[11:59] rbraley
|
such as implementing an octree for a game engine made of many communicating entities via the prefix of PUB/SUB connections
|
[12:01] rbraley
|
a mostly asynchronous game engine at that based on the actor model or communicating sequential processes depending on which is a better fit
|
[12:03] keffo
|
a bup/sub based distributed octree?
|
[12:04] rbraley
|
yessir keffo, a prefix will allow entities to only listen to events from nearby in their game region
|
[12:06] keffo
|
you mean over inproc I hope?
|
[12:07] rbraley
|
yes
|
[12:07] keffo
|
imo an octree is a poor choice to begin with..
|
[12:07] rbraley
|
it wouldn't have to be an octree
|
[12:08] keffo
|
gvh would fit much better
|
[12:08] keffo
|
oops, bvh
|
[12:09] keffo
|
that's besides the point though... You want to publish 'presense' for a region, or something like that?
|
[12:10] rbraley
|
presense?
|
[12:11] keffo
|
yeah, not fully clear on what you mean :)
|
[12:11] rbraley
|
every game entity would be a composition of components
|
[12:12] rbraley
|
each component would publish events
|
[12:12] keffo
|
yeah, that's what I meant, for instance "I entered this region", or something like that?
|
[12:12] rbraley
|
yeah
|
[12:14] rbraley
|
it is an idea I want to refine further, perhaps you'd be interested in helping me think it through?
|
[12:15] keffo
|
So for example (with a simple 2d grid here), an 'enemy' would subscribe to a topic which would be it's gridindex, and a 'player' would publish it's presese using the same gridindex-topic, triggerin for instance an 'attack' event in the 'enemy'? Something like that?
|
[12:15] rbraley
|
yes
|
[12:16] keffo
|
not an entirely bad idea, would remove the need for entities to have an active role
|
[12:16] rbraley
|
although a bit more circuitous
|
[12:16] keffo
|
other than pubish what they're doing, sortof
|
[12:16] rbraley
|
the AI component would subscribe to the local region looking for things to react to
|
[12:16] rbraley
|
then decide what to do based on that
|
[12:18] keffo
|
But this would again rely on the fact that the subscriber does the topicfiltering, not the publisher.. That's a major waste of resources.. If the filtering happened on the publisher, it would mean all entities would be completely idle until it reacts to something
|
[12:18] rbraley
|
entities' components would ideally do most of the publishing and subscribing for them
|
[12:19] rbraley
|
yes I'll need to take that into account
|
[12:21] keffo
|
not sure I see how the octree would fit in though
|
[12:21] rbraley
|
the entities could be completely asynchronous if they published messages with deltas of their variables and a timestamp, then the renderer could do a finite difference to intepolate a valid game state to render
|
[12:22] rbraley
|
every entity would then not need to update each "frame"
|
[12:22] keffo
|
they probably do more than just wait though
|
[12:22] keffo
|
animations and such
|
[12:23] rbraley
|
that could be hidden away in the render component
|
[12:23] rbraley
|
only when animations change would the entities themselves need to publish a change
|
[12:23] keffo
|
most game engines already treat entities in an async fashion to begin with
|
[12:23] keffo
|
the ones Ive worked on anyway
|
[12:24] keffo
|
fibers/coroutines/etc
|
[12:24] rbraley
|
most game engines don't have entities as simply compositions of components though, most of them are designed with deep inheritance hierarchies
|
[12:25] keffo
|
that's mostly a sideeffect of people insisting on inherently oo languages though :)
|
[12:26] rbraley
|
heh, it is more a side-effect of people's thinking. Composition is favored over inheritance by OO gurus anyway.
|
[12:27] pieterh
|
keffo_:, rbraley: you familiar with the http://www.bitsquid.se project?
|
[12:27] pieterh
|
different kind of game engine, maybe
|
[12:27] keffo
|
yes
|
[12:27] keffo
|
not really, it's where most engines tend to go towards
|
[12:27] pieterh
|
seems 0MQ makes a natural fabric for game engines
|
[12:28] keffo
|
valve & unreal etc, are all stuck in the quake engine swamp though
|
[12:28] rbraley
|
oooh
|
[12:28] rbraley
|
pieterh, I will dig into this
|
[12:29] pieterh
|
rbraley: i spoke to the bitsquid guys about 0MQ and they were "nah, messaging is too heavy, we'd rather do concurrency ourselves"
|
[12:30] keffo
|
lol, they already do json fiddling for that :)
|
[12:30] keffo
|
the whole industry seems to move towards msg passing for concurrency anyway
|
[12:30] keffo
|
for good reason, obviously
|
[12:30] pieterh
|
there's not a lot of choice
|
[12:30] pieterh
|
yeah
|
[12:31] keffo
|
valve employs a more worker oriented threading model now
|
[12:31] rbraley
|
hehe you really need a messaging infrastructure for more than task parallelism
|
[12:32] keffo
|
"The BitSquid engine accomplishes this by being completely job-based with no explicit thread synchronization points except the mandated frame flipping. Jobs operate on abstract data streams of simple struct data, which enables them to run transparently on co-processors through DMA transfer"
|
[12:32] keffo
|
I dont see how that could be anything other than msg passing, to be honest..
|
[12:33] rbraley
|
it's about time message passing came into use, it's been 40 years
|
[12:34] rbraley
|
and moore's law isn't a free ride anymore
|
[12:35] keffo
|
hehe no
|
[12:36] keffo
|
pieterh, Reading their blog, specifically http://bitsquid.blogspot.com/2010/03/task-management-practical-example.html, I dont understand their comment they gave to you..
|
[12:36] pieterh
|
standard NIH syndrome
|
[12:36] pieterh
|
they admitted they'd not looked at 0MQ in detail
|
[12:37] keffo
|
the game industry here isn't that big, odds are I'll run into them at some point (if I havent already, not sure)
|
[12:37] keffo
|
that blogpost isnt entirely far off from what I'm doing actually, except much more general and large scale (not purely a multithreading remedy)
|
[12:39] keffo
|
I prefer to see it as a 'distributed dependency graph of code' though :)
|
[12:39] rbraley
|
keffo_, some links to our game http://dungeonhack.sf.net http://dungeonhack.sourceforge.net/forums/viewtopic.php?f=2&t=744 http://dungeonhack.sourceforge.net/forums/viewtopic.php?f=2&t=794
|
[12:41] keffo
|
nice
|
[12:41] keffo
|
sadly, with most opensource game projects, it fails on artistry, not code :)
|
[12:41] rbraley
|
I see ZeroMQ as a large part of this, so I will probably need to make another addendum to the new game architecture once a design is worked out
|
[12:41] keffo
|
no idea why that is though
|
[12:41] pieterh
|
would either of you guys like to write a little blog post about 0MQ and game engines?
|
[12:42] rbraley
|
keffo_, I am actively recruiting artists ;)
|
[12:44] keffo
|
pieterh, I'd love to once I get more work done.. The main usage for my stuff now is a generic job framework, which I'll use to build "game-oriented" tools and plugins for modelling packages like 3dsmax/maya/zbrush etc.. not directly game engine
|
[12:44] keffo
|
currently calculating PI though :)
|
[12:45] rbraley
|
pieterh, yeah I would be interested, but I'd need to have a more solid grasp of 0MQ and how I'd use it. I'll probably have one up on the dungeonhack forums when I am ready. I would happily put it on your blog too, with appropriate edits.
|
[12:46] pieterh
|
rbraley: even if it's just speculative, it'd be nice to have some public discussion of 0MQ's potential for new generation game engines
|
[12:46] pieterh
|
np
|
[12:49] rbraley
|
yes, that sounds great. I could do that. I'll talk to you in the coming weeks and see if I can't get that done.
|
[12:51] rbraley
|
my skype is ryan.braley please add me pieterh, keffo_, and other interested parties if you have one
|
[12:51] pieterh
|
if skype would work on Android, I'd use it... :-/
|
[12:51] rbraley
|
it works on the droid
|
[12:52] keffo
|
maemo ftw! :)
|
[12:52] rbraley
|
but I don't have it for my G1
|
[12:52] rbraley
|
you could look for the .apk and see if it exists online
|
[12:52] rbraley
|
but it isn't in the market unless you have a droid or something
|
[12:53] keffo
|
skype should release a lib, so people arent forced to use that awful client
|
[12:54] rbraley
|
yeah
|
[12:58] AndrewBC
|
Mmm, I think they did release an API recently
|
[12:58] keffo
|
really?
|
[12:58] keffo
|
not the one where you need the client running at the same time?
|
[12:59] AndrewBC
|
I don't know. According to this it might be: http://developer.skype.com/accessories
|
[12:59] AndrewBC
|
Which is meh.
|
[13:00] AndrewBC
|
Oh, SkypeKit
|
[13:00] AndrewBC
|
.. in beta.
|
[13:00] AndrewBC
|
Meh.
|
[13:00] AndrewBC
|
http://developer.skype.com/public/skypekit
|
[13:02] rbraley
|
at least skype plays nice with linux... 64bit at that.
|
[13:07] rbraley
|
well if you folks don't use skype I'll just keep in touch with you here. My entire dungeonhack team uses it ;)
|
[13:08] pieterh
|
rbraley: at least this channel is public and anyone can participate
|
[13:10] rbraley
|
indeed :)
|
[13:20] rbraley
|
what does zmqircd do for you?
|
[13:29] pieterh
|
rbraley: someone (travlr, I think) logs this channel
|
[13:30] pieterh
|
sustrik: what could cause an assertion failure in rep.cpp:232?
|
[13:33] sustrik
|
let me see
|
[13:33] sustrik
|
trunk?
|
[13:33] sustrik
|
2.0.7?
|
[13:33] pieterh
|
i assume so
|
[13:34] pieterh
|
reported by David Briant on zeromq-dev, did not mention what version
|
[13:34] pieterh
|
that's the right line number on trunk, in any case
|
[13:34] sustrik
|
no assert at that line in trunk...
|
[13:35] pieterh
|
I'm looking at it now
|
[13:35] sustrik
|
aha, 2.07
|
[13:35] pieterh
|
// Get next part of the message.
|
[13:35] pieterh
|
bool fetched = in_pipes [current]->read (msg_);
|
[13:35] pieterh
|
zmq_assert (fetched);
|
[13:35] pieterh
|
maybe your recent changes on trunk shifted stuff around
|
[13:35] sustrik
|
the assert basically says:
|
[13:36] sustrik
|
after you've read the empty message part
|
[13:36] sustrik
|
(the bottom of the backtrace stack)
|
[13:36] sustrik
|
there should be at least one user-supplied message part
|
[13:36] pieterh
|
sustrik: i just updated my master snapshot and it's really line 232
|
[13:36] sustrik
|
the payload i mean
|
[13:37] pieterh
|
right
|
[13:37] pieterh
|
what could cause this message corruption in your opinion?
|
[13:37] sustrik
|
either a bug in 0MQ
|
[13:37] sustrik
|
or the user using XREQ with REP
|
[13:38] pieterh
|
we really need a better way of catching illegal socket mixes
|
[13:38] sustrik
|
and sending messages with a traceback stack but no payload
|
[13:38] pieterh
|
second one sounds most plausible, thanks
|
[13:38] sustrik
|
np
|
[15:53] keffo
|
or a wikipage listing all socket type combinations
|
[16:33] keffo
|
What's the common way to do heartbeat system?
|
[16:35] pieterh
|
mato: you back? :-)
|
[16:36] pieterh
|
keffo_: you have read http://api.zeromq.org/zmq_socket.html?
|
[16:36] pieterh
|
there are no documented examples for heartbeating afaik
|
[16:40] keffo
|
I have a weird bug, just want to know I'm on the right track before I commit to it
|
[16:42] keffo
|
there are plenty of pages on the wiki I really wish were complete. like the "failover and recovery", "a clock device", "reliable req/rep" and custom loadbalancing
|
[16:43] pieterh
|
ah, you mean in the user guide?
|
[16:43] keffo
|
yeah
|
[16:43] pieterh
|
it's a work in progress, fairly detailed work, every aspect needs working example code
|
[16:43] keffo
|
yeah :)
|
[16:43] pieterh
|
would it help if I put "Coming soon..." in every empty section?
|
[16:43] keffo
|
custom loadbalancing is quite important imo.. :)
|
[16:44] keffo
|
not really, it's quite obvious it's not there yet :)
|
[16:44] pieterh
|
"This page deliberately left blank"?
|
[16:44] pieterh
|
"Insert $$$ to see this text!"
|
[16:44] keffo
|
naa, dont waste time on that, everyone gets it´s not done yet :)
|
[16:44] pieterh
|
ah, I know... a link to somewhere to discuss that specific case
|
[16:45] pieterh
|
i want a forum for the user guide, to not pollute zeromq-dev
|
[16:45] keffo
|
mmm
|
[16:46] keffo
|
irc is a fantastic place for code talk though, probably why it doesnt die off :)
|
[16:46] pieterh
|
yeah, true
|
[16:46] pieterh
|
mato: summon... dude, you around?
|
[16:47] keffo
|
I keep trying to figure out what I did to learn stuff before the advent of internet, and I really have no idea... guess I just bruteforced everything until it worked..
|
[16:47] pieterh
|
like my son who could reconfigure the Ubuntu desktop before he was 3
|
[16:47] keffo
|
hehe :)
|
[16:47] pieterh
|
just clicked on every single option
|
[16:47] keffo
|
hehe
|
[16:48] pieterh
|
being able to ask for help does make us stupid sometimes
|
[16:48] keffo
|
I'd hardly call that configure though, more "apply entropy"
|
[16:48] pieterh
|
mato: please, why doesn't my example compile?
|
[16:49] pieterh
|
keffo_: I dunno, he was able to start youtube and find hello kitty videos
|
[16:49] pieterh
|
anyhow, i gotta take my wife to hospital, baby arriving any time soon
|
[16:49] pieterh
|
cyal
|
[16:56] mikko
|
http://gist.github.com/530821
|
[16:56] mikko
|
does the following backtrace make sense to anyone?
|
[16:57] mikko
|
php-zmq gets stuck on freebsd
|
[17:06] mikko
|
"the threading library on FreeBSD does not interact well with kqueue(); evidently, when kqueue() blocks, the entire process blocks, not just the calling thread. "
|
[17:06] mikko
|
is this still true with FreeBSD?
|
[17:07] sustrik
|
mikko: where does the backtrace come from?
|
[17:08] mikko
|
sustrik: gdb
|
[17:08] sustrik
|
ah, you are seeing it right now, right?
|
[17:08] mikko
|
yes
|
[17:08] mikko
|
the php code effectively creates new context
|
[17:09] mikko
|
and that blocks until all eternity
|
[17:09] mikko
|
just compiling things with more symbols
|
[17:09] sustrik
|
the trace is strange
|
[17:09] sustrik
|
pthread_create calling thread_kill?
|
[17:09] sustrik
|
hm
|
[17:11] mikko
|
at this point i have no idea whats going on
|
[17:11] mikko
|
will be back later ->
|
[17:11] sustrik
|
ok
|
[17:14] mikko
|
http://gist.github.com/530913
|
[17:14] mikko
|
there is with symbols
|
[17:19] sustrik
|
mikko: is it stuck
|
[17:19] sustrik
|
or does it fail?
|
[17:19] sustrik
|
ah, it's Ctrl+C
|
[17:19] sustrik
|
silly me
|
[17:20] sustrik
|
but anyway, it looks like it's stuck in pthread_create
|
[17:23] sustrik
|
hm, here's something about pthread_create blocking on freebsd:
|
[17:23] sustrik
|
http://markmail.org/message/xifffeeg7dpvu7iy
|
[17:23] sustrik
|
message back from 2004
|
[17:25] sustrik
|
some more discussion from 2008:
|
[17:25] sustrik
|
http://old.nabble.com/mysterious-hang-in-pthread_create-td19209314.html
|
[17:26] sustrik
|
can you possibly place a printf before and after pthread_create, just to see whether it is really stuck inside of it?
|
[18:12] mikko
|
sustrik: sure
|
[18:12] mikko
|
will start debugging properly soon
|
[18:13] mikko
|
just need dinner and a quick match of sc2
|
[18:21] rbraley
|
you are all smart folks. Do you know if it is possible to make an event system that triggers events based on a time (like cron) without busy waiting?
|
[18:55] mikko
|
rbraley: libev provides that
|
[18:55] mikko
|
maybe check the implementation from there
|
[18:55] rbraley
|
thanks
|
[18:58] mikko
|
sustrik: added printf debugging around line thread.cpp:66
|
[18:58] mikko
|
fprintf(stderr, "About to create\n");
|
[18:58] mikko
|
and fprintf(stderr, "Created\n");
|
[18:58] mikko
|
it seems to get stuck inside pthread_create
|
[19:37] pieterh
|
rbraley: what kind of 'events' do you want? messages?
|
[20:00] travlr
|
keffo_: you mentioned above about custom loadbalancing being important.. could you comment further on the zeromq-dev mail list thread I started to today with the title: "Load Balancing/Distributing/Queuing Algos: A Discussion"... thanks.
|
[20:20] rbraley
|
pieterh, yeah messages
|
[20:21] pieterh
|
rbraley: several things come to mind
|
[20:21] pieterh
|
first, you can use zmq_poll() with a timeout
|
[20:22] pieterh
|
this is a pretty classic way of generating timing events internally: you calculate the delay to the next event, use that as your timeout
|
[20:22] rbraley
|
doesn't poll busy wait though?
|
[20:22] pieterh
|
nope
|
[20:22] rbraley
|
oh
|
[20:23] pieterh
|
thread just relaxes until something happens
|
[20:23] pieterh
|
second, matt weinstein developed a simple protocol for timer events
|
[20:23] pieterh
|
the idea is you have, somewhere on your network, a node that uses zmq_poll() to wake up every so often and send a message to a PUB socket
|
[20:24] rbraley
|
nice!
|
[20:24] pieterh
|
so you can imagine the timer sending an event once a second
|
[20:24] pieterh
|
now you don't need to do that event time calculation everywhere
|
[20:24] pieterh
|
you just SUB to the timer stream, and include that socket in your pollset
|
[20:25] pieterh
|
now, matt's real neat trick
|
[20:25] pieterh
|
is every ms, send "1", except every 10ms, send "10", except every 100ms send "100" etc.
|
[20:25] pieterh
|
now you subscribe to "1", "10", or "100", etc.
|
[20:26] pieterh
|
giving you the resolution you need
|
[20:26] rbraley
|
nice
|
[20:26] pieterh
|
he doesn't use poll even, but usleep
|
[20:29] rbraley
|
so to use inproc:// I'd have to have it be a separate thread in my process
|
[20:30] pieterh
|
yup
|
[20:30] pieterh
|
it's a device, really
|
[20:30] rbraley
|
I don't understand the idiom of devices with 0MQ
|
[20:31] pieterh
|
did you read that section in the latest user guide?
|
[20:31] pieterh
|
i'd like to know if it's not clear on what a device is
|
[20:33] rbraley
|
I'll take a look
|
[20:33] pieterh
|
Section titled "Devil's Devices"
|
[20:35] rbraley
|
it was hard to find the user guide, it should be in the contents section of the nav bar
|
[20:38] pieterh
|
ok, i'll add it
|
[20:40] pieterh
|
done...
|
[20:41] rbraley
|
awesome!
|
[20:41] rbraley
|
I really like this user guide btw, the intro is inspiring
|
[20:42] travlr
|
yeah i have to concur with rbraley.. you've done a really nice job with it pieter
|
[20:44] travlr
|
btw, while i'm thinking about it... it would be nice if each zmq symbol could have a complete list of "properties" listed. this is more for mato's ref manual i guess
|
[20:47] pieterh
|
'inspiring'? wow...
|
[20:47] pieterh
|
travlr: what do you mean by 'symbol'?
|
[20:49] travlr
|
pieterh, well the different socket enums like ZMQ_PUSH, etc. should be more detailed as to what is under the hood. I still am trying to figure out details of what's under the hood for different zmq infrastructure.
|
[20:49] pieterh
|
ah, i get it
|
[20:49] pieterh
|
a glossary :-)
|
[20:50] pieterh
|
great idea, i like it
|
[20:51] pieterh
|
the man pages are pretty detailed about the socket types, for example (except they say DOWNSTREAM where the Guide says PUSH)
|
[20:51] travlr
|
well, not so much a glossary but a list of properties that define all the aspects of a particular symbol. there's so much beautiful magic under the hood, and mato's work is great but no complete, imo.
|
[20:51] pieterh
|
imo it is complete but too brief
|
[20:52] travlr
|
under the hood magic should be explicit in the ref
|
[20:52] pieterh
|
do you have an example?
|
[20:53] pieterh
|
i've not found anything missing from the reference manual, except xrep/xreq, and the multipart stack stuff
|
[20:54] travlr
|
let me look at it again and i'll expand my thought then, ok
|
[20:54] pieterh
|
if you hit things that are not explained, or poorly, just shout
|
[20:54] travlr
|
ok
|
[20:59] keffo
|
travlr, I'll read it tomorrow, quite late here now :)
|
[21:00] travlr
|
keffo_: thanks
|
[21:10] rbraley
|
pieterh, I read the section on devices but conceptually they remain murky for me
|
[21:10] travlr
|
pieterh: mato has done more work to the ref since i saw it last.. its consice and that's good. the only thing i can add right now, in general is... think of any magic that zmq does behind the scenes and if its not included there.. it should be listed there, imo.
|
[21:11] pieterh
|
travlr: yes, it should be complete
|
[21:11] pieterh
|
rbraley: they remained murky to me until I wrote one
|
[21:12] pieterh
|
they're just applications that you embed into the network
|
[21:12] rbraley
|
Devices are anything that sit between your real applications. this isn
|
[21:12] pieterh
|
so reusable, stateless, part of the fabric
|
[21:12] pieterh
|
think of network switches that understand 0MQ patterns
|
[21:12] rbraley
|
't really enough of a one-liner zinger to explain them
|
[21:13] pieterh
|
message-switching networks
|
[21:13] rbraley
|
I got that analogy
|
[21:13] rbraley
|
but I can't help but think that devices are more general than that
|
[21:14] pieterh
|
it's not a tight classification yet, over time i suspect it will become more so
|
[21:14] pieterh
|
it's about layers, imo
|
[21:14] pieterh
|
zmq core is one layer
|
[21:14] pieterh
|
on top of that you have devices
|
[21:14] pieterh
|
and on top of those two layers you have applications
|
[21:15] pieterh
|
if you make an app that is reusable and generic, it's potentially a device
|
[21:16] rbraley
|
perhaps I should think of devices as unix commands and 0MQ sockets as pipes
|
[21:16] pieterh
|
that kind of works, yes
|
[21:16] pieterh
|
filters, in Unix terms
|
[21:16] rbraley
|
right
|
[21:16] pieterh
|
except there are no pipes
|
[21:16] pieterh
|
:-)
|
[21:17] pieterh
|
i'm kidding, i mean there are no visible connections
|
[21:17] pieterh
|
and sockets are what we use to talk to the network, it's not the actual network
|
[21:18] rbraley
|
I was thinking that maybe the components of my game engine might be devices
|
[21:19] rbraley
|
with entities just being collections of sockets to them
|
[21:20] rbraley
|
a timer device would certainly help
|
[21:21] rbraley
|
no that doesn't work if devices should be stateless
|
[21:25] rbraley
|
ooh, a logging device would be neat
|
[21:28] pieterh
|
rbraley: that would be a logging application, not device >:-)
|
[21:28] pieterh
|
just kidding...
|
[21:28] pieterh
|
'device' really are things you plug into the network, and logging device makes perfect sense afaics
|
[21:30] rbraley
|
the thing that irks me about these examples is that threading policy is not decoupled from the concurrency policy
|
[21:31] rbraley
|
ideally you'd have just one thread per core and could multiplex everything to these threads
|
[21:33] CIA-20
|
jzmq: 03Alexey Ermakov 07master * r2f3afe8 10/ (perf/local_thr.java src/Socket.cpp src/org/zeromq/ZMQ.java): Changed setSubscribe(), setUnsubscribe() and setIdentity() to accept byte[]'s instead of Strings. - http://bit.ly/b2AZrL
|
[21:38] travlr
|
i'm beginning to realize that zmq is a 'stateless' model and that state where belongs is the app.. as said: not even in a custom device. i, on the other hand use a device on each box _for_ the state of the individual zmq endpoint components..
|
[21:38] travlr
|
s/state where/where state/ :-P
|
[21:41] rbraley
|
without decoupling the threading policy from the concurrency policy it isn't possible to scale 0MQ to model neurons or implement the actor model
|
[21:42] travlr
|
there are two threading aspects in zmq, afaik, the i/o threading and the app threading..
|
[21:43] travlr
|
and if i understand your comment correctly the app threading is decoupled
|
[21:44] travlr
|
from the context but not a socket... but that is to be changed soon, afaiu
|
[21:44] rbraley
|
What I am saying is the actual number concurrent nodes in the app should exceed the number of app threads
|
[21:45] rbraley
|
we can't have a thread per neuron for example
|
[21:45] pieterh
|
rbraley: right, if you say one node is one thread
|
[21:46] rbraley
|
the concurrent nodes should be multiplexed to the app threads
|
[21:47] rbraley
|
so 400000 of them can be run on 8 threads for example
|
[21:48] rbraley
|
when you can do that, the entire paradigm changes
|
[21:48] travlr
|
i'm thinking that is doable
|
[21:49] travlr
|
and thats probably where a 'device' fits into the scheme
|
[21:49] rbraley
|
suddenly you are dealing with abstract flow based programs that are pure computation and run optimally on any hardware that you put them on.
|
[21:50] travlr
|
ha! have you seen my ProDataLab project regarding that
|
[21:50] rbraley
|
it becomes a thing of great beauty and efficiency
|
[21:50] rbraley
|
no sir
|
[21:50] travlr
|
www.prodatalab.wikidot.com
|
[21:52] rbraley
|
that is pure sex, travlr
|
[21:52] travlr
|
lol, it is beautiful
|
[21:53] rbraley
|
ok we need to talk
|
[21:53] rbraley
|
where is this architecture spec?
|
[21:53] travlr
|
please! when ever you like
|
[21:53] travlr
|
spec is in my head atm
|
[21:54] rbraley
|
let's get that on paper
|
[21:54] travlr
|
let's talk sometime then
|
[21:54] rbraley
|
well not deadtree, you know what I mean ;)
|
[21:54] travlr
|
right
|
[21:55] travlr
|
i need a partner so look it over carefully and let me know if you are interested in developing it with me
|
[21:55] rbraley
|
we were planning on using a similar system for our artists to create entities in our game engine without knowing C++ or Go
|
[21:56] travlr
|
i don't understand why visual programming isn't more wide spread.
|
[21:56] rbraley
|
because, there isn't a conceptual model that works universally that people understand yet
|
[21:56] rbraley
|
I think this is it
|
[21:57] travlr
|
who difficult is a flow chart... directed graphs
|
[21:57] rbraley
|
marketing as well. Most people still think procedurally
|
[21:57] travlr
|
s/who/how
|
[21:57] travlr
|
yeah but for the user newbie
|
[21:57] travlr
|
vp is ideal
|
[21:58] travlr
|
the power of programming in the hands of a layman
|
[21:58] rbraley
|
so the optimal way to discuss this -- probably a recorded voice chat?
|
[21:58] CIA-20
|
jzmq: 03Gonzalo Diethelm 07master * r773e56f 10/ (perf/local_thr.java src/Socket.cpp src/org/zeromq/ZMQ.java):
|
[21:58] CIA-20
|
jzmq: Handle an error case when creating a byte array in C++ code.
|
[21:58] CIA-20
|
jzmq: Changed the names of some Socket methods:
|
[21:58] CIA-20
|
jzmq: getMulticastLoop() => hasMulticastLoop()
|
[21:58] CIA-20
|
jzmq: getReceiveMore() => hasReceiveMore()
|
[21:58] CIA-20
|
jzmq: setSubscribe(byte[] subscribe) => subscribe(byte[] topic)
|
[21:58] CIA-20
|
jzmq: setUnsubscribe(byte[] unsubscribe) => unsubscribe(byte[] topic) - http://bit.ly/aOecvc
|
[21:59] travlr
|
what ever works for you is works for me
|
[22:01] rbraley
|
lemme see if i can get a skype call recorder working
|
[22:03] travlr
|
what's the need for recording?
|
[22:03] rbraley
|
so we can transcribe and turn it into a spec later
|
[22:03] rbraley
|
also if we happen upon a great idea we won't lose it and can return to it later
|
[22:04] travlr
|
sure ok, but before we get into details, we should just have a chat first
|
[22:07] rbraley
|
what is your skype id travlr ?
|
[22:10] travlr
|
hold on a min
|
[22:13] rbraley
|
velaccel is my skype id
|
[22:13] travlr
|
did you get it?
|
[22:18] pieterh
|
travlr: so you are Mr ProDataLab... :-)
|