IRC Log

Tuesday August 17, 2010

[Time] NameMessage
[01:47] sustrik pieterh: there's a bug there
[01:47] sustrik fix is on the mailing list
[01:47] sustrik sent by jon dyte yesterdat
[01:47] sustrik yesterday
[01:48] jhawk28 hello, does anyone know if the Java bindings can compile on the mac?
[01:49] jhawk28 I'm getting this error: http://pastebin.com/Vng9YCNn
[08:43] pieterh sustrik: just saw that now, it didn't register...
[08:43] sustrik ok
[09:42] shekispeaks hi i am trying to install the Java Bindings on my mac
[09:42] shekispeaks a
[09:42] shekispeaks the ./configure always says JAVA_HOME not set
[09:42] shekispeaks i already have it set
[09:42] shekispeaks I am using java 1.6+
[09:48] sustrik skekispeaks: try to locate jni.h on your box
[09:48] sustrik where does it reside?
[09:49] shekispeaks should i point JAVA_HOME to that directory or its parent?
[09:49] sustrik its parent
[09:49] shekispeaks "/System/Library/Frameworks/JavaVM.framework/Home/include
[09:50] sustrik point it to Home
[09:50] shekispeaks ya JAVA_HOME is already set to Home still the config always throws the same error
[09:50] sustrik what's exactly the error?
[09:51] sustrik there's this in the configure.in:
[09:51] sustrik if test "x$JAVA_HOME" = "x"; then
[09:51] sustrik AC_MSG_ERROR([the JAVA_HOME environment variable must be set to your JDK location.]);
[09:51] sustrik fi
[09:52] sustrik so if that's the message you are seeing
[09:52] sustrik it looks like $JAVA_HOME resolves into empty string
[09:53] shekispeaks abhishek:jzmq abhishekk$ echo $JAVA_HOME
[09:54] shekispeaks "/System/Library/Frameworks/JavaVM.framework/Home"
[09:54] shekispeaks this is the variable
[09:54] shekispeaks and the next command is ./configure
[09:55] shekispeaks it ends in configure: error: the JAVA_HOME environment variable must be set to your JDK location.
[09:56] shekispeaks the variable is set
[10:10] pieterh shekispeaks: try 'export JAVA_HOME'
[10:10] shekispeaks i have exported it
[10:11] pieterh I'd suggest: edit configure script to echo $JAVA_HOME or 'set > vars' before error message
[10:11] shekispeaks abhishek:pyzmq abhishekk$ export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home
[10:11] shekispeaks abhishek:pyzmq abhishekk$ echo $JAVA_HOME
[10:11] shekispeaks "/System/Library/Frameworks/JavaVM.framework/Home"
[10:12] pieterh shekispeaks: not sure you can do export like that, try 'set JAVA_HOME=...; export JAVA_HOME'
[10:14] pieterh sustrik: is there any valid semantic in doing a blocking recv on a SUB socket with zero subscriptions?
[10:15] sustrik yes
[10:15] sustrik waiting for ETERM
[10:16] shekispeaks both set export does not work
[10:16] shekispeaks even though the variable is set in the shell
[10:16] shekispeaks it is printing a null in the script
[10:17] sustrik what about JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Home ./configure
[10:17] sustrik JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Home" ./configure
[10:17] pieterh shekispeaks: Darwin weirdness... are you using any non-standard shell?
[10:18] pieterh sustrik: isn't that an abuse of the SUB semantics? If you want to wait for ETERM, surely there are more explicit ways
[10:18] pieterh such as bind to private inproc endpoint and recv on it
[10:20] shekispeaks ok i just set the variable in the config script and it ran
[10:21] sustrik pieter: yes, you are right
[10:21] sustrik waiting for ETRM using recv is silly
[10:21] pieterh sustrik: i'd like to propose that blocking recv on SUB with no subscriptions is a semantic error -> assertion
[10:22] pieterh not even worth returning an error code, an app that does this should fail quickly
[10:22] pieterh let me throw this suggestion to the dev list...
[10:56] pieterh sustrik: the XML parser in foreign, it's only for devices, right?
[11:05] sustrik pieterh: yes, not used elsewhere so far
[11:05] pieterh i'd like to use libconfig as a 'standard' way of reading config files
[11:05] sustrik what's that?
[11:06] pieterh its a pretty standard library for parsing config files
[11:06] pieterh http://www.hyperrealm.com/libconfig/libconfig_manual.html
[11:06] pieterh the 0MQ of config files
[11:06] pieterh it would absolve us of having that foreign package
[11:07] pieterh i want to document how to use that to configure a device process
[11:08] sustrik for version 3.0?
[11:08] pieterh advantage of libconfig is lots of language support and standard package on many linuxes
[11:08] pieterh for version 2.1, I think
[11:08] pieterh devices were not documented up to now
[11:08] sustrik two problems i see:
[11:08] sustrik 1. one more dependency
[11:09] sustrik what platforms it is supported on?
[11:09] sustrik you have to check
[11:09] pieterh 1. yes, but only for main devices
[11:09] pieterh 2. pretty much everything
[11:09] sustrik OpenVMS?
[11:09] sustrik 2. it breaks backward compatibility, so be careful
[11:09] pieterh The library runs on modern POSIX-compilant systems, such as Linux, Solaris, and Mac OS X (Darwin), as well as on Microsoft Windows 2000/XP and later
[11:10] pieterh 2. we can leave the XML library in there until 3.0, for sure
[11:10] pieterh but it's a long term maintenance cost
[11:10] sustrik it was never changed since it was included
[11:10] pieterh yes, it was
[11:11] pieterh i made several security fixes
[11:11] sustrik i had to revert your changes
[11:11] pieterh ah
[11:11] sustrik they broke win32 build
[11:11] pieterh kind of proves my poibnt
[11:11] pieterh *point
[11:11] pieterh anyhow, i don't like xml config files (after using them for years!)
[11:11] sustrik as i said, it's up to
[11:11] pieterh any non-trivial 0MQ application will have dependencies
[11:12] sustrik if you don't care about exotic platforms and possible dependency problems, just do ti
[11:12] pieterh in fact... i'm thinking we don't need standard external device programs
[11:12] pieterh just zmq_device(3) and decent docs on how to write your own
[11:12] pieterh so I'd like to make a tutorial on how to write devices
[11:13] pieterh (already have most of that)
[11:13] pieterh in the guide
[11:13] sustrik yes, it's convenience stuff
[11:13] sustrik ok
[11:13] pieterh ok, great
[11:14] sustrik we can for example split pre-compiled into a separate project
[11:14] pieterh i think it's important that 0MQ users understand the internals of devices rapidly
[11:14] pieterh and write their own
[11:14] sustrik leaving 0mq to be what it is supposed to be, i.e. library
[11:14] pieterh indeed
[11:14] pieterh that turns zmq_deviced(1) into a worked example
[11:14] pieterh and template code that's trivial to reuse
[11:15] pieterh we don't need a separate project at all ... if you build the Guide you get these examples running
[11:15] pieterh it's all on github now
[11:15] pieterh this simplifies the packaging
[11:15] sustrik it's up to you
[11:15] sustrik precompiled devices may be handy sometimes though
[11:15] pieterh for me, when we can remove things, we should
[11:16] pieterh always make it lighter
[11:16] pieterh http://www.zeromq.org/docs:user-guide#toc24
[11:16] pieterh it is 20 lines of C
[11:17] pieterh for me, having precompiled devices actually became a barrier to understanding how they work
[11:18] sustrik actually, yes
[11:18] sustrik full blown devices can be stand alone projects
[11:18] pieterh yes
[11:18] pieterh i like this, it solves a number of things I was wondering about
[11:19] sustrik hm, inproc devices may become an annoyance
[11:19] sustrik rewriting same 20 lines of code over and over again
[11:20] pieterh what's an inproc device?
[11:20] sustrik zmq_device()
[11:20] sustrik have a look at mutli-threaded server example
[11:20] pieterh yes
[11:21] pieterh what's the code we're rewriting over and over?
[11:21] pieterh calling these inbuilt devices?
[11:21] sustrik queue.cpp forwarder.cpp streamer.cpp
[11:21] pieterh it's not clear to me
[11:22] pieterh otoh it's useful to have properly engineered devices there
[11:22] pieterh otoh, there's nothing special about these and why are they in the core?
[11:22] sustrik exactly
[11:22] sustrik it's clean vs. usable
[11:22] pieterh i'd rather, for example, have a reactor in the core
[11:22] sustrik a tradeoff
[11:22] pieterh and then implement devices using that
[11:23] pieterh a reactor is a generic tool
[11:23] sustrik dunno, i have no real opinion, ask on the list
[11:24] pieterh well, it works for now, i'd rather solve real problems
[11:24] sustrik ok
[11:24] pieterh if there really is an issue it'll become clear sooner or later
[11:25] sustrik yes, solving real problems makes the design cleaner than theoretical messing with the design
[11:38] rbraley http://www.cs.wustl.edu/~schmidt/PDF/proactor.pdf I'd prefer proactor to reactor
[11:53] pieterh rbraley: excellent idea, I'm curious to see what you make...
[11:57] rbraley I have an interview with Google tomorrow so I've been busy studying the best algorithms and data structures I could find. Haven't had much time to implement a proactor system for 0mq.
[11:57] rbraley I would like to, however
[11:57] keffo lol, everyone gets a call from google at some point :)
[11:58] rbraley I have some pretty ambitious ideas using 0mq
[11:59] rbraley such as implementing an octree for a game engine made of many communicating entities via the prefix of PUB/SUB connections
[12:01] rbraley a mostly asynchronous game engine at that based on the actor model or communicating sequential processes depending on which is a better fit
[12:03] keffo a bup/sub based distributed octree?
[12:04] rbraley yessir keffo, a prefix will allow entities to only listen to events from nearby in their game region
[12:06] keffo you mean over inproc I hope?
[12:07] rbraley yes
[12:07] keffo imo an octree is a poor choice to begin with..
[12:07] rbraley it wouldn't have to be an octree
[12:08] keffo gvh would fit much better
[12:08] keffo oops, bvh
[12:09] keffo that's besides the point though... You want to publish 'presense' for a region, or something like that?
[12:10] rbraley presense?
[12:11] keffo yeah, not fully clear on what you mean :)
[12:11] rbraley every game entity would be a composition of components
[12:12] rbraley each component would publish events
[12:12] keffo yeah, that's what I meant, for instance "I entered this region", or something like that?
[12:12] rbraley yeah
[12:14] rbraley it is an idea I want to refine further, perhaps you'd be interested in helping me think it through?
[12:15] keffo So for example (with a simple 2d grid here), an 'enemy' would subscribe to a topic which would be it's gridindex, and a 'player' would publish it's presese using the same gridindex-topic, triggerin for instance an 'attack' event in the 'enemy'? Something like that?
[12:15] rbraley yes
[12:16] keffo not an entirely bad idea, would remove the need for entities to have an active role
[12:16] rbraley although a bit more circuitous
[12:16] keffo other than pubish what they're doing, sortof
[12:16] rbraley the AI component would subscribe to the local region looking for things to react to
[12:16] rbraley then decide what to do based on that
[12:18] keffo But this would again rely on the fact that the subscriber does the topicfiltering, not the publisher.. That's a major waste of resources.. If the filtering happened on the publisher, it would mean all entities would be completely idle until it reacts to something
[12:18] rbraley entities' components would ideally do most of the publishing and subscribing for them
[12:19] rbraley yes I'll need to take that into account
[12:21] keffo not sure I see how the octree would fit in though
[12:21] rbraley the entities could be completely asynchronous if they published messages with deltas of their variables and a timestamp, then the renderer could do a finite difference to intepolate a valid game state to render
[12:22] rbraley every entity would then not need to update each "frame"
[12:22] keffo they probably do more than just wait though
[12:22] keffo animations and such
[12:23] rbraley that could be hidden away in the render component
[12:23] rbraley only when animations change would the entities themselves need to publish a change
[12:23] keffo most game engines already treat entities in an async fashion to begin with
[12:23] keffo the ones Ive worked on anyway
[12:24] keffo fibers/coroutines/etc
[12:24] rbraley most game engines don't have entities as simply compositions of components though, most of them are designed with deep inheritance hierarchies
[12:25] keffo that's mostly a sideeffect of people insisting on inherently oo languages though :)
[12:26] rbraley heh, it is more a side-effect of people's thinking. Composition is favored over inheritance by OO gurus anyway.
[12:27] pieterh keffo_:, rbraley: you familiar with the http://www.bitsquid.se project?
[12:27] pieterh different kind of game engine, maybe
[12:27] keffo yes
[12:27] keffo not really, it's where most engines tend to go towards
[12:27] pieterh seems 0MQ makes a natural fabric for game engines
[12:28] keffo valve & unreal etc, are all stuck in the quake engine swamp though
[12:28] rbraley oooh
[12:28] rbraley pieterh, I will dig into this
[12:29] pieterh rbraley: i spoke to the bitsquid guys about 0MQ and they were "nah, messaging is too heavy, we'd rather do concurrency ourselves"
[12:30] keffo lol, they already do json fiddling for that :)
[12:30] keffo the whole industry seems to move towards msg passing for concurrency anyway
[12:30] keffo for good reason, obviously
[12:30] pieterh there's not a lot of choice
[12:30] pieterh yeah
[12:31] keffo valve employs a more worker oriented threading model now
[12:31] rbraley hehe you really need a messaging infrastructure for more than task parallelism
[12:32] keffo "The BitSquid engine accomplishes this by being completely job-based with no explicit thread synchronization points except the mandated frame flipping. Jobs operate on abstract data streams of simple struct data, which enables them to run transparently on co-processors through DMA transfer"
[12:32] keffo I dont see how that could be anything other than msg passing, to be honest..
[12:33] rbraley it's about time message passing came into use, it's been 40 years
[12:34] rbraley and moore's law isn't a free ride anymore
[12:35] keffo hehe no
[12:36] keffo pieterh, Reading their blog, specifically http://bitsquid.blogspot.com/2010/03/task-management-practical-example.html, I dont understand their comment they gave to you..
[12:36] pieterh standard NIH syndrome
[12:36] pieterh they admitted they'd not looked at 0MQ in detail
[12:37] keffo the game industry here isn't that big, odds are I'll run into them at some point (if I havent already, not sure)
[12:37] keffo that blogpost isnt entirely far off from what I'm doing actually, except much more general and large scale (not purely a multithreading remedy)
[12:39] keffo I prefer to see it as a 'distributed dependency graph of code' though :)
[12:39] rbraley keffo_, some links to our game http://dungeonhack.sf.net http://dungeonhack.sourceforge.net/forums/viewtopic.php?f=2&t=744 http://dungeonhack.sourceforge.net/forums/viewtopic.php?f=2&t=794
[12:41] keffo nice
[12:41] keffo sadly, with most opensource game projects, it fails on artistry, not code :)
[12:41] rbraley I see ZeroMQ as a large part of this, so I will probably need to make another addendum to the new game architecture once a design is worked out
[12:41] keffo no idea why that is though
[12:41] pieterh would either of you guys like to write a little blog post about 0MQ and game engines?
[12:42] rbraley keffo_, I am actively recruiting artists ;)
[12:44] keffo pieterh, I'd love to once I get more work done.. The main usage for my stuff now is a generic job framework, which I'll use to build "game-oriented" tools and plugins for modelling packages like 3dsmax/maya/zbrush etc.. not directly game engine
[12:44] keffo currently calculating PI though :)
[12:45] rbraley pieterh, yeah I would be interested, but I'd need to have a more solid grasp of 0MQ and how I'd use it. I'll probably have one up on the dungeonhack forums when I am ready. I would happily put it on your blog too, with appropriate edits.
[12:46] pieterh rbraley: even if it's just speculative, it'd be nice to have some public discussion of 0MQ's potential for new generation game engines
[12:46] pieterh np
[12:49] rbraley yes, that sounds great. I could do that. I'll talk to you in the coming weeks and see if I can't get that done.
[12:51] rbraley my skype is ryan.braley please add me pieterh, keffo_, and other interested parties if you have one
[12:51] pieterh if skype would work on Android, I'd use it... :-/
[12:51] rbraley it works on the droid
[12:52] keffo maemo ftw! :)
[12:52] rbraley but I don't have it for my G1
[12:52] rbraley you could look for the .apk and see if it exists online
[12:52] rbraley but it isn't in the market unless you have a droid or something
[12:53] keffo skype should release a lib, so people arent forced to use that awful client
[12:54] rbraley yeah
[12:58] AndrewBC Mmm, I think they did release an API recently
[12:58] keffo really?
[12:58] keffo not the one where you need the client running at the same time?
[12:59] AndrewBC I don't know. According to this it might be: http://developer.skype.com/accessories
[12:59] AndrewBC Which is meh.
[13:00] AndrewBC Oh, SkypeKit
[13:00] AndrewBC .. in beta.
[13:00] AndrewBC Meh.
[13:00] AndrewBC http://developer.skype.com/public/skypekit
[13:02] rbraley at least skype plays nice with linux... 64bit at that.
[13:07] rbraley well if you folks don't use skype I'll just keep in touch with you here. My entire dungeonhack team uses it ;)
[13:08] pieterh rbraley: at least this channel is public and anyone can participate
[13:10] rbraley indeed :)
[13:20] rbraley what does zmqircd do for you?
[13:29] pieterh rbraley: someone (travlr, I think) logs this channel
[13:30] pieterh sustrik: what could cause an assertion failure in rep.cpp:232?
[13:33] sustrik let me see
[13:33] sustrik trunk?
[13:33] sustrik 2.0.7?
[13:33] pieterh i assume so
[13:34] pieterh reported by David Briant on zeromq-dev, did not mention what version
[13:34] pieterh that's the right line number on trunk, in any case
[13:34] sustrik no assert at that line in trunk...
[13:35] pieterh I'm looking at it now
[13:35] sustrik aha, 2.07
[13:35] pieterh // Get next part of the message.
[13:35] pieterh bool fetched = in_pipes [current]->read (msg_);
[13:35] pieterh zmq_assert (fetched);
[13:35] pieterh maybe your recent changes on trunk shifted stuff around
[13:35] sustrik the assert basically says:
[13:36] sustrik after you've read the empty message part
[13:36] sustrik (the bottom of the backtrace stack)
[13:36] sustrik there should be at least one user-supplied message part
[13:36] pieterh sustrik: i just updated my master snapshot and it's really line 232
[13:36] sustrik the payload i mean
[13:37] pieterh right
[13:37] pieterh what could cause this message corruption in your opinion?
[13:37] sustrik either a bug in 0MQ
[13:37] sustrik or the user using XREQ with REP
[13:38] pieterh we really need a better way of catching illegal socket mixes
[13:38] sustrik and sending messages with a traceback stack but no payload
[13:38] pieterh second one sounds most plausible, thanks
[13:38] sustrik np
[15:53] keffo or a wikipage listing all socket type combinations
[16:33] keffo What's the common way to do heartbeat system?
[16:35] pieterh mato: you back? :-)
[16:36] pieterh keffo_: you have read http://api.zeromq.org/zmq_socket.html?
[16:36] pieterh there are no documented examples for heartbeating afaik
[16:40] keffo I have a weird bug, just want to know I'm on the right track before I commit to it
[16:42] keffo there are plenty of pages on the wiki I really wish were complete. like the "failover and recovery", "a clock device", "reliable req/rep" and custom loadbalancing
[16:43] pieterh ah, you mean in the user guide?
[16:43] keffo yeah
[16:43] pieterh it's a work in progress, fairly detailed work, every aspect needs working example code
[16:43] keffo yeah :)
[16:43] pieterh would it help if I put "Coming soon..." in every empty section?
[16:43] keffo custom loadbalancing is quite important imo.. :)
[16:44] keffo not really, it's quite obvious it's not there yet :)
[16:44] pieterh "This page deliberately left blank"?
[16:44] pieterh "Insert $$$ to see this text!"
[16:44] keffo naa, dont waste time on that, everyone gets it´s not done yet :)
[16:44] pieterh ah, I know... a link to somewhere to discuss that specific case
[16:45] pieterh i want a forum for the user guide, to not pollute zeromq-dev
[16:45] keffo mmm
[16:46] keffo irc is a fantastic place for code talk though, probably why it doesnt die off :)
[16:46] pieterh yeah, true
[16:46] pieterh mato: summon... dude, you around?
[16:47] keffo I keep trying to figure out what I did to learn stuff before the advent of internet, and I really have no idea... guess I just bruteforced everything until it worked..
[16:47] pieterh like my son who could reconfigure the Ubuntu desktop before he was 3
[16:47] keffo hehe :)
[16:47] pieterh just clicked on every single option
[16:47] keffo hehe
[16:48] pieterh being able to ask for help does make us stupid sometimes
[16:48] keffo I'd hardly call that configure though, more "apply entropy"
[16:48] pieterh mato: please, why doesn't my example compile?
[16:49] pieterh keffo_: I dunno, he was able to start youtube and find hello kitty videos
[16:49] pieterh anyhow, i gotta take my wife to hospital, baby arriving any time soon
[16:49] pieterh cyal
[16:56] mikko http://gist.github.com/530821
[16:56] mikko does the following backtrace make sense to anyone?
[16:57] mikko php-zmq gets stuck on freebsd
[17:06] mikko "the threading library on FreeBSD does not interact well with kqueue(); evidently, when kqueue() blocks, the entire process blocks, not just the calling thread. "
[17:06] mikko is this still true with FreeBSD?
[17:07] sustrik mikko: where does the backtrace come from?
[17:08] mikko sustrik: gdb
[17:08] sustrik ah, you are seeing it right now, right?
[17:08] mikko yes
[17:08] mikko the php code effectively creates new context
[17:09] mikko and that blocks until all eternity
[17:09] mikko just compiling things with more symbols
[17:09] sustrik the trace is strange
[17:09] sustrik pthread_create calling thread_kill?
[17:09] sustrik hm
[17:11] mikko at this point i have no idea whats going on
[17:11] mikko will be back later ->
[17:11] sustrik ok
[17:14] mikko http://gist.github.com/530913
[17:14] mikko there is with symbols
[17:19] sustrik mikko: is it stuck
[17:19] sustrik or does it fail?
[17:19] sustrik ah, it's Ctrl+C
[17:19] sustrik silly me
[17:20] sustrik but anyway, it looks like it's stuck in pthread_create
[17:23] sustrik hm, here's something about pthread_create blocking on freebsd:
[17:23] sustrik http://markmail.org/message/xifffeeg7dpvu7iy
[17:23] sustrik message back from 2004
[17:25] sustrik some more discussion from 2008:
[17:25] sustrik http://old.nabble.com/mysterious-hang-in-pthread_create-td19209314.html
[17:26] sustrik can you possibly place a printf before and after pthread_create, just to see whether it is really stuck inside of it?
[18:12] mikko sustrik: sure
[18:12] mikko will start debugging properly soon
[18:13] mikko just need dinner and a quick match of sc2
[18:21] rbraley you are all smart folks. Do you know if it is possible to make an event system that triggers events based on a time (like cron) without busy waiting?
[18:55] mikko rbraley: libev provides that
[18:55] mikko maybe check the implementation from there
[18:55] rbraley thanks
[18:58] mikko sustrik: added printf debugging around line thread.cpp:66
[18:58] mikko fprintf(stderr, "About to create\n");
[18:58] mikko and fprintf(stderr, "Created\n");
[18:58] mikko it seems to get stuck inside pthread_create
[19:37] pieterh rbraley: what kind of 'events' do you want? messages?
[20:00] travlr keffo_: you mentioned above about custom loadbalancing being important.. could you comment further on the zeromq-dev mail list thread I started to today with the title: "Load Balancing/Distributing/Queuing Algos: A Discussion"... thanks.
[20:20] rbraley pieterh, yeah messages
[20:21] pieterh rbraley: several things come to mind
[20:21] pieterh first, you can use zmq_poll() with a timeout
[20:22] pieterh this is a pretty classic way of generating timing events internally: you calculate the delay to the next event, use that as your timeout
[20:22] rbraley doesn't poll busy wait though?
[20:22] pieterh nope
[20:22] rbraley oh
[20:23] pieterh thread just relaxes until something happens
[20:23] pieterh second, matt weinstein developed a simple protocol for timer events
[20:23] pieterh the idea is you have, somewhere on your network, a node that uses zmq_poll() to wake up every so often and send a message to a PUB socket
[20:24] rbraley nice!
[20:24] pieterh so you can imagine the timer sending an event once a second
[20:24] pieterh now you don't need to do that event time calculation everywhere
[20:24] pieterh you just SUB to the timer stream, and include that socket in your pollset
[20:25] pieterh now, matt's real neat trick
[20:25] pieterh is every ms, send "1", except every 10ms, send "10", except every 100ms send "100" etc.
[20:25] pieterh now you subscribe to "1", "10", or "100", etc.
[20:26] pieterh giving you the resolution you need
[20:26] rbraley nice
[20:26] pieterh he doesn't use poll even, but usleep
[20:29] rbraley so to use inproc:// I'd have to have it be a separate thread in my process
[20:30] pieterh yup
[20:30] pieterh it's a device, really
[20:30] rbraley I don't understand the idiom of devices with 0MQ
[20:31] pieterh did you read that section in the latest user guide?
[20:31] pieterh i'd like to know if it's not clear on what a device is
[20:33] rbraley I'll take a look
[20:33] pieterh Section titled "Devil's Devices"
[20:35] rbraley it was hard to find the user guide, it should be in the contents section of the nav bar
[20:38] pieterh ok, i'll add it
[20:40] pieterh done...
[20:41] rbraley awesome!
[20:41] rbraley I really like this user guide btw, the intro is inspiring
[20:42] travlr yeah i have to concur with rbraley.. you've done a really nice job with it pieter
[20:44] travlr btw, while i'm thinking about it... it would be nice if each zmq symbol could have a complete list of "properties" listed. this is more for mato's ref manual i guess
[20:47] pieterh 'inspiring'? wow...
[20:47] pieterh travlr: what do you mean by 'symbol'?
[20:49] travlr pieterh, well the different socket enums like ZMQ_PUSH, etc. should be more detailed as to what is under the hood. I still am trying to figure out details of what's under the hood for different zmq infrastructure.
[20:49] pieterh ah, i get it
[20:49] pieterh a glossary :-)
[20:50] pieterh great idea, i like it
[20:51] pieterh the man pages are pretty detailed about the socket types, for example (except they say DOWNSTREAM where the Guide says PUSH)
[20:51] travlr well, not so much a glossary but a list of properties that define all the aspects of a particular symbol. there's so much beautiful magic under the hood, and mato's work is great but no complete, imo.
[20:51] pieterh imo it is complete but too brief
[20:52] travlr under the hood magic should be explicit in the ref
[20:52] pieterh do you have an example?
[20:53] pieterh i've not found anything missing from the reference manual, except xrep/xreq, and the multipart stack stuff
[20:54] travlr let me look at it again and i'll expand my thought then, ok
[20:54] pieterh if you hit things that are not explained, or poorly, just shout
[20:54] travlr ok
[20:59] keffo travlr, I'll read it tomorrow, quite late here now :)
[21:00] travlr keffo_: thanks
[21:10] rbraley pieterh, I read the section on devices but conceptually they remain murky for me
[21:10] travlr pieterh: mato has done more work to the ref since i saw it last.. its consice and that's good. the only thing i can add right now, in general is... think of any magic that zmq does behind the scenes and if its not included there.. it should be listed there, imo.
[21:11] pieterh travlr: yes, it should be complete
[21:11] pieterh rbraley: they remained murky to me until I wrote one
[21:12] pieterh they're just applications that you embed into the network
[21:12] rbraley Devices are anything that sit between your real applications. this isn
[21:12] pieterh so reusable, stateless, part of the fabric
[21:12] pieterh think of network switches that understand 0MQ patterns
[21:12] rbraley 't really enough of a one-liner zinger to explain them
[21:13] pieterh message-switching networks
[21:13] rbraley I got that analogy
[21:13] rbraley but I can't help but think that devices are more general than that
[21:14] pieterh it's not a tight classification yet, over time i suspect it will become more so
[21:14] pieterh it's about layers, imo
[21:14] pieterh zmq core is one layer
[21:14] pieterh on top of that you have devices
[21:14] pieterh and on top of those two layers you have applications
[21:15] pieterh if you make an app that is reusable and generic, it's potentially a device
[21:16] rbraley perhaps I should think of devices as unix commands and 0MQ sockets as pipes
[21:16] pieterh that kind of works, yes
[21:16] pieterh filters, in Unix terms
[21:16] rbraley right
[21:16] pieterh except there are no pipes
[21:16] pieterh :-)
[21:17] pieterh i'm kidding, i mean there are no visible connections
[21:17] pieterh and sockets are what we use to talk to the network, it's not the actual network
[21:18] rbraley I was thinking that maybe the components of my game engine might be devices
[21:19] rbraley with entities just being collections of sockets to them
[21:20] rbraley a timer device would certainly help
[21:21] rbraley no that doesn't work if devices should be stateless
[21:25] rbraley ooh, a logging device would be neat
[21:28] pieterh rbraley: that would be a logging application, not device >:-)
[21:28] pieterh just kidding...
[21:28] pieterh 'device' really are things you plug into the network, and logging device makes perfect sense afaics
[21:30] rbraley the thing that irks me about these examples is that threading policy is not decoupled from the concurrency policy
[21:31] rbraley ideally you'd have just one thread per core and could multiplex everything to these threads
[21:33] CIA-20 jzmq: 03Alexey Ermakov 07master * r2f3afe8 10/ (perf/local_thr.java src/Socket.cpp src/org/zeromq/ZMQ.java): Changed setSubscribe(), setUnsubscribe() and setIdentity() to accept byte[]'s instead of Strings. - http://bit.ly/b2AZrL
[21:38] travlr i'm beginning to realize that zmq is a 'stateless' model and that state where belongs is the app.. as said: not even in a custom device. i, on the other hand use a device on each box _for_ the state of the individual zmq endpoint components..
[21:38] travlr s/state where/where state/ :-P
[21:41] rbraley without decoupling the threading policy from the concurrency policy it isn't possible to scale 0MQ to model neurons or implement the actor model
[21:42] travlr there are two threading aspects in zmq, afaik, the i/o threading and the app threading..
[21:43] travlr and if i understand your comment correctly the app threading is decoupled
[21:44] travlr from the context but not a socket... but that is to be changed soon, afaiu
[21:44] rbraley What I am saying is the actual number concurrent nodes in the app should exceed the number of app threads
[21:45] rbraley we can't have a thread per neuron for example
[21:45] pieterh rbraley: right, if you say one node is one thread
[21:46] rbraley the concurrent nodes should be multiplexed to the app threads
[21:47] rbraley so 400000 of them can be run on 8 threads for example
[21:48] rbraley when you can do that, the entire paradigm changes
[21:48] travlr i'm thinking that is doable
[21:49] travlr and thats probably where a 'device' fits into the scheme
[21:49] rbraley suddenly you are dealing with abstract flow based programs that are pure computation and run optimally on any hardware that you put them on.
[21:50] travlr ha! have you seen my ProDataLab project regarding that
[21:50] rbraley it becomes a thing of great beauty and efficiency
[21:50] rbraley no sir
[21:50] travlr www.prodatalab.wikidot.com
[21:52] rbraley that is pure sex, travlr
[21:52] travlr lol, it is beautiful
[21:53] rbraley ok we need to talk
[21:53] rbraley where is this architecture spec?
[21:53] travlr please! when ever you like
[21:53] travlr spec is in my head atm
[21:54] rbraley let's get that on paper
[21:54] travlr let's talk sometime then
[21:54] rbraley well not deadtree, you know what I mean ;)
[21:54] travlr right
[21:55] travlr i need a partner so look it over carefully and let me know if you are interested in developing it with me
[21:55] rbraley we were planning on using a similar system for our artists to create entities in our game engine without knowing C++ or Go
[21:56] travlr i don't understand why visual programming isn't more wide spread.
[21:56] rbraley because, there isn't a conceptual model that works universally that people understand yet
[21:56] rbraley I think this is it
[21:57] travlr who difficult is a flow chart... directed graphs
[21:57] rbraley marketing as well. Most people still think procedurally
[21:57] travlr s/who/how
[21:57] travlr yeah but for the user newbie
[21:57] travlr vp is ideal
[21:58] travlr the power of programming in the hands of a layman
[21:58] rbraley so the optimal way to discuss this -- probably a recorded voice chat?
[21:58] CIA-20 jzmq: 03Gonzalo Diethelm 07master * r773e56f 10/ (perf/local_thr.java src/Socket.cpp src/org/zeromq/ZMQ.java):
[21:58] CIA-20 jzmq: Handle an error case when creating a byte array in C++ code.
[21:58] CIA-20 jzmq: Changed the names of some Socket methods:
[21:58] CIA-20 jzmq: getMulticastLoop() => hasMulticastLoop()
[21:58] CIA-20 jzmq: getReceiveMore() => hasReceiveMore()
[21:58] CIA-20 jzmq: setSubscribe(byte[] subscribe) => subscribe(byte[] topic)
[21:58] CIA-20 jzmq: setUnsubscribe(byte[] unsubscribe) => unsubscribe(byte[] topic) - http://bit.ly/aOecvc
[21:59] travlr what ever works for you is works for me
[22:01] rbraley lemme see if i can get a skype call recorder working
[22:03] travlr what's the need for recording?
[22:03] rbraley so we can transcribe and turn it into a spec later
[22:03] rbraley also if we happen upon a great idea we won't lose it and can return to it later
[22:04] travlr sure ok, but before we get into details, we should just have a chat first
[22:07] rbraley what is your skype id travlr ?
[22:10] travlr hold on a min
[22:13] rbraley velaccel is my skype id
[22:13] travlr did you get it?
[22:18] pieterh travlr: so you are Mr ProDataLab... :-)

Web site design and content is copyright © 2010-2011 Peter Alexander, ProDataLab. Site content licensed under the Creative Commons Attribution-Share Alike 3.0 License. ØMQ is copyright © Copyright © 2007-2011 iMatix Corporation and Contributors. Contact us for professional support. ØMQ is free software licensed under the LGPL. ØMQ, ZeroMQ, and 0MQ are trademarks of iMatix Corporation. Terms of Use ? Privacy Policy