Wednesday May 18, 2011

[Time] NameMessage
[02:19] lestrrat mikko: would it be possible to run jenkins builds for perlzmq on win7 ? ( I guess you'd need to install Strawberry Perl )
[06:55] mikko lestrrat: i think it is
[06:55] mikko lestrrat: what sort of build is it?
[06:55] mikko lestrrat: autotools? msvc?
[06:57] lestrrat mikko: you know what, I think I don't have enough information to guide you. I'll research with somebody who is more familiar with windows before asking again.
[11:37] sustrik lestrrat: hi
[11:37] ianbarber evax or yrashk about?
[11:38] ianbarber getting a enif_send: env==NULL on non-SMP VMAborted on erlzmq2, - all of the calls in polling_thread do seem to be passing NULL as the env, but I'm not entirely clear why
[11:53] sustrik dysinger: hi
[11:53] sustrik i've noticed you've made an JNA wrapper for 0mq some time ago
[11:53] sustrik wouldn't it be worth linking it from site?
[11:55] dysinger nah - I wanted to make one - never got around to making it happen for real
[11:55] dysinger the clojure one I have is just a wrapper for the java client
[11:55] dysinger I didn't even finish that one :)
[11:56] dysinger I have been having too much fun with the haskell client
[11:57] lestrrat sustrik: hello
[11:58] sustrik hi
[11:58] sustrik i've noticed you are experimenting with renaming the perl binding
[11:58] sustrik are you doing it for consistency or because of the trademarks stuff?
[11:58] lestrrat yeah, just preparing.
[11:59] sustrik dysinger: i see
[11:59] sustrik i've seen the blog and though it actually works
[12:00] dysinger it works - just not complete
[12:00] sustrik well, linking it can make sense even if incomplete
[12:00] sustrik someone may try to finish it
[13:27] pieter_hintjens hi sustrik
[13:28] mikko hey pieter
[13:28] mikko jira is up and filled with issues
[13:28] pieter_hintjens hi mikko
[13:29] pieter_hintjens i'm going to make a page on 'how to make an issue'
[13:29] pieter_hintjens anything special you'd like me to explain there?
[13:29] mikko should we update the domain as well?
[13:29] pieter_hintjens mikko: no hurry imo
[13:29] pieter_hintjens i'll get a certificate, we can do it then
[13:29] mikko we can do like to forward there for now?
[13:29] pieter_hintjens yes
[13:29] pieter_hintjens i can do that immediately
[13:30] evax ianbarber: erlzmq2 needs an SMP VM
[13:30] pieter_hintjens I found a technical notebook from 1995
[13:30] pieter_hintjens I was designing an "IP toolkit"
[13:31] pieter_hintjens part 2: 'no-wait tcp/ip server', reads bunch of sockets, passes messages to dispatcher, which maintains queues of messages and routes to apps
[13:31] pieter_hintjens hmm... :)
[13:31] ianbarber evax: fair enough :)
[13:31] ianbarber thanks
[13:32] evax ianbarber: np, this is a limitation of the enif_send NIF call we're using to message back calling processes for non-blocking calls
[13:36] pieter_hintjens mikko: try
[13:36] mikko works
[13:37] pieter_hintjens excellent
[13:37] pieter_hintjens it's a hack, I'll kill it once we make the real custom domain etc.
[13:37] mikko yeah
[13:59] pieter_hintjens Anyone interested in a meetup in Seattle next week?
[14:00] pieter_hintjens I'll take that as a YES
[14:01] mikko hehe, a bit short notice to hop over
[14:02] mikko i'll be in california but in san diego
[14:03] pieter_hintjens America is so large
[14:35] jhawk28 hehe
[14:35] jhawk28 texas is bigger than most countries
[14:36] jhawk28 and Alaska is even bigger
[14:39] jhawk28 pieter_hinthens or sustrik: does the Poller do a busy poll? or does it use something like epoll?
[15:12] sustrik jhawk: on, it's not busy poll
[15:12] sustrik it's either poll or select
[15:31] sustrik pieter_hintjens: hi
[15:31] sustrik what's up?
[15:53] fdb might be a good idea using zmq directly on internet?
[15:54] fdb how may subscribers can handle a single instance of zmq?
[15:56] fdb and about authentication? is there best practices?
[16:49] sustrik fdb: it's not hardened for internet use
[16:49] sustrik it can handle thousands of connections
[17:00] pieter_hintjens sustrik: hi, sorry, I was distracted with a meeting
[17:00] pieter_hintjens was just thinking about how to make userspace transport plugins
[17:01] pieter_hintjens 12 hour drives are good for something :-)
[17:03] sustrik :)
[17:03] sustrik what's the idea?
[17:03] pieter_hintjens well, as already discussed, to make inproc bridges
[17:04] pieter_hintjens but with a little packaging so they look like transports
[17:04] pieter_hintjens e.g. bridge reads/writes UDP and speaks inproc:// back to applications
[17:04] pieter_hintjens it's started as a dependent thread, operates its own select or poll loop
[17:05] sustrik yes, that would work
[17:05] sustrik requires a lot of work in the bridge though
[17:05] pieter_hintjens it's possible to put native file handlers into zmq_poll, right?
[17:05] sustrik yes
[17:05] pieter_hintjens and vice-versa, to select() on zmq sockets...
[17:05] sustrik that's a bit harder
[17:06] pieter_hintjens edge-triggering?
[17:06] sustrik as you have to account for edge-triggered nature of ZMQ_FD
[17:06] sustrik yes
[17:06] pieter_hintjens ok, that's fine
[17:06] pieter_hintjens so my goal is to make a proof of concept for UDP
[17:06] pieter_hintjens i'd like to explore a fully async i/o model, if that's possible on linux
[17:06] sustrik how would that work? udp has no connections
[17:07] pieter_hintjens udp definitely has sockets
[17:07] sustrik have a look at DCCP
[17:07] pieter_hintjens sockets have activity, it's the same api
[17:07] pieter_hintjens you just omit the connect part
[17:07] tproa i wonder if what you're thinking about userspace transports meshes with some idle thoughts i had on trying to graft gssapi onto zmq
[17:07] pieter_hintjens tproa: very close, yes IMO
[17:08] pieter_hintjens i want then to extend this for security
[17:08] pieter_hintjens if it can be fast enough, IMO it can be used for most work in place of extending the native transports
[17:09] pieter_hintjens there are tradeoffs in doing in userspace, and advantages
[17:09] pieter_hintjens so it's an experiment, to be tested
[17:10] sustrik the problem with UDP is that you have no idea where to send the message
[17:10] sustrik as there are no connections
[17:10] pieter_hintjens you send to an ip address and port afair
[17:10] pieter_hintjens it's not particularly complex
[17:10] pieter_hintjens or you broadcast to a set of ip addresses
[17:10] pieter_hintjens set = all
[17:10] sustrik how would you know what IP and what port?
[17:11] sustrik i mean, how would the user specify it
[17:12] pieter_hintjens the code uses the sendto and recvfrom methods
[17:12] pieter_hintjens the user app would do bind and connect as usual (but using their userspace versions)
[17:13] sustrik ah, yes
[17:13] pieter_hintjens sendto and recvfrom have sockaddr structures that are used when sending and on reception
[17:13] sustrik that would work
[17:13] pieter_hintjens logically, you bind and connect
[17:13] pieter_hintjens i guess this is also how pgm does it
[17:13] pieter_hintjens you don't actually bind, but that sets up addressing
[17:14] pieter_hintjens the downside of doing this in userspace is that we end up with yet another layer on top of libzmq
[17:14] pieter_hintjens so my idea is to shove this into libzutil
[17:14] pieter_hintjens if/when it works
[17:14] pieter_hintjens it'll be a nice way to solicit contributions to libzutil
[17:15] pieter_hintjens and make them accessible to all language bindings
[17:15] sustrik ok
[17:16] pieter_hintjens do we have any docs on how to do the edge-triggered stuff properly?
[17:16] pieter_hintjens it seems to be a frequent question on the lists
[17:17] sustrik i don't think so
[17:17] sustrik the only point is that it's change of state that's reported
[17:17] sustrik not availability for reading/writing
[17:18] pieter_hintjens and 'change of state' is binary?
[17:18] pieter_hintjens or has multiple values?
[17:18] pieter_hintjens it's like "became readable" and "became writeable"
[17:18] pieter_hintjens but you only get that once?
[17:19] sustrik well, ZMQ_FD just signals "something happened"
[17:19] sustrik ie. state changed
[17:19] sustrik you then have to get ZMQ_EVENTS option to find out whether socker is readable/writeable
[17:20] pieter_hintjens ah, yes, I'm actually reading the man page where it's more or less explained... :-)
[17:20] sustrik goodo
[17:20] pieter_hintjens ok, it makes sense, this'll make a nice piece for the superadvanced section of the Guide
[17:20] pieter_hintjens thanks, Martin!
[17:22] sustrik np
[17:40] sustrik pieter_hintjens: problem with UDP:
[17:41] sustrik the peer doesn't get notified when you connect
[17:41] pieter_hintjens sure
[17:41] pieter_hintjens but in 0mq land, apps don't know either
[17:41] pieter_hintjens :-)
[17:41] sustrik which means that say PUB socket has no idea that it should start sending messages to you
[17:41] pieter_hintjens shrug
[17:41] pieter_hintjens this sounds so familiar :-)
[17:41] sustrik so pub/sub won't work
[17:42] pieter_hintjens sustrik: well, it's the same with pgm
[17:42] pieter_hintjens and with tcp under zmq
[17:42] sustrik nope
[17:42] sustrik with tcp the bind side is notified about the connection creation
[17:42] pieter_hintjens the only pubsub model that will work over UDP is broadcast
[17:42] pieter_hintjens otherwise, with an explicit pubsub device
[17:42] pieter_hintjens fine, we'll see what effective patterns emerge
[17:43] sustrik yes, broadcast
[17:43] sustrik devices won't work
[17:43] pieter_hintjens i'll show you devices :-)
[17:43] pieter_hintjens later
[17:44] sustrik ok