Friday June 17, 2011

[Time] NameMessage
[01:47] thborges Hello
[01:48] thborges Is possible to do this type of msg routing with 0mq? mama ask -> papa1 re-ask -> papa2 knows anwser and talk direct to -> mama
[02:52] ssi I wanna play with the objc bindings and see how zmq does on the iphone
[02:52] ssi could be an awesome solution for transport
[03:28] ssi ok this is bizarre
[03:28] ssi on one machine it works perfectly
[03:28] ssi on the other, an object which is serialized and sent across a socket comes out at the other end unable to be serialized
[03:28] ssi and the byte array message that is received is BIGGER than what was sent
[03:28] ssi ...but only on one machine, same source
[03:31] ssi oh weird, I'm running 2.1.8 on the machine that's misbehaving, and 2.1.7 on the one that works
[03:35] ssi and downgrading to 2.1.7 doesn't make a difference :/
[03:41] Moc is the chat example suposed to work ?
[03:42] Moc in python
[05:53] CIA-32 pyzmq: 03MinRK 07master * rf302507 10/ (zmq/core/socket.pyx zmq/tests/ allow setting/getting sockopts as attributes -
[05:53] CIA-32 pyzmq: 03MinRK 07master * r08f9063 10/ : add socket.pxd dependency on modules that cimport it -
[05:53] CIA-32 pyzmq: 03MinRK 07master * r19f8668 10/ (3 files in 2 dirs): allow subclasses to extend attributes -
[11:58] star Can some one help me with next problem: Assertion failed: (msg_->flags | ZMQ_MSG_MASK) == 0xff (zmq.cpp:211)
[11:58] star if i set ZMQ_HWM = 1
[12:00] star version of 0mq 2.1.7
[12:13] sustrik star_: it means you are using 0mq message after it have been closed
[12:13] sustrik or before it was initialised
[13:01] leandroa hi all, is there an example somewhere of a multiple rooms chat system that uses zeromq?
[13:08] mikko leandroa: there used to be a chat example
[13:08] mikko but not sure if there is out of a box example
[13:09] leandroa mikko: I've just found the code example, but it's pretty old. Anyway, found a nice example on github using Django, will give it a try.. Thanks!
[13:09] mikko enjoy
[13:13] leandroa I'm doing an API system, I'm doing many polling calls each certain time, do you think I should consider move those calls to 0mq too (besides the chat functionality)?
[13:14] Seta00 leandroa, can you link to that django example?
[13:14] leandroa those polling calls mostly asks for changes in the system
[13:14] leandroa Seta00: Sure, it's
[13:14] Seta00 fun, thanks
[15:45] Steve-o so, any verdict on NT 3.5 support?
[15:55] sustrik Steve-o: i would say, ditch it
[15:55] sustrik what does the last patch do
[15:55] sustrik ?
[15:56] sustrik the email seems to suggest that it adds (?) winsock1 support
[15:57] Steve-o well from Dirkjan's mail about removing NT 3.5 support from stable I noticed that stable doesn't actually support it in the public headers
[15:58] Steve-o only the internals
[15:58] sustrik right
[15:58] Steve-o so you have a choice to add to the public header, NT 3.5, or remove completely
[15:58] sustrik the relevant piece of code is ~2 years old
[15:58] sustrik and afaik nobody tested it since
[15:58] sustrik no point in adding something we cannot test anyway
[15:59] Steve-o so the next problem is with the mingw32 redefinition of winver
[15:59] Steve-o it occurs internally but not publically
[16:00] sustrik what's the problem with that?
[16:00] Steve-o it becomes a question of why bother doing it at all
[16:01] sustrik doing what?
[16:01] Steve-o defining WINVER=0x0501
[16:01] sustrik 0mq does that?
[16:01] Steve-o in windows.hpp
[16:02] Steve-o I also do it, I'm guessing there is a WINAPI structure dependent upon it, one of the timer APIs I think
[16:03] Steve-o I'll have to hunt it down, insufficient docs
[16:04] sustrik undocumented and behaves differently on different windows/mingw versions afaiu
[16:04] sustrik pretty bad mess
[16:04] Steve-o I'm thinking it's time_t
[16:05] sustrik what problem are you solving btw
[16:05] sustrik cygwin+mingw build?
[16:05] Steve-o Just refactoring the code as it is confusing and different
[16:06] Steve-o one of the patches is for mingw-w64
[16:07] sustrik although i am normally +1 for refactoring
[16:07] sustrik in this case it's probably better to use something like
[16:07] sustrik #ifdef ZMQ_HAVE_MINGW_W64 ...
[16:08] sustrik that way we won't break the black magic that makes the thing compile today in all the different setups
[16:08] Steve-o I'd like to ask the Mingw teams why _WINSOCK_API_ doesn't work
[16:08] Steve-o but today it looks like WIN32_LEAN_AND_MEAN is good everywhere
[16:08] sustrik i have no idea
[16:09] Steve-o So I'm trying to update the code documentation as to explain some of the choices
[16:09] sustrik well, if you feel like you understand how win build works, go on
[16:10] Steve-o as ultimately I'm trying to get some named pipe / iocp code working
[16:10] sustrik at the moment it's just a mess of small tweaks
[16:10] sustrik so if you are able to clean it up, that would be great
[16:11] sustrik however, my feeling is that the mess in 0mq reflects the mess in win build environments
[16:11] sustrik different win versions, msvc, mingw, cygwin etc.
[16:12] sustrik anyway
[16:12] Steve-o I have them all working in OpenPGM, and pretty confident with it
[16:12] sustrik goodo
[16:12] sustrik any patch i should apply?
[16:12] sustrik the last one you've sent?
[16:16] Steve-o I'll send a new mail with some changes and complete patch set
[16:16] sustrik great
[16:16] Steve-o remove NT 3.5 and the NOxxx definitions
[16:16] Steve-o because they're silly
[16:16] sustrik if you feel it'll work go for it
[16:17] sustrik btw, i can mark you as a maintainer of win build system
[16:17] sustrik for now, there's my name there which is as good as no maintainer at all
[16:17] Steve-o but I'm still hunting for the WINVER 501 definition reason, I've got to 403 minimum so far
[16:18] sustrik acl
[16:18] sustrik ack
[16:22] Steve-o it looks like 501 is required for IPv6 support in getaddrinfo()
[16:23] Steve-o very terse explanation in the docs -
[16:24] sustrik i see
[16:24] sustrik in theory, ipv6-related functionality can be made optional
[16:24] Steve-o as I guess mingw32 much like wine used to default to win95 or so
[16:25] ssi is it likely that someone that was skilled with windows development could build it and make a package for others within their organization to deploy?
[16:25] sustrik so mingw32 =>no IPv6, right?
[16:25] sustrik ssi: there are packages somewhere
[16:26] Steve-o The python guys made the packages, no idea how
[16:26] Steve-o I need to test what mingw32 defaults to for winver these days
[16:26] sustrik ok
[16:27] Steve-o there must be a better way around it
[16:28] taotetek sustrik: I'm struggling a bit internally at our company over what I think may be a misperception of what zmq is and what it gives you, mind if I bounce my thinking off of you in a few sentences to check my own thoughts?
[16:30] sustrik sure
[16:31] taotetek sustrik: my impression of zmq is that it's a set of tools for implementing messaging patterns. the problem I see is I see developers trying to use simple zmq sockets (like a push / pull connection) and then getting frustrated with things such as lost messages due to buffering, etc. my assertion is that zmq is a set of tools for -building- messaging systems, and is not a messaging system in and of itself.
[16:32] sustrik hm
[16:32] sustrik kind of
[16:32] taotetek sustrik: for instance, while a simple push / pull gives you very little reliability, it's relatively easy to throw together a ventilator / worker(s) / manager pattern with batched messaging and acknowledgements
[16:33] sustrik the thing is that the project is progressing still
[16:33] taotetek sustrik: so, I'm trying to check my own thinking on it and find a good way to communicate with others where zmq fits in and what it does, and does not provide
[16:33] sustrik so one day there'll probably be acks in push/pull
[16:33] sustrik but right now you have to implement them on top
[16:34] taotetek sustrik: so there are roadmap plans to implement such things, but for the moment it's a bit more do it yourself.
[16:34] CIA-32 pyzmq: 03MinRK 07master * r4c99e82 10/ (4 files in 3 dirs): Merge branch 'master' of -
[16:34] CIA-32 pyzmq: 03MinRK 07master * r4d8ee36 10/ (4 files in 2 dirs): add docs/examples about custom serialization ...
[16:34] taotetek sustrik: I find that acceptable as the "do it yourself" isn't really that complicated, it's more just a problem of getting everyone on the team in the same headspace
[16:35] sustrik taotetek: yes, if you are thinking in short-term
[16:35] sustrik the DIY is the only viable point of view
[16:36] sustrik i mean, if particular feature appears next year, it won't help you today to build your app
[16:37] sustrik hm, well, i guess it works that way with any software :)
[16:37] taotetek sustrik: so my goal is then to make sure people are listening to me when I say "this is a relatively new and rapidly moving project" and understand both the benefits and drawbacks of being involved with it at this stage :)
[16:37] sustrik yes, that makes sense
[16:38] taotetek sustrik: I find it an incredibly enabling technology already; I'm just running into a few issues with expectations with a couple of folks
[16:38] sustrik yes, the benefit is that there's a lot of complex functionality you don't have to build yourself
[16:38] taotetek sustrik: so I wanted to make sure my own expectations are level set appropriately
[16:38] sustrik the drawback is that some functionality that would be nice to have is still missing
[16:38] sustrik i think they are
[16:40] taotetek sustrik: I know that Ken (who is working on the rsyslog/zeromq integration) is very interested in digging into "reliable" push / pull. if he comes up with ideas should he feel free to send patches your way? I know he wants to contribute, but he wants to do so in a way that doesn't step on any established plans
[16:40] sustrik taotetek: yes, that would be great
[16:40] sustrik push/pull acks are on todo list for a long time
[16:41] sustrik what's missing is someone to implement it :)
[16:41] taotetek sustrik: well excellent then. I'll let him know!
[16:41] taotetek sustrik: thanks for the time this was very helpful
[16:41] sustrik you are welcome
[16:42] Steve-o So back to Mingw,
[16:42] Steve-o MinGW32 defaults to NT 4.0
[16:42] Steve-o MinGW-w64 defaults to XP SP2
[16:44] Steve-o So for IPv6 the header could check for WINVER < 0x0501 and redefine instead of wiping out completely
[16:44] ssi taotetek: I'm in somewhat of the same boat... I'm working on a prototype of a replacement for an existing six year old workflow processing engine in my org, using zmq as a transport
[16:44] ssi taotetek: I'm trying to hide all the zmq specifics inside the framework, so that developers and integrators don't have to use it at all
[16:46] taotetek ssi: that make sense. I'm certain once I establish the flows that I want, that I'll abstract them into higher level libraries that hide some of the implementation details
[16:46] sustrik Steve-o: ok
[16:47] ssi taotetek: the biggest concern I have is that every developer and integrator that uses this framework will need to have the zmq library and bindings installed in the environment where they're testing (this is a java framework)
[16:47] taotetek ssi: although I don't personally mind the implementation details, none of it is that far removed from normal socket programming / select loops etc
[16:47] ssi and right now that means building from source on lots of different platforms
[16:47] ssi I'm on OSX, and it's not too bad to get up and running, but it does require a little bit of fiddling... the windows side of it scares the bejeebus out of me
[16:47] taotetek ssi: yeah, in my situation we have a lot of front end stuff that's written in java and back end systems written in python
[16:48] ssi step 1: have MSVC++
[16:48] taotetek ssi: fortunately I don't have to deal with windows
[16:48] ssi for a shop full of java developers, that's not a common thing to deal with
[16:48] ssi I don't have to deal with it either, but it'll be a hard pill to swallow to tell the guys that are on windows machines that they have to figure this out on their own
[16:48] Steve-o ssi: so you are looking for multi-platform bin in jar package?
[16:49] ssi Steve-o_: I haven't started looking yet, but yea basically if there's a way that I can provide the lib and binding to everyone without them having to build from source, it'll make the transition much easier
[16:49] ssi if that means i have to build packages, then that's what I'll do :P
[16:50] ssi I'm not super concerned about it on the operations side.... our servers are all RHEL and ubuntu, and if I tell the ops guys that they need to build it from source, then they'll just bake it into their vm recipes and everything will be fine
[16:50] ssi it's more supporting the hundreds of developers on their dev machines
[16:51] Steve-o which would make a native jar really convenient
[16:51] ssi yes, very
[16:54] ssi the good news is, if I can get this in good enough shape to get the org to adopt it (which shouldn't be hard assuming everything works), I can almost certainly get them to pay for support
[16:54] ssi and it's kind of a big org :P
[16:56] Steve-o I might have a look after all these Windows changes,
[16:56] Steve-o I've never had to change anything with JNI builds
[17:41] nishttal2 is it possible to have multiple keys in a multipart message ?
[17:49] nishttal2 seb`, sybscription key
[17:49] nishttal2 sub*
[17:50] guido_g no it's not, only the first part of a message is checked
[17:52] taotetek since first part of message is checked if you're clever about namespacing your topics you can pull off some tricks
[17:54] nishttal2 thanks.. i am new to ZeroMQ and coming from Activemq
[17:54] nishttal2 in Activemq I created topics and subscribe by Data.Realtime.T for example
[17:55] nishttal2 is it possible to create named topics in zeromq and have consumers subscribe to that topic (by name)(
[18:02] nishttal2 i just found this.. Publisher Side Message Filtering on so does that mean this is not implemented yet?
[18:03] Seta00 nishttal2, right now you can only subscribe on the subscriber side
[18:03] Seta00 er. filter on the subscriber
[18:03] nishttal2 there is no way for the publisher to split the feed ?
[18:03] Seta00 not yet, unfortunately
[18:04] Seta00 pieterh would know more
[18:04] nishttal2 like trade data to one channel and Quote data on another channel
[18:04] guido_g use 2 sockets
[18:04] nishttal2 can this be simulated with different sockets though
[18:04] nishttal2 yeah ok
[18:05] guido_g ømq is not a replacement for activemq or other messaging products
[18:05] sustrik nishttal2: there's publisher-side filtering in the trunk
[18:05] nishttal2 is there a limit to # of sockets i can use
[18:06] sustrik but it's just couple of days old
[18:06] sustrik so it's something like alpha
[18:07] sustrik nishttal2: see src/config.hpp, max_sockets=512
[18:07] sustrik you can increase the limit if you need to
[18:09] nishttal2 i see.. so potentially i could simulate multiple topics by publishing on different sockets
[18:09] sustrik yes
[18:09] nishttal2 the only issue would be for subscribers to figure out what socket has what data
[18:10] nishttal2 i could publish that information on a standard fixed socket so all clients know which socket has the data they are interested in
[18:10] nishttal2 sustrik, does that make sense?
[18:11] sustrik there are pitfalls there
[18:11] sustrik what if "master feed" is broken
[18:11] sustrik etc.
[18:11] nishttal2 i could put the master feed over Active MQ, although that would be too many moving pieces
[18:12] sustrik yes
[18:12] sustrik alternatively, publisher side filtering is the real solution, but as i said, it's in alpha phase
[18:12] sustrik btw, whitepaper about pub-side filtering:
[18:13] sustrik cya
[18:13] nishttal2 hmm.. when do you anticipate the pub-side filtering to be release
[18:13] nishttal2 ok thanks
[18:23] nishttal2 does anyone know how I download the code from trunk that sustrik mentioned above
[18:32] guido_g use git clone
[18:34] guido_g <- repo
[18:34] nishttal2 got it thanks
[18:36] nishttal2 there is no configure script
[18:36] guido_g use
[18:39] nishttal2 that worked.. thanks
[19:13] CIA-32 pyzmq: 03MinRK 07master * rf0a6046 10/ (4 files in 2 dirs): add docs/examples about custom serialization ...