IRC Log

Monday November 1, 2010

[Time] NameMessage
[00:06] Guthur sustrik, I'm off to bed now, but if you get a chance to review that clrzmq code and suggestion any changes it would be great, its not a massive update to be fair, I'll probably provide the multithreading demo when the device code is finalised
[01:06] gandhijee Hi all, what flag to i need to pass gcc for it to link/use zeromq?
[01:06] gandhijee -l<what goes here for zeromq>?
[01:08] gandhijee bm
[01:08] gandhijee *nm
[04:05] gandhijee is there away to check the size of an incoming message?
[04:23] xraid gandhijee: http://api.zeromq.org/zmq_msg_size.html -- what binding are you using ?
[04:26] gandhijee xraid: C, i am using protocol buffers from google
[04:27] gandhijee and i am trying to figure out how to allocate the memory for the message after i get it from the sender
[04:29] gandhijee xraid: that was it, thanks!
[07:27] dccmx i have a question
[08:28] xraid hey whats up with the ML i am subscribed but cants send to it as i am not a member of the list ? i thought i was a member of the list if subscribed ?
[08:33] sustrik xraid: what's your address?
[08:34] xraid zmq iprobot.com
[08:38] sustrik let me see
[08:40] xraid hehe i tried to sub as xraid but i was banned . i then replied with a "why so" . no answer yet ...
[08:41] sustrik xraid: the subscription is see is "omq at iprobot.com"
[08:43] xraid ahhh that explains it then ...
[08:48] xraid sustrik: if possible i much prefer to change omq to xraid if possible
[08:54] sustrik well, do so
[08:54] sustrik ah, xraid is banned?
[08:55] xraid huh ;-9
[08:55] xraid how come
[08:56] sustrik hm, it's not in the ban list
[08:56] sustrik ???
[08:56] sustrik "hehe i tried to sub as xraid but i was banned"
[11:33] Ogedei do I understand correctly that msg's *can* be passed between threads (as long as they are accessed in a single-threaded way), and don't have restriction that sockets have?
[11:39] sustrik Ogedei: yes
[11:39] sustrik they are plain structures
[11:45] Ogedei sustrik: thank you
[11:47] sustrik you are welcome
[12:32] mato sustrik: are you there?
[12:32] sustrik hi
[12:33] mato hi
[12:33] mato sustrik: what exactly would you like me to test with the new signaler, performance-wise?
[12:33] sustrik not yet, just test whether it works for you
[12:33] mato ah, ok
[12:33] mato i'll do that now then
[12:33] sustrik ok
[12:35] mato sustrik: note that my userspace (Debian lenny) is too old for eventfd, so I'll be testing the socketpair implementation
[12:36] sustrik goosd
[12:36] mato i can maybe hack something together to try and make eventfd work on my systems, but not right now
[12:36] sustrik just test what you have
[13:01] CIA-21 zeromq2: 03Mikael Helbo Kjaer 07master * r0ad71f8 10/ (src/select.cpp src/select.hpp):
[13:01] CIA-21 zeromq2: select now uses Erase-Remove idiom for retired fds
[13:01] CIA-21 zeromq2: Signed-off-by: Mikael Helbo Kjaer <mhk@designtech.dk> - http://bit.ly/9mdkZd
[13:04] mato sustrik: seems to work so far... testing with up to 600 nodes
[13:04] sustrik mato: the patch you've sent means that INSTALL goes to git, right?
[13:04] mato sustrik: the topology setup time obviously increases, and i increased the listen backlog and turned wait_before_reconnect on as usual
[13:05] mato sustrik: yes, that's correct, just apply the patch as is
[13:05] mato sustrik: it removes INSTALL from .gitignore and adds it into git, yes
[13:05] sustrik interestingly, git apply doesn't add the new file to the git
[13:06] mato probably because it's in .gitignore
[13:06] sustrik np
[13:06] mato just add it in by hand or something
[13:06] sustrik i did
[13:06] mato and don't forget to remote it from .gitignore :-)
[13:06] sustrik that part was applied ok
[13:09] pieterh sustrik: without the foreign flag, autotools will generate INSTALL automatically
[13:10] mato pieterh: kind of, but with the foreign flag it breaks our automatic ChangeLog generation
[13:10] pieterh you mean without foreign flag?
[13:10] mato yes, I wrote about it
[13:10] pieterh yes
[13:10] sustrik patch looks like working ok
[13:10] sustrik committing it
[13:10] mato there's a reason for that foreign flag
[13:10] mato as I explained.
[13:11] pieterh mato: I was just explaining to sustrik that INSTALL is usually generated code
[13:11] sustrik it used to be
[13:11] mato no, it's stuck in as a template if it's not there
[13:11] sustrik not it's not
[13:11] pieterh we were looking for this yesterday but I couldn't see where the foreign flag was being set
[13:11] sustrik i don't care much
[13:12] mato whatever, the INSTALL file will eventually contain custom content anyway
[13:12] mato so it'll be in git anyway
[13:12] pieterh presumably, yes
[13:12] CIA-21 zeromq2: 03Martin Lucina 07maint * reb83678 10/ (.gitignore INSTALL):
[13:12] CIA-21 zeromq2: Add INSTALL to Git, thus making it a normal file
[13:12] CIA-21 zeromq2: INSTALL gets added in somewhat magically by automake, or not. Adding it into
[13:12] CIA-21 zeromq2: Git ensures it's always included in the distribution.
[13:12] CIA-21 zeromq2: Signed-off-by: Martin Lucina <mato@kotelna.sk> - http://bit.ly/bCAWcw
[13:17] sustrik mato, pieter: for development releases i propose we won't upload the man pages to the website
[13:17] sustrik they are included in the package
[13:18] mato sustrik: dunno, maybe... or we have two sets on the website
[13:18] sustrik and it would confuse users using the stable release
[13:18] mato to be determined
[13:18] sustrik it there are 2 versions of docs on the website
[13:18] pieterh to be determined
[13:18] pieterh how does, e.g. APR do this? How does any project do this?
[13:18] sustrik no idea
[13:19] sustrik do they have 'unstable' releases?
[13:19] pieterh sure
[13:19] mato yes
[13:19] pieterh every serious project has
[13:19] mato and they have multiple versions of the docs uploaded
[13:19] mato clearly marked
[13:19] pieterh indeed
[13:19] pieterh it's just code at a URL
[13:19] mato some people will of course not read the clear marking and get confused anyway, too bad
[13:19] pieterh indeed
[13:19] sustrik ok
[13:21] mato sustrik: ok, so what are we going to do with the new signaler?
[13:21] mato sustrik: it seems to work as far as I can tell, but as I say, I've only tested on a single platform and have not done any serious performance tests...
[13:21] CIA-21 zeromq2: 03Martin Lucina 07master * reb83678 10/ (.gitignore INSTALL):
[13:21] CIA-21 zeromq2: Add INSTALL to Git, thus making it a normal file
[13:21] CIA-21 zeromq2: INSTALL gets added in somewhat magically by automake, or not. Adding it into
[13:21] CIA-21 zeromq2: Git ensures it's always included in the distribution.
[13:21] CIA-21 zeromq2: Signed-off-by: Martin Lucina <mato@kotelna.sk> - http://bit.ly/bCAWcw
[13:21] CIA-21 zeromq2: 03Martin Sustrik 07master * rdbcd382 10/ (.gitignore INSTALL):
[13:21] CIA-21 zeromq2: Merge branch 'maint'
[13:21] CIA-21 zeromq2: * maint:
[13:21] CIA-21 zeromq2: Add INSTALL to Git, thus making it a normal file - http://bit.ly/aK31FB
[13:22] sustrik mato: well, it's kind of complex
[13:22] sustrik maybe you want just to patch your version as for now?
[13:22] mato why?
[13:22] sustrik time pressure?
[13:22] mato that all gets very confusing rapidly
[13:22] mato and i'd like to avoid extra patches if at all possible...
[13:22] sustrik so, for starters we need win32 implementation of event
[13:23] sustrik i've only did linux one
[13:23] sustrik also, i've merged the hp-ux/aix code with linux code
[13:23] mato right, i saw that
[13:23] sustrik hopefully, brett will test that part
[13:23] mato that doesn't matter too much, if hp-ux/aix breaks then i'd expect those users to speak up
[13:23] mato we cannot obviously test on platforms we don't have
[13:24] mato and hp-ux/aix are a minority platform anyway
[13:24] sustrik so, we need win implementation
[13:24] mato right...
[13:24] sustrik test the eventfd implementation
[13:24] sustrik i've did a bit, but maybe running some larger payload would be good
[13:24] sustrik then we need to optimise it
[13:25] sustrik will you do the win32 part?
[13:26] mato i could do that, yes
[13:26] sustrik thx
[13:28] sustrik mato: btw, i copied your release process here:
[13:28] sustrik http://www.zeromq.org/docs:procedures#toc7
[13:29] mato sustrik: oh, good
[13:29] sustrik it's not perfect, but at least it's documented
[13:30] mato yup, good
[13:34] pieterh sustrik: very nice, we were missing this
[14:34] mato sustrik_: hmm, event_t::wait() is supposed to busy-loop?
[14:35] mato sustrik_: ah, sorry, i misread the code
[15:10] mato sustrik_: here?
[15:25] mato sustrik_: um, the signaler has always been ignoring EINTR as far as I can tell...
[15:26] mato sustrik_: so it would seem that is the *correct* behaviour
[15:26] mato sustrik_: and the do { recv() } while() loop has to stay for POSIX platforms
[15:26] sustrik hm
[15:26] sustrik ignoring = looping?
[15:27] mato yes
[15:27] sustrik even blocking read?
[15:27] sustrik let me see
[15:27] mato ah, hang on
[15:27] mato the different implementations are a mess...
[15:28] sustrik if (nbytes == -1 && (errno == EAGAIN || errno == EINTR))
[15:28] sustrik return -1;
[15:28] mato yeah, because that implementation uses MSG_DONTWAIT
[15:28] mato which is specific to god knows what platforms
[15:28] sustrik yep
[15:28] sustrik we don't need that now so we can drop it
[15:29] sustrik in any case if (EINTR) return -1
[15:29] mato yeah but your event_t has no notion of returning anything (all the methods are void)
[15:29] mato i'm just wondering how that is supposed to work...
[15:30] sustrik we'll have to add the reurn codes then
[15:30] sustrik the idea is that application thread exits on EINTR
[15:30] sustrik while I/O thread restarts the operation
[15:31] mato so that means that signaler_t::recv() must return EINTR if the underlying blocking call is interrupted, right?
[15:31] sustrik tes
[15:31] sustrik yes
[15:31] sustrik actually, i am not sure EINTR can even happen in I/O thread
[15:32] mato so both event_t::wait() and event_t::reset() need to be able to return -1 and errno=EINTR, right?
[15:32] sustrik given that signals are switched off in those threads
[15:32] mato good question, no idea
[15:32] sustrik even set can return EINTR
[15:32] mato if ::send() is interrupted?
[15:32] sustrik yes
[15:33] mato right, so all of those need to be int
[15:33] sustrik ack
[15:33] mato i will change that
[15:36] sustrik ok, I/O thread handles EINTR correctly (restarts the operation)
[15:37] mato ok, but what about signaler_t::send()?
[15:37] mato that method does not return anything...
[15:37] mato so what is it supposed to do if event_t::set() returns EINTR? restart it?
[15:39] sustrik hmm
[15:40] sustrik the problem is that by restarting you loose the signal
[15:40] sustrik but the signal handling is not 100% efficient anyway
[15:40] sustrik so yes, we can just restart the send
[15:41] pieterh sustrik: you have a second?
[15:41] pieterh question on code structure
[15:41] pieterh why does select.cpp do a bunch of stuff for windows, and then invoke select.hpp that does the same?
[15:42] pieterh afaics select.hpp isn't used by any other code
[15:42] sustrik pieterh: historic reasons
[15:43] pieterh ?
[15:43] pieterh i mean, is this by design or just bogus?
[15:43] sustrik the poll algo classes should be refactored
[15:43] sustrik bogus
[15:43] pieterh ok... cause there's duplicate code but it's not quite the same in both cases
[15:43] pieterh actually it's worse, there's also windows.hpp that does the same again
[15:44] sustrik i know, but it's hard to change without breaking anything
[15:44] mato there is much historical weird stuff to clean up wrt windows.wpp and the bits that depend on it
[15:44] mato but, as sustrik says, it's all pretty fragile, so if it ain't broke, don't fix it
[15:44] pieterh well, select.hpp is only used in one place
[15:44] pieterh and it is broken
[15:45] pieterh and I do intend to fix it, the FD_SETSIZE issue has been there for years and should go
[15:45] sustrik sure, send the patch
[15:45] mato oh dear, not that again
[15:45] pieterh well, my question is whether you really want me to 'fix' three places that are all actually the same?
[15:45] mato FD_SETSIZE is a problem because zmq.h depends on winsock for SOCKET *
[15:45] pieterh mato: please
[15:45] pieterh we do understand the case pretty well
[15:46] pieterh the problem here is just 0MQ's code, not the fix in question
[15:46] mato sure
[15:46] pieterh winsock.h is included 4 times
[15:46] pieterh that is the problem
[15:46] sustrik i don't know about FD_SETSIZE, but if there's a bug in select.cpp, just fix it and send the patch
[15:46] mato if you can make all the code work *correctly* *and* avoid redefining an *application's* view of FD_SETSIZE, go for it
[15:46] pieterh mato: we've been over this
[15:47] pieterh code that calls a library must defacto use that library's value for FD_SETSIZE
[15:47] mato pieterh: I will not accept any patch that redefines FD_SETSIZE of an *application* including zmq.h
[15:47] mato for reasons that I have explained
[15:47] pieterh you've not explained, and/or have not understood
[15:47] mato zmq.h is the *API* header
[15:47] pieterh every library that wraps select() must define a value for FD_SETSIZE
[15:47] mato it has no business changing the *application's* view of FD_SETSIZE
[15:48] mato yes, sure
[15:48] pieterh lll
[15:48] mato i have no problem with that
[15:48] pieterh it changes the API's definition
[15:48] pieterh this is standard stuff
[15:48] pieterh now, including zmq.h magically sets FD_SETSIZE to 64
[15:48] pieterh which is not a feature, but a bug
[15:48] mato no it doesn't, it leaves it at whatever the default is
[15:48] pieterh no it doesn't
[15:49] pieterh winsock.h defines it
[15:49] mato bah
[15:49] pieterh unless it's already defined
[15:49] mato exactly
[15:49] pieterh well... did you think I'd ignore a previous definition?
[15:49] pieterh / Raise max sockets from default of 64 to more usable value
[15:49] pieterh / Must be consistent in any code that shares FD sets
[15:49] pieterh #ifndef FD_SETSIZE
[15:49] pieterh #define FD_SETSIZE 1024
[15:49] pieterh #endif
[15:49] sustrik afaiu, the problem is that 0mq and the application have to have same value of FD_SETSIZE to work well
[15:49] sustrik right?
[15:49] pieterh it's just an array max for the select call
[15:49] mato no, i don't think so
[15:50] pieterh so any code that shares fdsets must agree on the max
[15:50] mato sustrik_: libzmq.dll's FD_SETSIZE AFAICT has nothing to do with the app's FD_SETSIZE
[15:50] pieterh mato: there is no 'app's FD_SETSIZE'
[15:50] mato huh?
[15:50] pieterh every select call has an FD_SETSIZE
[15:50] pieterh it's not per application or per binary
[15:50] pieterh it's per select() call...
[15:50] mato sure
[15:50] mato my point is...
[15:51] mato what if you have an existing application which is mixing normal sockets and zmq code
[15:51] mato including zmq.h should not go and magically change that app's FD_SETSIZE...
[15:51] pieterh (a) that is 1% of the case where as 'normal 0MQ apps' are 99% of the case
[15:51] pieterh (b) you can set your own FD_SETSIZE beforehand
[15:51] pieterh (c) you need to recompile ALL your code then to match
[15:51] pieterh including 0MQ
[15:51] pieterh that's pretty straight-forward
[15:52] mato does it really work that way? (c) included?
[15:52] pieterh what is not acceptable is "install 0MQ, build app, watch it break at 64 sockets'
[15:52] pieterh yes mato
[15:52] pieterh define the macro in your project, rebuild
[15:52] pieterh period
[15:52] pieterh i've been doing this for 15 years or so with Xitami / SFL /...
[15:52] mato ok, ok, i believe you
[15:53] mato i was just trying to avoid zmq.h doing anything "surprising"
[15:53] pieterh the _only_ difficulty here is that 0MQ includes winsock 4 times in different places
[15:53] pieterh which is... less than elegant
[15:53] mato yes, that needs to be refactored
[15:53] mato oh
[15:53] mato there is one other complexity
[15:53] mato when building zmq on windows
[15:53] pieterh yes?
[15:54] mato FD_SETSIZE must be defined correctly the 1st time winsock.h is included in the app
[15:54] pieterh there is no 'app'
[15:54] mato which for zmq basically means changing all those places
[15:54] pieterh you keep speaking of 'apps'
[15:54] mato i'm sorry, zeromq *library*
[15:54] pieterh every source file
[15:54] pieterh is compiled with a certain value for FD_SETSIZE
[15:54] pieterh winsock.h is included 4 times in different places
[15:55] pieterh some are obviously redundant (select.cpp, select.hpp)
[15:55] pieterh others are less so
[15:55] sustrik can someone explain what FD_SETSIZE actually affects? if every call to select can have different FD_SETSIZE why not define it to 1024 in our cpp files and leave the header alone?
[15:55] pieterh sustrik_, it tells the win32 library how large the fdset arrays are
[15:56] sustrik and can it differ throughout the program?
[15:56] mato no one knows :-)
[15:56] pieterh if you include winsock.h or winsock2.h without defining this, it sets a value of 64
[15:56] pieterh so each source file can have different values compiled into it
[15:56] pieterh winsock.h passes the macro value to the win32 functions
[15:56] pieterh so it can differ in every call to select()
[15:57] pieterh does that make sense?
[15:57] mato if that's how it actually works, then yes, i think so
[15:57] sustrik then why not go for defining FD_SETSIZE only for select.cpp and zmq.cpp?
[15:57] pieterh sustrik: if those are the only places you call select(), yes
[15:57] pieterh in fact only in select.cpp then
[15:57] mato zmq.cpp also calls select()
[15:57] sustrik yes
[15:57] pieterh right
[15:57] mato for the zmq_poll() implementation
[15:58] pieterh ok, i'll make a patch and ask for volunteers to test that under win32
[15:58] pieterh if it's stable we can include it
[15:58] pieterh zmq.h and select.cpp only then
[15:59] mato zmq.h?
[15:59] sustrik noo
[15:59] mato zmq.cpp you mean
[15:59] pieterh well, it has to be defined _before_ including winsock.h
[15:59] pieterh that happens in zmq.h
[15:59] mato yes, so?
[16:00] pieterh so you want me to add this code to the very start of zmq.cpp?
[16:00] pieterh wow
[16:00] mato then put the #define in zmq.cpp before it include anything
[16:00] mato yes
[16:00] pieterh no way
[16:00] pieterh that's so ugly
[16:00] mato the whole FD_SETSIZE thing is ugly
[16:00] mato not our fault
[16:00] pieterh 'here's our global header file but wait, some magic stuff happens before that!'
[16:00] pieterh no, it's the fault of this code structure
[16:00] pieterh it should be done in windows.hpp
[16:01] pieterh or somewhere for 'include all random windows stuff'
[16:01] mato yes, but not in zmq.h
[16:01] mato that's the only point i'm making, pieter
[16:01] mato do it any way you like, just not in zmq.h please
[16:01] sustrik windows.hpp is actually a nice place to set it
[16:01] mato if you clean up the mess with windows.hpp while you're at it, so much the better
[16:01] pieterh you're not making a point, you're just imposing an arbitrary limitation
[16:02] mato it's not arbitrary
[16:02] sustrik as it's included from all 0mq compilation units
[16:02] sustrik but not from zmq.h
[16:02] pieterh it should be _consistent_ for the whole library
[16:02] pieterh and it should be easy to override from the project file
[16:02] pieterh period
[16:02] pieterh look, I'm willing to spend some effort cleaning this up, but not to navigate random opinions
[16:02] mato sustrik_: do you understand why making the change in zmq.h seems like a bad idea? or am I alone in this?
[16:02] pieterh the windows build of 0MQ is broken
[16:02] sustrik mato: i do
[16:02] pieterh 64 sockets is not realistic, period
[16:03] pieterh you are both wrong, then
[16:03] sustrik pieterh: try putting it into windows.hpp, that should work ok
[16:03] pieterh any application that mixes tcp and 0mq must in any case define a FD_SETSIZE
[16:03] pieterh externally
[16:03] pieterh in its project file
[16:03] pieterh this applies to any library that wraps select()
[16:03] pieterh sustrik: nope, windows.hpp is not used by select.cpp for instance
[16:04] sustrik it should
[16:04] pieterh yeah, it should
[16:04] pieterh hence my original question about the code structure
[16:04] pieterh the platform code shouldn't be spread out like that IMO
[16:04] sustrik historic reasons
[16:04] pieterh sorry, poor excuse :-)
[16:05] sustrik well, fix it then
[16:05] mato exactly
[16:05] mato stop bitching about it
[16:05] mato and fix it
[16:05] pieterh that's all I was asking for
[16:05] mato all i'm saying, and sustrik is agreeing, is don't change FD_SETSIZE in zmq.h
[16:05] mato and all of this *can* be done without that
[16:05] pieterh i'm curious to see how you will do that, mato
[16:06] mato ?
[16:06] pieterh since zmq.h includes winsock.h
[16:06] pieterh well, since you know best, I step back
[16:06] mato yes, so you make sure FD_SETSIZE is defined before zmq.h is included in the relevant places
[16:07] pieterh that IMO makes the code dirtier and I won't do that, sorry
[16:07] pieterh zmq.h is the system-wide header file
[16:07] sustrik afaiu, each compilation unit has to have a consistent value of FD_SETSIZE
[16:07] mato it's not the syste-wide header file
[16:07] mato it's the exported API header file
[16:07] sustrik but there's no requirement that different compilation units have to have the same value
[16:07] pieterh then it should absolutely not call winsock.h
[16:07] pieterh because today it effectively sets FD_SETSIZE to 64
[16:08] pieterh don't you get this?
[16:08] pieterh zmq.h already makes a mess for any app
[16:08] mato *that* is an entirely different problem
[16:08] pieterh ?
[16:08] mato the reason it's included winsock.h is...
[16:09] mato that the zmq_pollitem_t definition needs a type for "OS sockets"
[16:09] mato so it needs access to SOCKET
[16:09] sustrik yeah, that's a really annoying problem
[16:09] pieterh it sets FD_SETSIZE to 64
[16:09] sustrik in reality SOCKET is void*
[16:09] pieterh that is a serious fault
[16:09] pieterh i'm proposing to make it a less serious faulyt
[16:09] pieterh *fault
[16:10] mato no you're not
[16:10] sustrik what about using void* instead of SOCKET
[16:10] mato well, would that work?
[16:10] sustrik it's not nice but all the other alternatives are even worse
[16:10] pieterh "zmq.h should not magically go and change the app's FD_SETSIZE":
[16:10] pieterh but that's what it does today
[16:11] sustrik i think if holds for all win platforms
[16:11] sustrik hard to check though
[16:11] mato well, there's an easy way to check actually
[16:11] pieterh look, there are several problems here
[16:11] sustrik mato: how?
[16:11] pieterh - windows code is spread out through multiple header files
[16:11] mato sustrik_: in zmq_init, assert (sizeof (SOCKET) == sizeof (void *))
[16:11] pieterh - select() is used with bogus limit of 64
[16:11] sustrik :)
[16:11] mato sustrik_: that would do for starters...
[16:11] sustrik ok, that solves the problem, no?
[16:12] mato it would seem so, yes
[16:12] pieterh yes, that solves zmq.h
[16:12] sustrik when we have that
[16:12] pieterh if zmq.h no longer whacks FD_SETSIZE then it can be safely set per source file where needed
[16:13] mato sustrik_: in fact, you can add a compile-time dummy check
[16:13] mato sustrik_: even better
[16:13] sustrik good
[16:13] mato sustrik_: SOCKET foo; void *bar; bar = foo
[16:13] mato sustrik_: or something like that
[16:13] sustrik ok, however, let's do this after 2.1.0 release
[16:14] sustrik it can break 0mq on some win systems (if we are wrong about void*)
[16:14] mato 2.1.0 is development
[16:14] mato it shoudn't matter too much
[16:14] sustrik thus it would be good to have one functional dev release
[16:14] mato up to you
[16:14] mato anyway, looks like we have a sensible solution
[16:15] sustrik pieter: you can fix it on a topic branch
[16:15] sustrik we can merge it in after 2.1.0 is out
[16:17] pieterh sustrik_, sure
[16:17] pieterh I submit a patch, do what you like with it :-)
[16:20] pieterh please tell me when you've removed the winsock.h call from zmq.h, then I'll make the change
[16:23] mato sustrik_: fd_t is the canonical type used internally for sockets, right?
[16:23] sustrik yes
[16:23] mato sustrik_: i.e. it's SOCKET on windows and int everywhere else
[16:29] sustrik mato: in zmq.h we are using _WIN32 to find out whether we are on windows or somewhere else
[16:30] sustrik does that meen windows.h has to be included prior to zmq.h?
[16:30] mato sustrik_: _WIN32 should be defined by the compiler
[16:30] sustrik ok
[16:30] sustrik what abour win64 btw?
[16:30] mato i hope so anyway, otherwise it's totally braindead
[16:30] mato _WIN32 just means "Win32 API"
[16:31] sustrik i see
[16:31] mato irrelevant to x64 target
[16:35] sustrik ok, i've checked it, MSVC does define _WIN32
[16:35] sustrik good
[16:35] sustrik hopefully, MinGW does the same thing
[16:35] mato it must
[16:41] sustrik i'm checking what SOCKET actually is :(
[16:41] mato oh dear ... :-)
[16:41] sustrik if _WIN64 is defined
[16:42] sustrik it's uinsigned __int64
[16:42] sustrik otherwise it's
[16:42] sustrik __w64 unsigned int
[16:42] sustrik whatever that means
[16:48] pieterh sustrik_, __w64 just means "this integer can be a pointer, safely"
[16:48] pieterh __w64 unsigned int on a 64-bit Windows box is the same as __int64
[16:49] pieterh 'unsigned __int64', sorry
[17:16] mato sustrik_: event_t::reset () should also be restarted automagically on EINTR in signaler_t::recv() ?
[17:17] mato sustrik_: or should it just drop out with EINTR?
[17:19] sustrik mato: if it's hard you can drop it
[17:19] mato sustrik_: it's not hard, i'm asking what it should do...
[17:19] mato sustrik_: what does "set" and "reset" mean, exactly?
[17:20] mato sustrik_: semantically...
[17:21] mato sustrik_: bah, /me is confused...
[17:23] mato sustrik_: i think i need to look at the signaler code together with you, otherwise i'll get it wrong
[17:24] mato sustrik_: anyway, i do have a nicely refactored event.hpp/event.cpp that tries to minimize duplication
[17:25] mato sustrik_: what needs to be protected by the mutex? queue and signaled or just queue?
[17:51] CIA-21 zeromq2: 03Martin Lucina 07wip-signaler * rf78cb39 10/ (8 files in 3 dirs):
[17:51] CIA-21 zeromq2: WIP version of new signaler_t/event_t
[17:51] CIA-21 zeromq2: Signed-off-by: Martin Lucina <mato@kotelna.sk> - http://bit.ly/9BPDCp
[17:52] Guthur sustrik: Thanks for the comments
[17:53] Guthur unfortunately C# doesn't support friends, I had thought of that
[17:54] Guthur And I didn't realise about spaces vs tabs
[17:54] sustrik it makes the code ugly when your editor is set differently
[17:55] Guthur No problem
[17:55] Guthur I'll use spaces
[17:55] sustrik anyway, if there are no friends, it probably have to stay as is
[17:55] Guthur The reason for Queue methods was really to just improve the interface slightly
[17:56] Guthur but I can understand wanting to reflect the underlying API as much as possible
[17:56] Guthur Queue, Forwarder, and Streamer methods*
[17:57] sustrik yes
[17:58] sustrik also, it's easier to tweak constants
[17:58] sustrik than tweaking functions
[17:58] sustrik in case anything changes in the future
[17:58] Guthur Reasonable enough, I'll change to reflect your suggestion
[17:59] sustrik btw, the clrzmq project is kind of orphaned, do you want to help with it?
[18:00] sustrik how it can be done is you'll get write access to the repo
[18:00] sustrik if it is rather one-off patch, then send the patch to the mailing list
[18:01] Guthur sustrik_, Sure no problem
[18:02] Guthur is Jeffrey no longer involved?
[18:04] sustrik haven't heard anything about clrzmq for quite a long time
[18:04] sustrik but maybe i've just missed it?
[18:05] sustrik last update aug, 30
[18:06] Guthur Has there been many interface changes to zmq since then?
[18:06] Guthur or is it mostly under the hood
[18:06] sustrik it's under the hood
[18:06] sustrik some new socket options
[18:07] Guthur I'll try to work through the examples and add the missing ones
[18:07] Guthur Good way to find any missing features I would imagine
[18:07] sustrik just have a look at zmq.h
[18:07] Guthur that would also work, hehe
[18:08] sustrik for the master branch there are these new options:
[18:08] sustrik #define ZMQ_FD 14
[18:08] sustrik #define ZMQ_EVENTS 15
[18:08] sustrik #define ZMQ_TYPE 16
[18:08] sustrik #define ZMQ_LINGER 17
[18:08] sustrik #define ZMQ_RECONNECT_IVL 18
[18:08] sustrik #define ZMQ_BACKLOG 19
[18:08] sustrik first two are irrelevant from clrzmq's point of view
[18:08] sustrik that leaves us with 4 new options
[18:10] Guthur what about ZMQ_RCVMORE
[18:11] sustrik it's not there?
[18:11] Guthur not in what I cloned
[18:11] Guthur let me double check github
[18:11] sustrik what's your name on github?
[18:12] Guthur Guthur, though I notice it gives my real name in the commits
[18:12] Guthur Its not in the clrzmq
[18:12] Guthur ZMQ_RCVMORE that is
[18:12] sustrik ok, you have commit access to clrzmq
[18:13] sustrik be carefull not to break it
[18:13] sustrik let me check...
[18:13] mato sustrik_: should i be disabling interrupt coalescing only for RX or also for TX?
[18:13] toni hi there. I am using zmq with python. The docs of pyzmq only cover sockets. Does anybody know where I can find more information about pyzmq?
[18:13] sustrik Guthur: there's no getsockopt function at all :(
[18:14] sustrik mato: both
[18:14] Guthur sustrik_: Definitely needs some work then
[18:14] sustrik toni_: what do you mean?
[18:14] Guthur I'll try to bring it up to current
[18:15] sustrik Guther: great!
[18:16] toni sustrik_: I downloaded the pyzmq. Building the docs with sphinx I only get some info about the methods of the socket object. I need a more detailed documentation of the python api
[18:16] sustrik socket object presumably _is_ python api
[18:18] mato sustrik_: hmm, i don't entirely trust this switch
[18:18] mato sustrik_: but the results I see *appear* to show that I get ~60us latency with the old code, and ~78us with the new cose
[18:18] mato *code
[18:18] sustrik uhm
[18:19] sustrik ok, we'll need some optimisation
[18:19] mato sustrik_: throughput is weird, it seemed to fluctuate but now appears stablew
[18:19] sustrik throughput is a weird metric
[18:19] mato I guess ideally I should test w/o the switch, no?
[18:19] sustrik fluctuating almost by definition
[18:19] sustrik ideally, but normally switch is not the bottleneck
[18:20] mato well, it's only 100 EUR worth of switch so it won't be *that* great
[18:20] mato i can test tomorrow with a direct link, not now...
[18:20] sustrik i don't think it's needed
[18:20] sustrik the latency test is relevant
[18:20] sustrik 18us delta is too much
[18:20] mato ok, well that consistently shows the same result
[18:21] sustrik we have to optimise
[18:21] mato well, let's sit down and look at it together tomorrow
[18:21] sustrik ok
[18:21] mato now, beer?
[18:21] sustrik allegedly, there's some klezmer-tango concert in nuspirit club
[18:22] mato ah, yes, someone did mention something the other day
[18:22] mato i forget who
[18:22] sustrik your cousin?
[18:22] Guthur sustrik_: By the looks of it someone imported the setsockopt twice instead of doing getsockopt
[18:22] mato dunno
[18:22] sustrik Guthur: quite possible
[18:22] sustrik as I said, the project is kind of orphaned
[18:23] sustrik looks like .net devs have not much love for open source
[18:23] Guthur yeah that tends to be the case
[18:24] Guthur I would have thought mono might be reversing the trend
[18:24] Guthur I'm only interested in it because I am paid to be, hehe
[18:25] sustrik i'm afraid that's the case with most people using the .net binding
[19:27] PythonMQ hello, does anyone knows where can I find a compative version of pyzmq for old Python 2.4?
[19:28] PythonMQ when I try to create an extension for Python 2.4, it asks for .nET 1.1 SDK, Vs 2003, etc... which I dont have
[20:32] mikko good evening
[21:18] sustrik travlr: hi, are you there by chance?
[21:39] Guthur sustrik_, when setting the identity through setsockopt is it string length + 1?
[21:39] Guthur for the optvallen
[21:40] mikko Guthur: no need for +1 as you give the length
[21:41] mikko as far as i remember it doesnt matter as long as it's consistent
[21:41] Guthur oh ok, its just unless I did that it was corrupting the last char
[21:41] Guthur but it could be something wrong with the C# marshal code
[21:42] sustrik i think mikko is right
[21:42] sustrik the identity is a binary blob, not a string
[21:43] sustrik if it corrupts the last byte, it's a bug
[21:44] Guthur ok, I'll investigate a little
[22:53] Guthur sustrik_, should I do some checking so that people don't do silly things like set an affinity of -1

Web site design and content is copyright © 2010-2011 Peter Alexander, ProDataLab. Site content licensed under the Creative Commons Attribution-Share Alike 3.0 License. ØMQ is copyright © Copyright © 2007-2011 iMatix Corporation and Contributors. Contact us for professional support. ØMQ is free software licensed under the LGPL. ØMQ, ZeroMQ, and 0MQ are trademarks of iMatix Corporation. Terms of Use ? Privacy Policy