IRC Log


Tuesday April 20, 2010

[Time] NameMessage
[10:58] sjampoo Quick question, zmq_msg_init_data(3) is relatively cheap to call right?
[10:59] sjampoo run in to the following issue, while working with Python bindings and non-copy. I want resend a message received with zmq_recv, but i think i have to reinit that with zmq_msg_init_data to get access to the callback
[11:01] sjampoo Because if i just resend the received message again without explicitly closing+reinitiazing it i can run in all sorts of problems.
[11:07] sustrik sjampoo: i don't know how pyzmq works
[11:07] sustrik however, with raw C API
[11:07] sustrik sending a message deinitialises it
[11:07] sustrik there's no copy made, existing copy is sent to I/O thread
[11:08] sustrik if you want to send the same message to multiple sockets
[11:08] sustrik you have to copy it
[11:08] sustrik (zmg_msg_copy)
[11:08] sjampoo aha
[11:08] sustrik note that zmq_msg_copy actually just adds refcount rather than doing deep copy
[11:09] sjampoo Yes.
[11:09] sjampoo why is the callback function attached to the message and not to the close call?
[11:13] sustrik message is handed to the I/O thread
[11:13] sustrik once that's done user cannot access it
[11:13] sustrik so he has no way to supply the free function
[11:32] sjampoo hmm it makes a little bit more sense now, i never understood the existence of msg_copy and msg_move
[11:33] sjampoo So, what do you mean with de-initializes the message on send? What would happen if you would still send the message with the C-api?
[11:35] sustrik it would be empty
[11:35] sustrik the whole point is zero-copy
[11:35] sustrik message object owns a buffer
[11:36] sustrik when you send the message, buffer is passed to the I/O thread
[11:36] sustrik message has no buffer any more
[11:39] sjampoo Okay, say i'll have a message M1 i will have to copy it to M2 before sending M1 out. Now what does happen if i send M1?
[11:39] sjampoo M2 is just a pointer to M1 I believe with an added ref. count to M1
[11:44] sustrik exactly
[11:46] sjampoo but then, if i have send M1 out, M2 would point to a deinitialzed message right?
[11:52] sustrik actually bith M1 and M2 point to shared buffer
[11:52] sustrik ref count is 2
[11:52] sustrik then you send M1
[11:53] sustrik M1 is deinitialised, but the buffer is passed to I/O thread
[11:53] sustrik refcount is still 2
[11:53] sustrik when I/O thread sends the message it doesn't need it any more so it decerements the refcount and drops the pointer
[11:53] sustrik now the refcount is 1
[11:54] sustrik (only M2 points to the buffer)
[12:32] sjampoo ok, got it. thanks for the info.
[13:37] pieter_hintjens anyone familiar with MPI or OpenMP?
[13:37] pieter_hintjens especially how these compare to 0MQ...
[14:04] sustrik pieter: i have a dim idea
[14:11] sophacles pieter_hintjens: compare in a numbers/performance sense or compare in a user interface (api) sense?
[14:12] sophacles in the case of the latter, 0mq is nicer for pure messaging, but mpi does cool rpc and call balancing stuff for you... it really depends on the use case
[14:14] sophacles also mpi has semantics for addressing remote memory a bit easier, so its real nice for distributed matrix multiplies and the like
[14:16] sophacles the downside to mpi is that unless you really need the features for parallel computing, it is way too complex.
[14:41] pieter_hintjens sophacles: thanks, this is helpful
[14:43] sophacles y/w
[16:13] mikko hello
[16:13] sophacles oh cool: talk tomorrow titled: Towards Secure Message Systems. ill send the list my notes :) (also if anyone is in the middle of illinois i can send you details)
[16:15] sustrik mikko__: hi
[16:15] sustrik sophacles: sounds interesting
[16:25] sophacles yeah the abstract is a little.. er.. abstrct, so im only 50% on if it actually applies to 0mq but there is also pizza there, so i figured at worst i still get lunch out of it.