MATLAB Answers

1

Data transfer specifics for a queue

Asked by Jim Hokanson on 6 Jan 2018
Latest activity Commented on by Jim Hokanson on 6 Jan 2018
Documentation for parallel.pool.DataQueue and parallel.pool.PollableDataQueue says next to nothing about how data is transferred between the client and workers. I have two questions in particular about this functionality, although any related details would be much appreciated.
  1. What can or can't be sent through the queue. During debugging I sent a daq session through the queue and although it appeared to transfer fine all the properties of the class were empty.
  2. My understanding is that workers and the client are in separate processes (with separate memory management). When sending data through a queue what sort of locking takes place to transfer data between the processes? For example, does the worker somehow lock the client's memory management system while sending a message to the client, and similarly, have to wait for that lock if the client is doing memory allocation?

  0 Comments

Sign in to comment.

1 Answer

Answer by Walter Roberson
on 6 Jan 2018
 Accepted Answer

A message type interface is used, implemented through Java. The source elements are serialized, the serialized version transmitted, and the destination deserializes. The destination receives the information when convenient for it, so it cannot interfere with memory allocation.
Objects that are controlling a hardware resource cannot usually transfer the resource control through the queue.

  4 Comments

Show 1 older comment
Also, I'm really surprised it is done in Java since most of the data handling is done in C++. Do you have a source for this? I don't think it changes much, other than perhaps heap limit concerns, but I'm still curious.
You can read the source.
toolbox/distcomp/cluster/+parallel/+pool/DataQueue.m calls toolbox/distcomp/cluster/+parallel/+internal/+pool/AbstractDataQueue.m which does
import com.mathworks.toolbox.distcomp.pmode.DataMessageDispatcher
and then does DataMesageDispatcher.createRemoteQueue and for that,
>> which com.mathworks.toolbox.distcomp.pmode.DataMessageDispatcher.createRemoteQueue
createRemoteQueue is a Java method % static method or package function
I always forget to try reading the source code! Thanks.

Sign in to comment.