Thread Subject: Shared memory in parfor

Dear list,

Is there a way to declare that a variable is shared and not
changed during a parfor loop? A variable containing problem
data can be pretty large. When the data is read-only, there
is no need to copy the data to all of the workers on a
shared memory system.

Example code:

data = rand(N, N); % generate some (static) problem data
parfor j=1:N
  res = do_something(data, j);
  output(j) = res;
end

% data is unchanged

Sincerely,
Sebastiaan

Sebastiaan,

Unfortunately, there is currently no way of achieving this
with parfor. Out of curiosity, how many MB is the problem
data that you have?

Thanks,

Narfi

"Sebastiaan " <s.breedveld@erasmusmc.REMOVE.BOO.BOO.nl>
wrote in message <ftv8ea$19m$1@fred.mathworks.com>...
> Dear list,
>
> Is there a way to declare that a variable is shared and not
> changed during a parfor loop? A variable containing problem
> data can be pretty large. When the data is read-only, there
> is no need to copy the data to all of the workers on a
> shared memory system.
>
> Example code:
>
> data = rand(N, N); % generate some (static) problem data
> parfor j=1:N
> res = do_something(data, j);
> output(j) = res;
> end
>
> % data is unchanged
>
> Sincerely,
> Sebastiaan

Dear Narfi,

the problem data is currently ~400MB, but could grow to a
few GBs in the feature. Currently, it takes longer to
transfer the data to 4 threads than a single thread execution.

The only alternative I can think of is to split the data a
priori, and then submit the jobs to the scheduler.

However, this still doubles the memory usage (assumed that
the data is not copied as well), and assumes that the data
can be split.

Sincerely,
Sebastiaan

Hi all. I have also had this problem and wrote a mex program to handle this situation. It requires a POSIX compliant system, such as Linux.
http://smlv.cc.gatech.edu/2010/08/27/shared-memory-in-matlab/

Cheers,

Josh


"Sebastiaan " <s.breedveld@erasmusmc.REMOVE.BOO.BOO.nl> wrote in message <fu2t91$l6v$1@fred.mathworks.com>...
> Dear Narfi,
>
> the problem data is currently ~400MB, but could grow to a
> few GBs in the feature. Currently, it takes longer to
> transfer the data to 4 threads than a single thread execution.
>
> The only alternative I can think of is to split the data a
> priori, and then submit the jobs to the scheduler.
>
> However, this still doubles the memory usage (assumed that
> the data is not copied as well), and assumes that the data
> can be split.
>
> Sincerely,
> Sebastiaan
>
>