Rank: 5327 based on 15 downloads (last 30 days) and 1 file submitted
photo

dan

E-mail
Company/University
University of Sydney

Personal Profile:
Professional Interests:

 

Watch this Author's files

 

Files Posted by dan
Updated   File Tags Downloads
(last 30 days)
Comments Rating
05 Aug 2014 mat2piv.m Flatten an N-dimensional tensor into an Excel pivottable ready form Author: dan pivot, tensor, pivottable 15 0
Comments and Ratings by dan View all
Updated File Comments Rating
19 Jul 2011 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon

Joshua, thank you for this routine. It's a godsend.

When writing to a shared variable inside a parfor loop I found I couldn't use shared_x=y I had to use shared_x(1:end)=y(1:end). Like this:

sharedmatrix('clone', x_key, x); clear x;
parfor 1:10
local_x = sharedmatrix('attach', x_key);
local_y = rand(size(local_x));

% THIS DOES NOT WORK
local_x = local_y;

% but this does
local_x(1:end) = local_y(1:end);

end; %end parfor

I assume this has to do with the mysteries of MATLAB copy-on-write. Anyway just wanted to post this important tip.

ps. This is Matlab r2010b on Linux x64.

26 Mar 2011 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon

Further tests reveal... yes, top includes shared memory in its report of each process's RES and VIRT usage.

Joshua this may be worth a mention in the documentation.

25 Mar 2011 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon

Another VERY grateful user; thank you for this contribution!

But in my tests and monitoring using top it seems that each parfor process creates a local copy of the shared global variable. Does 'top' include shared memory when reporting each process' RAM use?

Here's my test:

matlabpool('open');
C = rand(13000,13000);
shmkey=1234567;
shmsiz=sharedmatrix('clone',shmkey,C);
clear C;

parfor i = 1:5, MyTest(shmkey, i); end;

sharedmatrix('free',shmkey);

And MyTest.m():

function MyTest( shmkey, threadID )

pC = sharedmatrix('attach',shmkey);

%% This operation should not require a local copy of pC since it should be read-only.
result = nnz(pC);
result = result * threadID;
whos('pC'); % reports 1.3gb
disp(result);

sharedmatrix('detach',shmkey,pC);
clear pC;

end

When I run this test and monitor it using top, it shows each worker MATLAB process momentarily using >1gb of RAM.

Is this an artifact of 'top' -- does top report process memory inclusive of shared memory it's accessing? Or does nnz(x) actually create a local copy of x in each worker?

Contact us