Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

To resolve issues starting MATLAB on Mac OS X 10.10 (Yosemite) visit: http://www.mathworks.com/matlabcentral/answers/159016

error with parfor and big matrices

Asked by Kathrin on 19 Jan 2013

I have a code which uses large multidimensional matrices inside a parfor loop. However, for the size of matrices I need to use, I get the following error (for smaller matrices there is no problem):

Warning: Error caught during construction of remote parfor code.
The parfor construct will now be run locally rather than
on the remote matlabpool. The most likely cause of this is
an inability to send over input arguments because of a
serialization error. The error report from the caught error is:
Error using distcompserialize
Error during serialization
Error in distcomp.remoteparfor (line 36)
    obj.SerializedInitData =
    distcompMakeByteBufferHandle(distcompserialize(varargin));
Error in parallel_function (line 437)
        P = distcomp.remoteparfor(W, @make_channel, parfor_C);
> In parallel_function at 450 

If I run the same code with for instead of parfor, the program runs through without any error. My code looks like this (a simple example which throws this warning):

matlabpool open
BigMat = randn(30,30,30,30,30,30);
parfor i = 1:20
    CopyOfBigMat = BigMat;
end
matlabpool close

Also, I tried to run it with only one worker, but I still get the same error. So I don't think that the problem is caused by not enough physical memory (I run 64bit R2012b with 16GB RAM).

Since my problem is similar to the one posted here

http://www.mathworks.com/matlabcentral/answers/20126-parfor-problems

I also tried to use the WorkerObjWrapper function which was suggested in that post. My code then looks like this:

matlabpool open
BigMat = randn(30,30,30,30,30,30);
w = WorkerObjWrapper(BigMat);
parfor i = 1:20
    data = w.Value; %#ok<*PFBNS>
    CopyOfBigMat = data;
end
matlabpool close

but this actually throws another error:

Error using distcompserialize
Error during serialization
Error in spmdlang.RemoteSpmdExecutor/initiateComputation (line 82)
                fcns  = distcompMakeByteBufferHandle( ...
Error in spmdlang.spmd_feval_impl (line 14)
    blockExecutor.initiateComputation();
Error in spmd_feval (line 8)
        spmdlang.spmd_feval_impl( varargin{:} );
Error in WorkerObjWrapper>(spmd) (line 155)
            spmd
Error in WorkerObjWrapper/workerInit (line 155)
            spmd
Error in WorkerObjWrapper (line 97)
            WorkerObjWrapper.workerInit( tmpId, ctor, args, dtor );

Does anyone know what causes the error during parfor? And has anyone found a solution/workaround for it? Thanks a lot!

0 Comments

Kathrin

2 Answers

Answer by Matt J on 19 Jan 2013
Edited by Matt J on 19 Jan 2013
Accepted answer

Seems like it could be a memory error to me, in spite of your insistence that it isn't. BigMat is 5.4 GB, so a matlabpool of even 2 workers will result in 3 clones of BigMat, easily exceeding your 16GB RAM. Though you say it works with 1 worker, this will still result in a minimum of 11GB memory consumption and 5.4GB seems like a lot to transmit. Maybe there's some sort of time-out error? Or maybe you have other arrays of similar size as BigMat in the workspace that are causing memory limits to be exceeded?

In any case, comparing PARFOR to FOR isn't meaningful, because an ordinary for-loop doesn't do any data cloning.

6 Comments

Edric Ellis on 22 Jan 2013

I think Matt is exactly right here - you need to work out how to fabricate/load/whatever the data directly on the workers.

Kathrin on 25 Jan 2013

Thank you both for your answers. I tried to follow your advice but unfortunately didn't succeed in my setting. This is what I do:

The code solves an economic model where agents optimize their current behavior taking into account the effects on the future. Hence I solve the model backwards over time (using a for loop) and the optimization is done inside the parfor loop:

for time = T:-1:1
   parfor state=1:S    [solve for all independent states]
     [optimize given results from time+1]
   end
end

Unfortunately, I don't see how I can generate the data directly on the workers in this case. And loading it within the parfor loop is even slower than running it without parallelization given the large number of states S.

So for now I am running the code without parallelization because the matrices I need are bigger than the 2GB which seem to be the limit of how much can be transferred to the workers. This is a shame, in economic models you could benefit a lot by parallelization, but this limit makes it impossible for sophisticated models which need to use big matrices.

Matt J on 25 Jan 2013

Could you perhaps elaborate on the structure of the computations, explaining among other things why the entire data set is needed by each parallel job? Parallel processing would make more sense here if each job required only a piece of the data, so that you could then slice the data up into smaller pieces and distribute them to the workers. Or possibly you could use a GPU-based approach, which is better designed for sharing large amounts of data (assuming you have a graphics card with enough RAM).

Matt J
Answer by Edric Ellis on 1 May 2013

Please note that this problem is fixed in R2013a.

3 Comments

Kathrin on 6 May 2013

Edric, thank you very much for this update!

Please, could you elaborate on the details of the change? Does that mean that there is no built in limit anymore (other than physical RAM of course) in how much data can be transferred to the workers in the parfor loop?

Thank you!

Edric Ellis on 7 May 2013

Hi Kathrin - for systems where the MATLAB client and matlabpool workers are all 64-bit, we can send data to and from PARFOR loops up to the limit of the amount of RAM on your system.

Kathrin on 7 May 2013

That's great news, thank you!

Edric Ellis

Contact us