Parallel Computing Toolbox: Lost Data when Error occur

Hi there,

recently I have been using Parallel Computing Toolbox to help speed up the simulation with matlabpool, parfor, etc.

However, if somehow an error occurs (lost connection to lab 2 somehow), the final results that I want would not be stored in the workspace. It is unlike evaluating cell, where all of your results so far would still be stored before the point when an error occurs.

Since PCT could also take quite a lot of time to run the simulation, it would be a disaster if we don't have anything in the end...

If it could store the so-far results when an error occur, then I can still run the same code at the point when an error occurs and continue the rest of simulation.

Is there anyway to do that?

thanks,

Answer 1

[Edric Ellis]

There's no built-in way to checkpoint your computations. You could either split your PARFOR loop into smaller chunks and save the results as you go, or have the workers save partial results to disk. Having the workers save partial results is a bit tricky - you'd probably want to save the results in each iteration of the PARFOR loop to a file named after the iteration of the loop.