How to SAVE data to disk in a background thread?

21 views (last 30 days)
Has anyone figured out a way to run the SAVE command in a background 'thread', so that MATLAB can keep on running code while the file is getting written to the disk?

Accepted Answer

Jan
Jan on 13 Jan 2013
You can call a MEX-function, which opens a new thread, and the main thread returns to Matlab. The thread running in the background is not allowed to call any Matlab functions, so anything starting with "mx..." and "mex..." must be avoided. When you want to write a binary file, fwrite is fine.
You have to be sure, that the data are not changed during writing. E.g. a pointer to the data of a Matlab array might lead to inconsistencies, when the data are modified or free'd during the writing. But this concerns all multi-threading applications, such that I assume, that you have solved this already.
  3 Comments
Jan
Jan on 14 Jan 2013
I cannot imagine that this could be faster than doing this in a single-threaded application. You can create a standalone application, which is able to save MAT files. Perhaps you can forward the values using http://www.mathworks.com/matlabcentral/fileexchange/28572-sharedmatrix.
Eric Sampson
Eric Sampson on 14 Jan 2013
Hi Jan, I understand that the SAVE operation itself would not be any faster, and that's OK - the situation I'm encountering is that we are writing a lot of data to disk, so each call to save can take 20+ seconds. I'd like to find a way to initiate the save in the background and then immediately return to executing the next line of our code...

Sign in to comment.

More Answers (1)

Image Analyst
Image Analyst on 14 Jan 2013
I don't have it, but doesn't the Parallel Computing Toolbox have that capability?
  1 Comment
Eric Sampson
Eric Sampson on 14 Jan 2013
I think you're right IA, looks like I could use the PCT to fire off a batch job on a local worker that runs a function to save the variables to disk. I just would need to justify the toolbox cost :) This seems like basically similar to the 'home grown' approach described above of using a MEX file to spawn a MATLAB Engine process which then does the save. Just a decision of 'build or buy' I guess!

Sign in to comment.

Categories

Find more on MATLAB Parallel Server in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!