Why is saving a large matrix to a MAT file very slow in MATLAB 7.7 (R2008b)?

18 views (last 30 days)
I have a single huge (4 GB) variable of type double. I want to save it to a MAT file and reload it at many different places in my application to use it. I am experiencing that it takes a very long time to write it to a MAT file.
Here is what I am trying to do:
%---------USING SAVE/LOAD---------
>> a=rand(5e9/8,1);
>> tic,save -v7.3 largefile.mat a,toc
Elapsed time is 370.750804 seconds.
>> clear a
>> tic,load largefile.mat,toc
Elapsed time is 55.672785 seconds.
When I compare this with FWRITE I get significantly improved performance.
%--------USING FWRITE/FREAD----------
>> tic, fid=fopen('bigfile','wb'), count=fwrite(fid,a,'double'),fclose(fid),toc
fid =
3
count =
625000000
Elapsed time is 68.695837 seconds.
>> tic, clear a, fid=fopen('bigfile','rb'), a=fread(fid,'double');
fclose(fid), toc
fid =
3
Elapsed time is 16.190452 seconds.

Accepted Answer

MathWorks Support Team
MathWorks Support Team on 26 Apr 2016
Saving to a MAT file can be slower than saving to a raw bit binary file.
This is because MAT file is a structured and compressed file format. Structured means that (in comparison to a "raw" stream of bits) the MAT file describes its own contents (e.g. whos -file), allows growth (e.g. adding other variables), access to sub-parts (e.g. load one variable out of many), names its data (e.g. so LOAD can restore a workspace) and other features supported by the MAT API. Compression means you can trade (file) space against time. All these tasks add overhead while saving to a MAT file as compared to writing to a raw binary file.
If none of these features are important, then you can certainly get much better performance by just streaming the raw bits to a binary file using FWRITE.

More Answers (0)

Products


Release

R2008b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!