File Exchange

image thumbnail

Save MAT files more quickly

version (1.84 KB) by Tim Holy

Tim Holy (view profile)

Saves large numeric arrays more quickly than Matlab's built-in save command


Updated 04 Jan 2013

View License

When working with large arrays, Matlab's 'save' command can be very slow, presumably because Matlab attempts to use compression. This function provides a much faster alternative, at the cost of larger files. It only improves speed for variables that are plain arrays; elements of structures or cell arrays are not altered.

% First make some data
>> ops = struct('algorithm', 'greedy');
>> A = int32(randi(20, 1000, 1200, 40));
>> B = randn(500, 1800, 60);
% Now save it
>> tic; save /tmp/test ops A B; toc
Elapsed time is 22.980294 seconds.
>> tic; savefast /tmp/test ops A B; toc
Elapsed time is 0.571098 seconds.

Happy saving!

Comments and Ratings (24)

Surely the fact that it cannot handle complex data should be mentioned in the documentation, preferably in bold text and on the first line?

Having happily used it on only real valued data for the last year, I've just wasted more time than I am willing to describe here conducting analysis on what I thought was the absolute value of a complex number, only to discover it's the real component only.

This is otherwise a fantastic script, and a great contribution to the community, but I have spent the last few days thinking I was going insane, before I was finally able to track down the source of error. Thankfully before publication...


adi (view profile)

Hi! does it work with version v7.3? I need to save and load parts of a variable from a mat file, and the files are larger than 2 Gib.


Daniel (view profile)

How can we append files to data already saved?

He Qihong

It doesn't work correctly in Matlab2014b. It will makes Matlab2014b crash.


Coming back to the complex-number issue discussed by Andre and Raj. A small addition e.g.

saves all complex numbers using the default (slow) command instead of hdf5 without compression. This would have saved me some trouble. Maybe add a comment to the file description as well.

All in all the plugin is very nice.

Tim Holy

Tim Holy (view profile)

@arnold, does it work if you list variables explicitly? It's possible it's just missing a variant that saves all the variables in your workspace.


arnold (view profile)

I don't seem to understand the syntax, it doesn't behave like 'save'.


Does produce a file with the appropriate name but it doesn't seem to contain anything. Loading it does not restore any of the workspace variables.

Tim Holy

Tim Holy (view profile)

@zhang, the problem is that '-v6' has some serious limitations, for example it can't store arrays bigger than 2GB. It's also not as portable to other languages, etc; '-v7.3' is based on HDF5, so many different languages can read the files.


zhang (view profile)

This is unnecessary. Use the build-in save(filename, variables,'-v6') is even much faster than this file.

e.g., test showed that:
if not use '-v6', it takes 7.7 seconds for a file.
Using this file, it's 0.73 seconds.
But using the '-v6' option, it's ONLY 0.17 seconds!

Tim Holy

Tim Holy (view profile)

@yaron, off the top of my head I have no idea. What is the error message, and what are version(s) of Matlab on each?


yaron (view profile)

Great contribution - but I have an issue with using it on a mac. For some reason, while the files load on a PC, the save fast .mat files cause an error on my mac. Any idea for how to fix this?

Tim Holy

Tim Holy (view profile)

Ali, it can only handle certain types, I didn't try to reverse engineer the entire format.

But see the comment from Raj, immediately below yours, about hlp_serialize.


Ali (view profile)

Thank you for providing this code. I tried to use it for fast saving of complex-sparse matrices and it ends with the following error:

Error using hdf5lib2
The HDF5 library encountered an unknown error.
Error in H5D.write (line 72)

do you think it there is a solution to this error or it is not applicable to large complex sparse data format?


Raj (view profile)

Andre..may be your problem might be resolved if you first serialize the data using hlp_serialize( and store using savefast function and later deserialize the data. This worked even when I had complex vector is a structure...definitely improved saving files with marginal increase in memory!


Raj (view profile)

Andre Kuehne

Andre Kuehne

Great submission, however it will not work for complex (real/imaginary) datatypes! The Matlab highlevel hdf5 routines apparently do not support complex datatypes and thus only the real part is saved. This is probably not easily fixed, as you would need to utilize low-level hdf5 functions.

Tim Holy

Tim Holy (view profile)

Robin, that's good to know about parfor. Thanks for providing a version that works under such circumstances.


Robin (view profile)

This is great. One problem was that 'evalin' doesn't work properly within a parfor loop. So I made a minor modification to allow 'save -struct' style saving which works fine in parfor:

Tim Holy

Tim Holy (view profile)

Hi Jeremy,

Your performance benefit will depend a lot on your hard drive configuration. In my case I was writing to an SSD RAID. But on my laptop (with a single SSD) I can write in about 2.1 sec, which is still a 10x benefit. What kind of performance benefit are you getting?


Jeremy (view profile)

I am not seeing anywhere close to that save time using your A,B,ops sample. I'm running 2012b.

Your example is showing that you saved ~600MB in .57 sec. That seems a bit suspect as it is over 1GB/sec.


Robin (view profile)

MATLAB Release Compatibility
Created with R2012b
Compatible with any release
Platform Compatibility
Windows macOS Linux

Discover Live Editor

Create scripts with code, output, and formatted text in a single executable document.

Learn About Live Editor