save('file.mat', '-v7.3') is extremely slow (still)

Hello Matlab community,

I'm looking for a solution to extremely slow save() and load() times for large mat files (3-5 gigs) using the '-v7.3' flag.

I've seen threads from 2011 and 2015 that address this issue and am looking for any updated info on this topic.

I'm saving a cell array with 53 elements. Each element is a structure that contains 1160 fields nested in 5-6 layers.

Using vs.2014a, save(myarray, '-v7.3') required 19.3 minutes while load() required 6.34 minutes. (Windows 7 Pro, 64bit OS, 8g RAM).

Using vs2016a, save(myarray, '-v7.3') required 10.5 minutes, while load() required 4.2 minutes.

These processing times are unbearable and the arrays and structures will likely grow in size in the near future. Merely copying the mat file on windows just took a few seconds.

I'm hoping matlab has a solution to this issue. Any advice?

Thanks, Adam

Answer 1

[Walter Roberson]

In the File Exchange, there are a few serialize / deserialize routines, including https://www.mathworks.com/matlabcentral/fileexchange/34564-fast-serialize-deserialize and https://www.mathworks.com/matlabcentral/fileexchange/29457-serialize-deserialize

Potentially you might be able to get reasonable performance by serializing the struct and fwrite() the result to a file.

By adjusting some of the code there, perhaps you could have pieces written out to the file as they are produced, to reduce the memory usage.