Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

To resolve issues starting MATLAB on Mac OS X 10.10 (Yosemite) visit: http://www.mathworks.com/matlabcentral/answers/159016

Save Structure files in les space consuming format.

Asked by Priya on 12 Jun 2013

I have many structure files (.mat) as output, whose size is around 1.9 GB each. Is there any way , by which i can save this structure file in low size or space format.

Or is there any way by which I can reduce the size of this structure file, so that it takes less space on my pc.

4 Comments

Priya on 12 Jun 2013

I have cell array, having both strings and integers. I have around 800 cell arrays as fields in a structure file and each cell array has order of around 400 X 15 . Size of fields vary.

Walter Roberson on 13 Jun 2013

A first-pass estimate of the storage requirements would be:

800 fields times 400 times 15 cells per field times Q doubles per cell times 8 bytes per double

which is (Q * 38.4) megabytes where Q is the average number of doubles per cell (you did not give us an estimate of that value)

You ask about fitting that into "kilobytes". For the purpose of discussion, I will take that as meaning you would like to have at most 500 kilobytes of storage, as after that one would round up to speak in terms of megabytes instead of kilobytes.

>> syms Q; double(solve(1000*(500*1000) = 38400000*Q))
13.02

so if there is an average of less than 13 doubles per cell, the storage requirements could potentially be "kilobytes".

If your cells average more than 13 doubles per cell, it would be necessary to use compression techniques.

There would be overhead in marking the boundaries of which part of memory goes with which cell, so you would not be able to achieve as small as Q*38.4 megabytes with varying length arrays; the amount of overhead would depend upon the maximum length of each array.

Priya on 13 Jun 2013

It takes more than an hour just to save the files. there is one structure file which is around 3GB.

Is there any way, by which I can convert structure file into some other type like double or text, which takes less space.

However whenever I import it into matlab, it should look like same structure file as before.

Priya

Products

No products are associated with this question.

1 Answer

Answer by Jan Simon on 12 Jun 2013
Accepted answer

Try to save it in different Mat-formats:

data = load('YourFile.mat');
save('test1.mat', '-v6')
save('test2.mat', '-v7')
save('test3.mat', '-v7.3')

Is one of them significantly smaller?

Do you store large integer arrays in the type double? Then a conversion can reduce the memory.

5 Comments

Jan Simon on 13 Jun 2013

@Priya: It is impossible for us to guess how much your data can be compressed. You could save the data to a not compressed MAT-file (perhaps using '-v4', if it is still supported, because even -v6 applies a compression under certain cirsumstances) and let e.g. 7zip compress the data with the maximum level. The resulting size is an upper limit for the achievable compression ratio. Perhaps another method can squeeze some further percent out of the data, but this is not very likely.

I'm convinced that the wanted reduction from 50MB to some kB is not possible.

Priya on 14 Jun 2013

It takes more than an hour just to save the files. there is one structure file which is around 3GB.

Is there any way, by which I can convert structure file into some other type like double or text, which takes less space.

However whenever I import it into matlab, it should look like same structure file as before.

Priya on 20 Jun 2013

Thnks

Jan Simon

Contact us