MATLAB Answers

Priya

Save Structure files in les space consuming format.

Asked by Priya
on 12 Jun 2013

I have many structure files (.mat) as output, whose size is around 1.9 GB each. Is there any way , by which i can save this structure file in low size or space format.

Or is there any way by which I can reduce the size of this structure file, so that it takes less space on my pc.

  4 Comments

Priya
on 12 Jun 2013

I have cell array, having both strings and integers. I have around 800 cell arrays as fields in a structure file and each cell array has order of around 400 X 15 . Size of fields vary.

Walter Roberson
on 13 Jun 2013

A first-pass estimate of the storage requirements would be:

800 fields times 400 times 15 cells per field times Q doubles per cell times 8 bytes per double

which is (Q * 38.4) megabytes where Q is the average number of doubles per cell (you did not give us an estimate of that value)

You ask about fitting that into "kilobytes". For the purpose of discussion, I will take that as meaning you would like to have at most 500 kilobytes of storage, as after that one would round up to speak in terms of megabytes instead of kilobytes.

>> syms Q; double(solve(1000*(500*1000) = 38400000*Q))
13.02

so if there is an average of less than 13 doubles per cell, the storage requirements could potentially be "kilobytes".

If your cells average more than 13 doubles per cell, it would be necessary to use compression techniques.

There would be overhead in marking the boundaries of which part of memory goes with which cell, so you would not be able to achieve as small as Q*38.4 megabytes with varying length arrays; the amount of overhead would depend upon the maximum length of each array.

Priya
on 13 Jun 2013

It takes more than an hour just to save the files. there is one structure file which is around 3GB.

Is there any way, by which I can convert structure file into some other type like double or text, which takes less space.

However whenever I import it into matlab, it should look like same structure file as before.

Products

No products are associated with this question.

1 Answer

Answer by Jan Simon
on 12 Jun 2013
 Accepted answer

Try to save it in different Mat-formats:

data = load('YourFile.mat');
save('test1.mat', '-v6')
save('test2.mat', '-v7')
save('test3.mat', '-v7.3')

Is one of them significantly smaller?

Do you store large integer arrays in the type double? Then a conversion can reduce the memory.

  5 Comments

Jan Simon
on 13 Jun 2013

@Priya: It is impossible for us to guess how much your data can be compressed. You could save the data to a not compressed MAT-file (perhaps using '-v4', if it is still supported, because even -v6 applies a compression under certain cirsumstances) and let e.g. 7zip compress the data with the maximum level. The resulting size is an upper limit for the achievable compression ratio. Perhaps another method can squeeze some further percent out of the data, but this is not very likely.

I'm convinced that the wanted reduction from 50MB to some kB is not possible.

Priya
on 14 Jun 2013

It takes more than an hour just to save the files. there is one structure file which is around 3GB.

Is there any way, by which I can convert structure file into some other type like double or text, which takes less space.

However whenever I import it into matlab, it should look like same structure file as before.

Priya
on 20 Jun 2013

Thnks


Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply today