MATLAB Central Newsreader - Writing Binary file to reduce file size

Writing Binary file to reduce file size

Pete sherer — Mon, 21 Jul 2008 16:42:03 -0400

Hi,
Are there any example how to use Matlab to make the binary
file of data. The goal is to reduce the file size as well as
the accessing time to the file. Most of the data contain
zero, so I hope binary file can be much smaller than the
ascii file.

Thanks a lot,

Re: Writing Binary file to reduce file size

Peter Boettcher — Mon, 21 Jul 2008 16:56:25 -0400

"Pete sherer" <tsh@abg.com> writes:

> Hi,
> Are there any example how to use Matlab to make the binary
> file of data. The goal is to reduce the file size as well as
> the accessing time to the file. Most of the data contain
> zero, so I hope binary file can be much smaller than the
> ascii file.

Yes indeed.

If you intend to load the data back into MATLAB, and only MATLAB, just
use "save" and "load". The resulting file is "binary", though not in a
simple format.

If you wish to access the data from another programming language, see
the documentation for fopen, fwrite, fread, and fclose.

-Peter

Re: Writing Binary file to reduce file size

Bill McKeeman — Thu, 24 Jul 2008 04:47:01 -0400

Peter Boettcher <boettcher@ll.mit.edu> wrote in message
<muy4p6j89c6.fsf@G99-Boettcher.llan.ll.mit.edu>...
> "Pete sherer" <tsh@abg.com> writes:
>
> > Hi,
> > Are there any example how to use Matlab to make the
binary
> > file of data. The goal is to reduce the file size as
well as
> > the accessing time to the file. Most of the data contain
> > zero, so I hope binary file can be much smaller than the
> > ascii file.
>
> Yes indeed.
>
> If you intend to load the data back into MATLAB, and only
MATLAB, just
> use "save" and "load". The resulting file is "binary",
though not in a
> simple format.
>
> If you wish to access the data from another programming
language, see
> the documentation for fopen, fwrite, fread, and fclose.
>
> -Peter

I suggest you try

>> help zip

/s/ Bill

Re: Writing Binary file to reduce file size

Pete sherer — Thu, 24 Jul 2008 05:09:01 -0400

I think what I want to do is to write a binary file such
that I can use other programs besides matlab to read the file.
I just need an example how to write and read it, at least
from Matlab workspace. Finally I will use other
applications to read the data file.
I think this is a common example, but I can't seem to find
the solution.
Thanks for all the advice in advance.

Re: Writing Binary file to reduce file size

NZTideMan — Thu, 24 Jul 2008 06:33:31 -0400

On Jul 24, 5:09=A0pm, "Pete sherer" <t...@abg.com> wrote:
> I think what I want to do is to write a binary file such
> that I can use other programs besides matlab to read the file. =A0
> I just need an example how to write and read it, at least
> from Matlab workspace. =A0Finally I will use other
> applications to read the data file. =A0
> I think this is a common example, but I can't seem to find
> the solution.
> Thanks for all the advice in advance.

netCDF is the answer.
Google it.

Re: Writing Binary file to reduce file size

Rune Allnor — Thu, 24 Jul 2008 07:17:01 -0400

On 24 Jul, 07:09, "Pete sherer" <t...@abg.com> wrote:
> I think what I want to do is to write a binary file such
> that I can use other programs besides matlab to read the file. =A0
> I just need an example how to write and read it, at least
> from Matlab workspace. =A0Finally I will use other
> applications to read the data file. =A0
> I think this is a common example, but I can't seem to find
> the solution.

The trick is to use FREAD and FWRITE. If you want to read data
from a file known to contain one 100 x 200 array of doubles,
you do something like

fid =3D fopen('myfile.dat');
A=3Dfread(fid,[100,200],'double');
fclose(fid);

and similar when you write bac to file.

Just be aware of somewhat awkward issues with file formats
(rows-first or column-first storage) and binary numerical
formats (big-endian or little-endian formats).

Rune

Re: Writing Binary file to reduce file size

NZTideMan — Thu, 24 Jul 2008 09:02:14 -0400

On Jul 24, 7:17=A0pm, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 24 Jul, 07:09, "Pete sherer" <t...@abg.com> wrote:
>
> > I think what I want to do is to write a binary file such
> > that I can use other programs besides matlab to read the file. =A0
> > I just need an example how to write and read it, at least
> > from Matlab workspace. =A0Finally I will use other
> > applications to read the data file. =A0
> > I think this is a common example, but I can't seem to find
> > the solution.
>
> The trick is to use FREAD and FWRITE. If you want to read data
> from a file known to contain one 100 x 200 array of doubles,
> you do something like
>
> fid =3D fopen('myfile.dat');
> A=3Dfread(fid,[100,200],'double');
> fclose(fid);
>
> and similar when you write bac to file.
>
> Just be aware of somewhat awkward issues with file formats
> (rows-first or column-first storage) and binary numerical
> formats (big-endian or little-endian formats).
>
> Rune

IMHO, using fread and fwrite when you want to maybe read the data in
using a different program is not advisable. I've been there and done
that and everytime, reading the binary files on a different platform
or in a different program was a problem.
That's why I suggested netCDF. Files written in netCDF on a Cray for
example can be read on a PC or Mac in Matlab, Fortran, C, or
whatever. It is totally portable.

Re: Writing Binary file to reduce file size

David — Thu, 24 Jul 2008 09:44:03 -0400

"Pete sherer" <tsh@abg.com> wrote in message <g62e8r$i37
$1@fred.mathworks.com>...
> Hi,
> Are there any example how to use Matlab to make the
binary
> file of data. The goal is to reduce the file size as
well as
> the accessing time to the file. Most of the data contain
> zero, so I hope binary file can be much smaller than the
> ascii file.
>
> Thanks a lot,
>

and of course 'binary' isn't always smaller. if you write
your zero data as doubles with fwrite it takes 64 bits.
if you format it properly in a tab or comma delimited file
it can be only 16 bits. create your own binary format and
you could reduce it even more using simple compression
techniques, but then you would have to write the reader to
uncompress it.