MATLAB Central Newsreader - file length

file length

Dominic — Wed, 03 Jun 2009 14:00:19 -0400

Hi,
I was wondering if anyone knew of a way to find the length (preferably the number of rows) in a .dat file without having to load it into MATLAB?
Thank you

Re: file length

Lothar Schmidt — Wed, 03 Jun 2009 15:15:49 -0400

Dominic schrieb:
> Hi,
> I was wondering if anyone knew of a way to find the length (preferably the number of rows) in a .dat file without having to load it into MATLAB?
> Thank you

number of bytes you can find as follows:

fileinfo=dir('filename.dat');
fileinfo.size

difficult to guess the number of row separators without reading the
file... perhaps meanwhile someone has written crystalball.m and put it
in the FEX.

Lothar

Re: file length

Dominic — Wed, 03 Jun 2009 15:50:17 -0400

Lothar Schmidt <vapooroop@gmx.net> wrote in message <h06437$kts$1@online.de>...
> Dominic schrieb:
> > Hi,
> > I was wondering if anyone knew of a way to find the length (preferably the number of rows) in a .dat file without having to load it into MATLAB?
> > Thank you
>
> number of bytes you can find as follows:
>
> fileinfo=dir('filename.dat');
> fileinfo.size
>
> difficult to guess the number of row separators without reading the
> file... perhaps meanwhile someone has written crystalball.m and put it
> in the FEX.
>
> Lothar

Is there any way for me to use the number of bytes in the file to deduce the number of data points or something along those lines?

Re: file length

Jan Simon — Wed, 03 Jun 2009 16:08:01 -0400

Dear Dominic!

> I was wondering if anyone knew of a way to find the length (preferably the number of rows) in a .dat file without having to load it into MATLAB?

It depends on what the format of DAT is in your case.
If you create the DAT files by your own, it would be a trivial idea to write the numbers of rows as initial UINT32.
If the DAT file contains numbers in ASCII format, you can read a single line with S=FGETL and look for the number of elements after SSCANF(S, '%f').
Nevertheless, more details about the DAT files must be known to solve the problem.

Good luck, Jan Simon

Re: file length

Dominic — Wed, 03 Jun 2009 16:37:02 -0400

"Jan Simon" <matlab.THIS_YEAR@nMINUSsimon.de> wrote in message <h06751$f6f$1@fred.mathworks.com>...
> Dear Dominic!
>
> > I was wondering if anyone knew of a way to find the length (preferably the number of rows) in a .dat file without having to load it into MATLAB?
>
> It depends on what the format of DAT is in your case.
> If you create the DAT files by your own, it would be a trivial idea to write the numbers of rows as initial UINT32.
> If the DAT file contains numbers in ASCII format, you can read a single line with S=FGETL and look for the number of elements after SSCANF(S, '%f').
> Nevertheless, more details about the DAT files must be known to solve the problem.
>
> Good luck, Jan Simon

The files are all in ASCII format I believe. I created them using dlmwrite so that is what format the should be in. I am a bit confused as to how I can get the number of elements from sscanf without loading the file into the workspace.
Thank you

Re: file length

Steven Lord — Wed, 03 Jun 2009 17:46:44 -0400

"Dominic " <dcg48@cornell.edu> wrote in message
news:h0663p$4te$1@fred.mathworks.com...
> Lothar Schmidt <vapooroop@gmx.net> wrote in message
> <h06437$kts$1@online.de>...
>> Dominic schrieb:
>> > Hi,
>> > I was wondering if anyone knew of a way to find the length (preferably
>> > the number of rows) in a .dat file without having to load it into
>> > MATLAB?
>> > Thank you
>>
>> number of bytes you can find as follows:
>>
>> fileinfo=dir('filename.dat');
>> fileinfo.size
>>
>> difficult to guess the number of row separators without reading the
>> file... perhaps meanwhile someone has written crystalball.m and put it
>> in the FEX.
>>
>> Lothar
>
> Is there any way for me to use the number of bytes in the file to deduce
> the number of data points or something along those lines?

What OS are you using? If you're on Linux or Solaris (and probably Mac as
well, though I haven't tried it) call out to the system (SYSTEM, DOS, UNIX,
or !) and use the wc system function. This will give you, among other
information, the newline count for the file. See "man wc" for more
information.

--
Steve Lord
slord@mathworks.com

Re: file length

Dominic — Wed, 03 Jun 2009 17:52:01 -0400

"Steven Lord" <slord@mathworks.com> wrote in message <h06ctf$er4$1@fred.mathworks.com>...
>
> "Dominic " <dcg48@cornell.edu> wrote in message
> news:h0663p$4te$1@fred.mathworks.com...
> > Lothar Schmidt <vapooroop@gmx.net> wrote in message
> > <h06437$kts$1@online.de>...
> >> Dominic schrieb:
> >> > Hi,
> >> > I was wondering if anyone knew of a way to find the length (preferably
> >> > the number of rows) in a .dat file without having to load it into
> >> > MATLAB?
> >> > Thank you
> >>
> >> number of bytes you can find as follows:
> >>
> >> fileinfo=dir('filename.dat');
> >> fileinfo.size
> >>
> >> difficult to guess the number of row separators without reading the
> >> file... perhaps meanwhile someone has written crystalball.m and put it
> >> in the FEX.
> >>
> >> Lothar
> >
> > Is there any way for me to use the number of bytes in the file to deduce
> > the number of data points or something along those lines?
>
> What OS are you using? If you're on Linux or Solaris (and probably Mac as
> well, though I haven't tried it) call out to the system (SYSTEM, DOS, UNIX,
> or !) and use the wc system function. This will give you, among other
> information, the newline count for the file. See "man wc" for more
> information.
>
> --
> Steve Lord
> slord@mathworks.com
>
No I'm using Windows

Re: file length

dpb — Wed, 03 Jun 2009 17:56:39 -0400

Dominic wrote:
> "Steven Lord" <slord@mathworks.com> wrote in message <h06ctf$er4$1@fred.mathworks.com>...
...
>> ... use the wc system function. This will give you, among other
>> information, the newline count for the file. See "man wc" for more
>> information.
...
> No I'm using Windows

There are any number of Windows ports--here's one...

"WordCount is a small freeware Windows 95/98/NT command-line program
that counts characters, words and lines in one or more files -- my
attempt to bring the features of the Unix "WC" utility to Windows...."

<http://www.tawbaware.com/wc.htm>

(google is your friend)

--

Re: file length

Dominic — Wed, 03 Jun 2009 18:33:01 -0400

dpb <none@non.net> wrote in message <h06dme$7ii$1@aioe.org>...
> Dominic wrote:
> > "Steven Lord" <slord@mathworks.com> wrote in message <h06ctf$er4$1@fred.mathworks.com>...
> ...
> >> ... use the wc system function. This will give you, among other
> >> information, the newline count for the file. See "man wc" for more
> >> information.
> ...
> > No I'm using Windows
>
> There are any number of Windows ports--here's one...
>
> "WordCount is a small freeware Windows 95/98/NT command-line program
> that counts characters, words and lines in one or more files -- my
> attempt to bring the features of the Unix "WC" utility to Windows...."
>
> <http://www.tawbaware.com/wc.htm>
>
> (google is your friend)
>
> --

I am not familiar with that kind of program. Is it possible to call it in MATLAB? The way my program runs now I am working through an entire folder of .dat files using the dir command and a loop so that I run each file.
Thank you,

Re: file length

dpb — Wed, 03 Jun 2009 22:15:15 -0400

Dominic wrote:
> dpb <none@non.net> wrote in message <h06dme$7ii$1@aioe.org>...
>> Dominic wrote:
>>> "Steven Lord" <slord@mathworks.com> wrote in message
>>> <h06ctf$er4$1@fred.mathworks.com>...
>> ...
>>>> ... use the wc system function. This will give you, among
>>>> other information, the newline count for the file. See "man
>>>> wc" for more information.
>> ...
>>> No I'm using Windows
>> There are any number of Windows ports--here's one...
>>
>> "WordCount is a small freeware Windows 95/98/NT command-line
>> program that counts characters, words and lines in one or more
>> files -- my attempt to bring the features of the Unix "WC" utility
>> to Windows...."
>>
>> <http://www.tawbaware.com/wc.htm>
>>
>> (google is your friend)
>>
>> --
>
> I am not familiar with that kind of program. Is it possible to call
> it in MATLAB? The way my program runs now I am working through an
> entire folder of .dat files using the dir command and a loop so that
> I run each file. Thank you,

Just pass the command to the OS as Steve suggested...or, likely you
could simply run the utility directly on the files w/ a wild card and
pipe its output to a file for later use.

--

Re: file length

Jan Simon — Wed, 03 Jun 2009 22:54:01 -0400

Dear Dominic!

> The files are all in ASCII format I believe. I created them using dlmwrite so that is what format the should be in. I am a bit confused as to how I can get the number of elements from sscanf without loading the file into the workspace.

My suggested FGETL method reads just the first line of the file:
  FID = fopen(FileName, 'rb');
  Line1 = fgetl(FID);
  fclose(FID);
  Data = sscanf(Line1);
  NumberOfColums = length(Data);

Nevertheless, if I read your original question, I find, that you want the number of **rows**, not columns...

Beside the smart WC command, you could try this function in pure Matlab:

function nRow = CountLines(FileName)
FID = fopen(FileName, 'rb'); % ADD CHECK FOR: FID==-1
nRow = 0;
while ischar(fgets(FID))
  nRow = nRow + 1;
end
fclose(FID);
return;

This is faster than loading the file into the workspace, because the ASCII lines are not parsed from text to doubles.

Good luck, Jan

Re: file length

Jos — Thu, 04 Jun 2009 10:51:01 -0400

"Dominic " <dcg48@cornell.edu> wrote in message <h05vlj$csf$1@fred.mathworks.com>...
> Hi,
> I was wondering if anyone knew of a way to find the length (preferably the number of rows) in a .dat file without having to load it into MATLAB?
> Thank you

For ascii files, this might do the job

Nrows = numel(textread('mydata.txt','%1c%*[^\n]'))

Note that it does read in some data (temporarily!): one character per non-empty row.

hth
Jos

Re: file length

Kenneth Eaton — Thu, 04 Jun 2009 13:02:02 -0400

"Jos " <#10584@fileexchange.com> wrote in message <h088ul$e0s$1@fred.mathworks.com>...
> For ascii files, this might do the job
>
> Nrows = numel(textread('mydata.txt','%1c%*[^\n]'))
>
> Note that it does read in some data (temporarily!): one character per non-empty row.

One caveat: This solution won't count empty rows that consist of only a newline character. However, if you know there will always be something in every row, this should work great.

Note: Newer versions of MATLAB urge the use of TEXTSCAN over TEXTREAD. I'm guessing eventually TEXTREAD will be phased out.

Re: file length

Dominic — Thu, 04 Jun 2009 13:50:18 -0400

"Kenneth Eaton" <Kenneth.dot.Eaton@cchmc.dot.org> wrote in message <h08gka$aim$1@fred.mathworks.com>...
> "Jos " <#10584@fileexchange.com> wrote in message <h088ul$e0s$1@fred.mathworks.com>...
> > For ascii files, this might do the job
> >
> > Nrows = numel(textread('mydata.txt','%1c%*[^\n]'))
> >
> > Note that it does read in some data (temporarily!): one character per non-empty row.
>
> One caveat: This solution won't count empty rows that consist of only a newline character. However, if you know there will always be something in every row, this should work great.
>
> Note: Newer versions of MATLAB urge the use of TEXTSCAN over TEXTREAD. I'm guessing eventually TEXTREAD will be phased out.

The function worked great. Thanks!