MATLAB Central Newsreader - Text file with numbers and text, want to just read numbers

Text file with numbers and text, want to just read numbers

Matt — Thu, 03 Jul 2008 20:18:01 -0400

I have a series of text files I want to read. The files
begin with header information. The number of rows of
header information changes, there is no commenting symbol
such as %%, and the header information sometimes has
numbers such as a date or a time. I want to read in the
number and only the numbers.

This is an example of a file: (Again, the header info
changes, different # of lines, different text)

Vector Signal [Measurement4]: 09/27/2006, 09:05:24
Units: EU
Time Real
            0 0.053732
     9.6e-005 0.11968
     0.000192 0.085483
     0.000288 0.11968
     0.000384 0.18562

Currently I have Matlab code something like this:

[filename, pathname, filterindex] = uigetfile
({'*.txt','Text file (*.txt)';'*.*','All Files
(*.*)'},'Pick Time History File');
fid = fopen(filename);
C_text = textscan(fid,'%s');
fclose(fid);

I tried to sort out the header from C_text by creating a
loop searching for numbers. Example, seeing if the
absolute value is greater than one. This method failed
when it ran into dates and times.

Is there a better way? I've been trying to use textscan,
but I have also been looking at fscan and textread. I
think there may be some formatting with C_text that could
work, but I couldn't get it to work.

Re: Text file with numbers and text, want to just read numbers

Paul — Fri, 04 Jul 2008 06:55:06 -0400

"Matt " <matthew.r.duncan@boeing.com> wrote in message
<g4jc5p$341$1@fred.mathworks.com>...
> I have a series of text files I want to read. The files
> begin with header information. The number of rows of
> header information changes, there is no commenting symbol
> such as %%, and the header information sometimes has
> numbers such as a date or a time. I want to read in the
> number and only the numbers.
>
> This is an example of a file: (Again, the header info
> changes, different # of lines, different text)
>
> Vector Signal [Measurement4]: 09/27/2006, 09:05:24
> Units: EU
> Time Real
> 0 0.053732
> 9.6e-005 0.11968
> 0.000192 0.085483
> 0.000288 0.11968
> 0.000384 0.18562
>
> Currently I have Matlab code something like this:
>
> [filename, pathname, filterindex] = uigetfile
> ({'*.txt','Text file (*.txt)';'*.*','All Files
> (*.*)'},'Pick Time History File');
> fid = fopen(filename);
> C_text = textscan(fid,'%s');
> fclose(fid);
>
>
> I tried to sort out the header from C_text by creating a
> loop searching for numbers. Example, seeing if the
> absolute value is greater than one. This method failed
> when it ran into dates and times.
>
> Is there a better way? I've been trying to use textscan,
> but I have also been looking at fscan and textread. I
> think there may be some formatting with C_text that could
> work, but I couldn't get it to work.
>
>
>
>
>

I would read the headers with fgetl and then decide if the
first character is a letter, which indicates a header line
in your example. Once past the headers, I would then read
the data with a textscan.

Re: Text file with numbers and text, want to just read numbers

Andres — Fri, 04 Jul 2008 07:24:02 -0400

Re: Text file with numbers and text, want to just read numbers

Jos — Fri, 04 Jul 2008 09:32:02 -0400

"Matt " <matthew.r.duncan@boeing.com> wrote in message
<g4jc5p$341$1@fred.mathworks.com>...
> I have a series of text files I want to read. The files
> begin with header information. The number of rows of
> header information changes, there is no commenting symbol
> such as %%, and the header information sometimes has
> numbers such as a date or a time. I want to read in the
> number and only the numbers.
>
> This is an example of a file: (Again, the header info
> changes, different # of lines, different text)
>
> Vector Signal [Measurement4]: 09/27/2006, 09:05:24
> Units: EU
> Time Real
> 0 0.053732
> 9.6e-005 0.11968
> 0.000192 0.085483
> 0.000288 0.11968
> 0.000384 0.18562
>
> Currently I have Matlab code something like this:
>
> [filename, pathname, filterindex] = uigetfile
> ({'*.txt','Text file (*.txt)';'*.*','All Files
> (*.*)'},'Pick Time History File');
> fid = fopen(filename);
> C_text = textscan(fid,'%s');
> fclose(fid);
>
>
> I tried to sort out the header from C_text by creating a
> loop searching for numbers. Example, seeing if the
> absolute value is greater than one. This method failed
> when it ran into dates and times.
>
> Is there a better way? I've been trying to use textscan,
> but I have also been looking at fscan and textread. I
> think there may be some formatting with C_text that could
> work, but I couldn't get it to work.
>
>
>
>
>

1) read all lines using
   s = textscan(filename,'%s','delimiter','\n') ;

2) determine at which lines data starts, e.g. line X
   This is the real problem!
   p = regexp(s,'[A-Za-z]') will give a cell array with non-
empty cells when a line contains text (but how to deal with
scientific notation ...). See help regexp for more options.
   X = max(find(~cellfun('isempty',p))) + 1

3) read again skipping the first (X-1) lines:
   data = textread(filename,'%f','headerlines',X-1) ;

4)

Re: Text file with numbers and text, want to just read numbers

Andres — Fri, 04 Jul 2008 10:52:01 -0400

"Jos " <DELjos@jasenDEL.nl> wrote in message
<g4kqmi$ook$1@fred.mathworks.com>...
[..]
> 2) determine at which lines data starts, e.g. line X
> This is the real problem!
> p = regexp(s,'[A-Za-z]') will give a cell array with
non-
> empty cells when a line contains text (but how to deal
with
> scientific notation ...). See help regexp for more
options.
> X = max(find(~cellfun('isempty',p))) + 1
>
[..]

That's the crux! Depending on what you know in advance
about the file, you have to care about scientific notation,
as you mentioned, final header lines containing delimiters,
signs, whitespaces or other special characters only or just
being empty, and possibly NaN and Inf occurrences in the
number section.
All in all, this can result in quite a lengthy piece of
code.

Andres