MATLAB Central Newsreader - read multi-line data with multiple delimiters

read multi-line data with multiple delimiters

boivie@gmail.com — Fri, 24 Oct 2008 15:04:27 -0400

Hi all - I've scoured the net and mathworks and haven't come across
how to solve this. I haven't used Matlab in quite a while, and it
seems like this shouldn't be very hard, but I haven't managed to get
this yet.

The file I am reading in has data (after several file header lines)
that consists of one line of header info ("Line" or "Tie" and the
number - both of which I need to extract) and two lines of comma
delimited data associated with that header. I seem to be able to
extract the first line of data using textscan with comma as the
delimiter. Using regexp, I have gotten both lines of data, but as 4
variables, rather than 8. Any pointers or code help would be much
appreciated. Thanks.

-Ethan

This is what the file looks like:

/Data
/X,Y,Longitude,Latitude
//Flight 0
//Date 2008/09/27
Line 0
-450.0,50.0,147.5445363050,-76.4478227596
450.0,50.0,182.4554636951,-76.4478227596
Line 10
-450.0,-50.0,146.3500677843,-77.3018107353
450.0,-50.0,183.6499322158,-77.3018107353
Tie -10
-360.0,-100.0,149.3523576944,-77.9633702276
-360.0,100.0,151.3351657935,-76.2267188392
Tie 0
-160.0,-100.0,157.9105218900,-78.3092770253
-160.0,100.0,158.8372826671,-76.5286269834

Re: read multi-line data with multiple delimiters

Rune Allnor — Fri, 24 Oct 2008 15:23:07 -0400

On 24 Okt, 17:04, boi...@gmail.com wrote:
> Hi all - I've scoured the net and mathworks and haven't come across
> how to solve this. =A0I haven't used Matlab in quite a while, and it
> seems like this shouldn't be very hard, but I haven't managed to get
> this yet.
>
> The file I am reading in has data (after several file header lines)
> that consists of one line of header info ("Line" or "Tie" and the
> number - both of which I need to extract) and two lines of comma
> delimited data associated with that header. =A0I seem to be able to
> extract the first line of data using textscan with comma as the
> delimiter. =A0Using regexp, I have gotten both lines of data, but as 4
> variables, rather than 8. =A0Any pointers or code help would be much
> appreciated. =A0Thanks.

It seems you are almost there. REGEXP is the method
I would use to detect the header lines. However,
you might want to use SSCANF to read the two data lines.
Just remember that you read each data line one at the
time and need to store the corresponding data somewhere.

The basic program would go something like (pseudocode!)

while (~feof(fid))
   while (~regexpmatch('tie'|'line'))
      s =3D fgetl(fid)
   end
   extract number from present line that contains
   'tie' or 'line'
   s =3D fgetl(fid)
   A =3D sscanf(s,'%f,%f,%f,%f');
   s =3D fgetl(fid)
   B =3D sscanf(s,'%f,%f,%f,%f');
   store A and B wherever they need to go
end

Rune

Re: read multi-line data with multiple delimiters

boivie@gmail.com — Fri, 24 Oct 2008 19:43:28 -0400

On Oct 24, 11:23=A0am, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 24 Okt, 17:04, boi...@gmail.com wrote:
>
> > Hi all - I've scoured the net and mathworks and haven't come across
> > how to solve this. =A0I haven't used Matlab in quite a while, and it
> > seems like this shouldn't be very hard, but I haven't managed to get
> > this yet.
>
> > The file I am reading in hasdata(after several file header lines)
> > that consists of one line of header info ("Line" or "Tie" and the
> > number - both of which I need to extract) and two lines of comma
> > delimiteddataassociated with that header. =A0I seem to be able to
> > extract the first line ofdatausing textscan with comma as the
> > delimiter. =A0Using regexp, I have gotten both lines ofdata, but as 4
> > variables, rather than 8. =A0Any pointers or code help would be much
> > appreciated. =A0Thanks.
>
> It seems you are almost there. REGEXP is the method
> I would use to detect the header lines. However,
> you might want to use SSCANF to read the twodatalines.
> Just remember that you read eachdataline one at the
> time and need to store the correspondingdatasomewhere.
>
> The basic program would go something like (pseudocode!)
>
> while (~feof(fid))
> =A0 =A0while (~regexpmatch('tie'|'line'))
> =A0 =A0 =A0 s =3D fgetl(fid)
> =A0 =A0end
> =A0 =A0extract number from present line that contains
> =A0 =A0'tie' or 'line'
> =A0 =A0s =3D fgetl(fid)
> =A0 =A0A =3D sscanf(s,'%f,%f,%f,%f');
> =A0 =A0s =3D fgetl(fid)
> =A0 =A0B =3D sscanf(s,'%f,%f,%f,%f');
> =A0 =A0store A and B wherever they need to go
> end
>
> Rune

Rune,

Thanks very much for the help. I'm getting an error using the OR
operator '|' in this line: while (~regexpmatch('Tie'|'Line')).
Matlab returns:
'Error using =3D=3D> or
Inputs must have the same size.'

I tried a few things, including using '||' instead of '|', and
breaking it up into two parts: 'while (~regexpmatch('Tie') &&
~regexpmatch('Line'))'

Neither has worked. Any suggestions? Thanks.

Ethan

read multi-line data with multiple delimiters

Andres — Fri, 24 Oct 2008 19:48:02 -0400

boivie@gmail.com wrote in message <70d4e42d-48e0-4f97-a9d2-83aa4eeed8c2@l76g2000hse.googlegroups.com>...
[..]
> This is what the file looks like:
>
> /Data
> /X,Y,Longitude,Latitude
> //Flight 0
> //Date 2008/09/27
> Line 0
> -450.0,50.0,147.5445363050,-76.4478227596
> 450.0,50.0,182.4554636951,-76.4478227596
> Line 10
> -450.0,-50.0,146.3500677843,-77.3018107353
> 450.0,-50.0,183.6499322158,-77.3018107353
> Tie -10
> -360.0,-100.0,149.3523576944,-77.9633702276
> -360.0,100.0,151.3351657935,-76.2267188392
> Tie 0
> -160.0,-100.0,157.9105218900,-78.3092770253
> -160.0,100.0,158.8372826671,-76.5286269834

Hi Ethan,
if it is ok that your 'Line' or 'Tie' flags are represented as 0 or 1 in a double array, there is also a (more convenient?) single command option with txt2mat from the file exchange:

A = txt2mat('c:file.txt','ReplaceExpr',{{'Line','0'},{'Tie','1'}});

The header lines and delimiters are detected automatically (see the doc), and the matrix is padded with NaNs to fill up the rows.

With your example, A would be

0 0 NaN NaN
-450 50 147.5445 -76.4478
450 50 182.4555 -76.4478
0 10 NaN NaN
-450 -50 146.3501 -77.3018
450 -50 183.6499 -77.3018
1 -10 NaN NaN
-360 -100 149.3524 -77.9634
-360 100 151.3352 -76.2267
1 0 NaN NaN
-160 -100 157.9105 -78.3093
-160 100 158.8373 -76.5286

(short notation just for the ng)

Hth,
regards,
Andres

Re: read multi-line data with multiple delimiters

Walter Roberson — Fri, 24 Oct 2008 20:26:14 -0400

boivie@gmail.com wrote:
> while (~regexpmatch('Tie'|'Line')).

isempty(regexp(s, '^Tie|^Line', 'once'))

and you will have to adjust Rune's pseudo-code to read a line
into s at the beginning.