Thread Subject: reading mixed fortran data

I would like to be able to read some output files in an automated manner that have string headers and fortran formatted data. As of now, I copy the data into excel and create a text file to dlmread, or delete the headers and dlmread.

Ideally, this would be a lloop to evaluate that if a line contains a string it would skip the line and then dlmread any line with numerical input. I have tried combinations of fread, importdata, dlmread amongst other things. If a loop is not possible, is there a way to just skip the headings and then dlmread a specified number of lines or dlmread a range of lines (using the range method fails because of the header text).

An example of my data would be:

                     XMAX= 0.4161
  ACCELERATION VALUES 1 - Soil Profile No. 1
 0.003801 0.003580 0.005192 0.008181 0.011606 1
 0.013532 0.011207 0.009334 0.008696 0.009565 2
 0.016459 0.015466 0.013259 0.010606 0.008381 3
 0.007241 0.005412 0.001210-0.005376-0.013538 4
-0.032571-0.029456-0.024083-0.017933-0.012364 5
                     XMAX= 0.4161
  ACCELERATION VALUES 1 - Soil Profile No. 1
 0.003801 0.003580 0.005192 0.008181 0.011606 1
 0.013532 0.011207 0.009334 0.008696 0.009565 2
 0.016459 0.015466 0.013259 0.010606 0.008381 3
 0.007241 0.005412 0.001210-0.005376-0.013538 4
-0.032571-0.029456-0.024083-0.017933-0.012364 5


Thanks!!!!

Right after posting this I saw this post:
http://www.mathworks.com/matlabcentral/newsreader/view_thread/254344

If I open and textscan past the headers, then I am left with a double that I am unsure how to convert into a usable body of data (nx1 matrix)

Acc=fopen('test_rr.a10')
Accel=textscan(Acc, '%12f', 'Headerlines', 3)
fclose(Acc)

yields: Accel = [8192x1 double]

Also, will the text scan know to stop at the next set of headers or do I need more code?

"Tyler Barker" <tylerb@dnfsb.gov> wrote in message <h1r9r6$m4r$1@fred.mathworks.com>...
> Right after posting this I saw this post:
> http://www.mathworks.com/matlabcentral/newsreader/view_thread/254344
>
> If I open and textscan past the headers, then I am left with a double that I am unsure how to convert into a usable body of data (nx1 matrix)
>
> Acc=fopen('test_rr.a10')
> Accel=textscan(Acc, '%12f', 'Headerlines', 3)
> fclose(Acc)
>
> yields: Accel = [8192x1 double]
>
> Also, will the text scan know to stop at the next set of headers or do I need more code?

doc fgetl

% and look at the Example to get you started.

fgetl returns strings, ie it looks at a line of numbers as 80 char. not as numbers
As a result it does not let me compare numbers to characters and sort appropriately.

Is there a way to effectively sort through this data and evaluate text versus numbers?

Thanks!

Tyler Barker wrote:
> fgetl returns strings, ie it looks at a line of numbers as 80 char.
> not as numbers As a result it does not let me compare numbers to
> characters and sort appropriately.
>
> Is there a way to effectively sort through this data and evaluate
> text versus numbers?

 From the earlier posting of a segment of the data file, depends on what
you want.

If you only want a segment, textread() w/ the headerlines and 'N' repeat
count could work.

If you want to read multiple sections, the simpler way is to use the
file structure and write for it. In this case it appears there are two
header lines followed by 5 rows of data values and that is repeated.

Depending on whether you want/need to parse the values out of the
headers, something otoo

fid = fopen('file.dat', 'rt');
l1 = fgetl(fid); % 1st header line
l2 = fgetl(fid); % 2nd
x = fscanf(fid,'%f', [6,5])';

will get each segment into an array. You could then wrap this code
snippet in a function w/ some control loops over the number of data
sets, etc., to make a more convenient interface.

See sscanf() and strtok() and friends for ideas on extracting the values
from the header lines if need them.

help iofun
help strfun

for ideas...

--

On 23 Jun, 20:54, "Tyler Barker" <tyl...@dnfsb.gov> wrote:
> I would like to be able to read some output files in an automated manner that have string headers and fortran formatted data.  As of now, I copy the data into excel and create a text file to dlmread, or delete the headers and dlmread.
>
> Ideally, this would be a lloop to evaluate that if a line contains a string it would skip the line and then dlmread any line with numerical input.  I have tried combinations of fread, importdata, dlmread amongst other things.  If a loop is not possible, is there a way to just skip the headings and then dlmread a specified number of lines or dlmread a range of lines (using the range method fails because of the header text).
>
> An example of my data would be:
>
>                      XMAX=  0.4161                              
>   ACCELERATION VALUES  1 - Soil Profile No.  1                
>  0.003801 0.003580 0.005192 0.008181 0.011606      1
>  0.013532 0.011207 0.009334 0.008696 0.009565      2
>  0.016459 0.015466 0.013259 0.010606 0.008381      3
>  0.007241 0.005412 0.001210-0.005376-0.013538      4
> -0.032571-0.029456-0.024083-0.017933-0.012364      5
>                      XMAX=  0.4161                              
>   ACCELERATION VALUES   1 - Soil Profile No.  1                
>  0.003801 0.003580 0.005192 0.008181 0.011606      1
>  0.013532 0.011207 0.009334 0.008696 0.009565      2
>  0.016459 0.015466 0.013259 0.010606 0.008381      3
>  0.007241 0.005412 0.001210-0.005376-0.013538      4
> -0.032571-0.029456-0.024083-0.017933-0.012364      5

1) read each line with FGETL
2) Scan it with a regular expression to see if it contains letters
3) Store the data from lines that don't contain letters using SSCANF

Rune

"Tyler Barker" <tylerb@dnfsb.gov> wrote in message <h1r8c9$fll$1@fred.mathworks.com>...
> I would like to be able to read some output files in an automated manner that have string headers and fortran formatted data. As of now, I copy the data into excel and create a text file to dlmread, or delete the headers and dlmread.
>
> Ideally, this would be a lloop to evaluate that if a line contains a string it would skip the line and then dlmread any line with numerical input. I have tried combinations of fread, importdata, dlmread amongst other things. If a loop is not possible, is there a way to just skip the headings and then dlmread a specified number of lines or dlmread a range of lines (using the range method fails because of the header text).
>
> An example of my data would be:
>
> XMAX= 0.4161
> ACCELERATION VALUES 1 - Soil Profile No. 1
> 0.003801 0.003580 0.005192 0.008181 0.011606 1
> 0.013532 0.011207 0.009334 0.008696 0.009565 2
> 0.016459 0.015466 0.013259 0.010606 0.008381 3
> 0.007241 0.005412 0.001210-0.005376-0.013538 4
> -0.032571-0.029456-0.024083-0.017933-0.012364 5
> XMAX= 0.4161
> ACCELERATION VALUES 1 - Soil Profile No. 1
> 0.003801 0.003580 0.005192 0.008181 0.011606 1
> 0.013532 0.011207 0.009334 0.008696 0.009565 2
> 0.016459 0.015466 0.013259 0.010606 0.008381 3
> 0.007241 0.005412 0.001210-0.005376-0.013538 4
> -0.032571-0.029456-0.024083-0.017933-0.012364 5
>
>
> Thanks!!!!

Hello, It's been a while since you post it but if you're still trying to read shake output, create the input files or running it I have files that do all of that, just email me.


Regards,
Gonzalo Montalva

Hi

I realise this post is a little old.. but I have a similar problem. I have a data file combined from a few files which each have headers, such as

NVAR: 8
VARNAMES: CID Z ZERR DLMAG DLMAGERR AV AVERR
 
SN: 9567881 0.2883 0.0025 45.775 0.081 0.010 0.024
SN: 92387878 0.2835 0.0025 99.42 0.036 0.002 0.200

 NVAR: 8
VARNAMES: CID Z ZERR DLMAG DLMAGERR AV AVERR
 
SN: 9567881 0.2883 0.0025 45.775 0.081 0.010 0.024
SN: 92387878 0.4535 0.0025 99.42 0.036 0.002 0.200
SN: 9289278 0.255 0.015 75.42 0.056 0.002 0.200
SN: 921283 0.2315 0.0025 59.42 0.266 0.002 0.200

I want to use the "SN:" string as an identifier for the lines of interest - so that I can ignore the headers and only take the entries for the 8 variables.

The number of variables should be constant, although I would read in the first header to determine how many variables there are.

I have been searching fgetl, but am not sure how to rip the remaining entries in a line after the SN identifier?

thanks
Renee

On Dec 11, 7:56 am, "Renee Hlozek" <renee.hlo...@gmail.com> wrote:
> Hi
>
> I realise this post is a little old.. but I have a similar problem. I have a data file combined from a few files which each have headers, such as
>
> NVAR:  8
> VARNAMES: CID Z ZERR DLMAG DLMAGERR AV AVERR
>
> SN: 9567881   0.2883  0.0025 45.775  0.081  0.010  0.024  
> SN: 92387878 0.2835  0.0025 99.42  0.036  0.002  0.200
>
>  NVAR:  8
> VARNAMES: CID Z ZERR DLMAG DLMAGERR AV AVERR
>
> SN: 9567881   0.2883  0.0025 45.775  0.081  0.010  0.024  
> SN: 92387878 0.4535  0.0025 99.42  0.036  0.002  0.200
> SN: 9289278 0.255  0.015 75.42  0.056  0.002  0.200
> SN: 921283 0.2315  0.0025 59.42  0.266  0.002  0.200
>
> I want to use the "SN:"  string as an identifier for the lines of interest - so that I can ignore the headers and only take the entries for the 8 variables.
>
> The number of variables should be constant, although I would read in the first header to determine how many variables there are.
>
> I have been searching fgetl, but am not sure how to rip the remaining entries in a line after the SN identifier?
>
> thanks
> Renee

line=fgetl(fid);
if strcmp(line(1:3),'SN:')
a=sscanf(line(4:end), etc
I've done most of the work.
You can finish it off.