MATLAB Answers

0

Help using textscan on a .dat file

Asked by Nathaniel Howard on 17 Jul 2015
Latest activity Commented on by dpb
on 19 Jul 2015
Hi guys,
I'm having some trouble trying to automaticallly load .dat files in my matlab code.
I'm pretty sure I know what the problem is but here's the error message anyway:
Error using load
Number of columns on line 201
of ASCII file G:\Thesis
DATA\New
Test\sunday_test_4.dat must be
the same as previous lines.
Basically my .dat file is pretty standard, it has a time column followed by 16 channel of transducer depth values.
The problem is the final line of the file has a date line which has less columns than the rest and so the process fails here.
An example of the final lines of text within the file is shown below.
17:01:07.458 10.28 10.40 10.50 10.44 10.46 10.40 10.32 10.20 10.23 10.40 10.68 10.66 10.80 10.80 10.74 10.83
17:01:15.459 10.26 10.40 10.49 10.50 10.47 10.43 10.32 10.23 10.29 10.38 10.57 10.65 10.77 10.81 10.71 10.86
17:01:23.459 10.28 10.44 10.43 10.46 10.49 10.47 10.32 10.28 10.34 10.38 10.56 10.62 10.83 10.81 10.71 10.77
17:01:31.459 10.34 10.40 10.49 10.52 10.49 10.50 10.38 10.26 10.29 10.44 10.56 10.65 10.80 10.81 10.72 10.75
Data end time 07/12/2015 17:03:55
And here is my code so far:
[filename2, directory_name2] = uigetfile('*.dat', 'Load the test data');
fullname2 = fullfile(directory_name2, filename2);
C3=load(fullname2);
C1 =textscan(C3,'%s %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32');
Simply put, I need the code to either ignore the final line within the .dat file or to store it somehow (the latter would be ideal).
However the number of lines within the .dat file changes depending on how long I leave the transducers running for, therefore I cant simply make the program ignore the 201'st line because it may be different the next time.
However it IS always the last line, so if there's a way of doing it, it shouldn't be that difficult. I'm just not sure how.
Any help would be greatly appreciated, Many Thanks, Nath

  4 Comments

Show 1 older comment
Hi Star Strider, Many thanks for you help. I have since attached my dat file, it is compressed because for some reason Mathworks does support .dat files as attachments?!!
Hope this helps and you assistance is very much appreciated. Cheers, Nathan
For some reason, I can’t unzip it. Your original file doesn’t look so large that there would be a problem uploading it. If the .dat extension is the problem, open it in ‘notepad’ and save a copy of it as a .txt file, then upload that version.
Hi again Star Strider, Again, many thanks for your help. Ive since attached a .txt file version of the data file. Yeah for some reason Mathworks doesn't support .dat...strange. Cheers again. Nathan

Sign in to comment.

2 Answers

Answer by Walter Roberson
on 17 Jul 2015
 Accepted Answer

[filename2, directory_name2] = uigetfile('*.dat', 'Load the test data');
fullname2 = fullfile(directory_name2, filename2);
fid = fopen(fullname2, 'rt');
C1 = textscan(fid,'%s %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32 %f32', 'CommentStyle', 'Data');
fclose(fid);
That is, lines that begin with 'Data' are to be treated as comments and discarded.

  4 Comments

Show 1 older comment
dpb
on 17 Jul 2015
I thought "either ignore the final line within the .dat file or to store it somehow (the latter would be ideal)." Hence reading the full file and then processing the N-1 lines; the last line is still around from which to parse the end date/time.
Hi dpb, Sorry for the late reply. I'm not very good with parsing and the date of acquisition isnt really essential to me. So I went with Walter's code as it did what I needed.
Many thanks however for your assistance. It's greatly appreciated.
dpb
on 19 Jul 2015
Don't be intimidated--it's really the same solution as Walter's other than reading the file into memory first. Once it's there, parsing is the same as his on the input file except from memory instead.

Sign in to comment.


dpb
Answer by dpb
on 17 Jul 2015

Either just use fgetl on each line and test for the end line characteristic string, parsing each line and building the arrray or read the full file as a character array and find the number of lines then use textscan on the array in memory.
dat=textread('nat.dat','%s','delimiter','\n','whitespace',''); % read to cell array
dat=char(dat(1:length(dat)-1)); % convert excepting last line to character array
Parse the dat array for the numeric data and then read the last line specifically.

  2 Comments

Hi dpb, Thank you very much for your response. However I'm afraid I'm not experinced enough with Matlab to do anything with the character variable:
Also the code does not appear to omit the last line:
Here is my code thus far, would you be able to tell me how I can now convert the char file into it's useful fields, Namely time and each channels recording?
[filename2, directory_name2] = uigetfile('*.dat', 'Load the test data');
fullname2 = fullfile(directory_name2, filename2);
dat=textread(fullname2,'%s','delimiter','\n','whitespace',''); % read to cell array
dat=char(dat(1:length(dat)-1)); % convert excepting last line to character array
Many thanks , Nathan
dpb
on 17 Jul 2015
Same formatting as Walter used(*) excepting from the char() array instead of the file. Use the first N-1 lines as noted in the conversion but if do want to get the last date/time from the last line don't overwrite the original variable as the above does.
(*) Or, convert directly to mo,da,yr,h,min,etc., with '%2d:%2d:%2d.%3f' instead of keeping the string; either is suitable for datenum()

Sign in to comment.