import data function breaks at NaN or not a number

6 views (last 30 days)
I am trying to use import_data function to import dataset from the data file. This data set is generated from iterations of an experiment based upon the response from the subjects. This kind of data set is bound to end up have a Not a Number or a missing response and not extract anything after that. I am trying to figure out a way to import data and not have the function break at Not A Number or a missing response. The function extracts the row before the Not a Number and not after the NaN row. I am attaching the data file that i am interested in. Any help would be greatly appreciated.
P.S. The experiment running the experiment generates a .txt file everytime and i am not allowed to mess with that program.I am including the .txt file as well

Accepted Answer

Guillaume
Guillaume on 11 Dec 2015
Edited: Guillaume on 11 Dec 2015
With format like this you have no choice but to write your own parser. Here is a draft:
function testresult = import_testresult(filename)
%import_testresult imports an individual test results stored in a text file
%
% filename: full path of the file to import
% tesresult: a structure containing the test results
filecontent = fileread(filename); %read whole file at once
sections = strsplit(filecontent, ''''''''''''''''''''''''''''''''''''''''''''''''''''''''); %split into individual sections
%the use of quotes for delimiting section is unfortunate as each quote needs to be escaped in matlab
sections = cellfun(@(s) strsplit(s, {'\n', '\r'}), sections, 'UniformOutput', false); %split each sections into lines
%at this point, you have 4 sections (+1 empty one for the last line of the file) split into individual lines
%parse each section.
%
%section 1: assume all values are text. Note that the last cell is empty (due to the way the file is parsed earlier)
propvalues = regexp(sections{1}(1:end-1), '\[([^\]]+)\] (.+)', 'tokens', 'once'); %extract property names and values
propvalues = cellfun(@(pv) {matlab.lang.makeValidName(pv{1}), pv{2}}, propvalues, 'UniformOutput', false); %convert property names into valid matlab variable names
propvalues = vertcat(propvalues{:})'; %concatenate into a single cell array
testdescription = struct(propvalues{:}); %convert into structure
%
%section 2: assume all values are numbers. Note that the first and last cell are empty (due to the way the file is parsed earlier)
propvalues = regexp(sections{2}(2:end-1), '\[([^\]]+)\] (.+)', 'tokens', 'once'); %extract property names and values
propvalues = cellfun(@(pv) {matlab.lang.makeValidName(pv{1}), str2double(pv{2})}, propvalues, 'UniformOutput', false); %convert property names into valid matlab variable names
propvalues = vertcat(propvalues{:})'; %concatenate into a single cell array
testsummary = struct(propvalues{:}); %convert into structure
%
%section 3: assume it's a table of numbers with two header lines. missed and no response are converted to NAN
%Note that the first and last cell are empty (due to the way the file is parsed earlier)
results = cellfun(@(l) strsplit(l, '\t'), sections{3}(4:end-1)', 'UniformOutput', false); %split each line at the tab character
results = str2double(vertcat(results{:})); %convert to a matrix
%
%section 4: assume it's a column of number with a header.
%Note that the first and last cell are empty (due to the way the file is parsed earlier)
rawtimes = str2double(sections{4}(3:end-1)');
%put everything together into one structure
testresults = struct('description', testdescription, 'summary', testsummary, 'results', results, 'rawtimes', rawtimes);
end
There's no validation anywhere that the file conforms to the expectations, you would have to add that to make the code robust.

More Answers (1)

Walter Roberson
Walter Roberson on 10 Dec 2015
I have no problem with that file even on a Mac (which cannot use Excel to open the file.)
[num, txt, raw] = xlsread('time-react.xlsx');
The num array has NaN for the missing or string responses. txt or raw can be used to get to the corresponding strings.
  7 Comments
Walter Roberson
Walter Roberson on 13 Dec 2015
There was originally an excel file that would have been imported without difficulty; that was replaced with the tricky text file.
Sanwal Yousaf
Sanwal Yousaf on 17 Dec 2015
The excel file was made by me for ease. The text file is something that the function would have to deal with. There is a program that runs the experiment and then generates the data in a text file and i am not allowed to mess with that program. As a result, the code that i write must be adept at dealing with .txt files for the forseeable future

Sign in to comment.

Categories

Find more on Data Import from MATLAB in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!