Importing data from a text file into MATLAB as multiple variables of differing sizes

Hello,

I have an nx12 matrix in a text file with tab delimiters and would like to import certain chunks of information. I need columns 9&10 from rows 3 through n, so long as column 4 has a value of '16'. Column 4 will output '177' occasionally, which is a signal to create a new variable. The number of times column 4 outputs '177' will determine the number of variables output and each variable will be a 2-column matrix with a different number of rows. Here's what I have so far using the import tool to manually select data:

 % 
% This is from using the import tool to manually determine start and end rows from the data
% I would like to automate this process for larger files so I don't have to manually select the data and copy-paste. 
%
filename = 'C:\Program Files (x86)\Mission Planner\WP Files\Constraints\Constraints_0m.waypoints';
delimiter = '\t';
startRow = [3,10,29,42,49,58,67,76,86,95,110]; % always start on row 3
endRow = [8,27,40,47,56,65,74,84,93,108,113];  % ends at 'n' which is 113 here.
% rows [9,28,41,48,57,66,75,85,94,109] had '177' in column 4... skip.
% The rest had '16' in column 4. This leaves 'm' chunks of data. 
% 'm' and 'n' will vary for each file.
%
formatSpec = '%*s%*s%*s%*s%*s%*s%*s%*s%f%f%*s%*s%[^\n\r]';
fileID = fopen(filename,'r');
%
dataArray = textscan(fileID, formatSpec, endRow(1)-startRow(1)+1, 'Delimiter', delimiter, 'TextType', 'string', 'HeaderLines', startRow(1)-1, 'ReturnOnError', false, 'EndOfLine', '\r\n');
%
for block=2:length(startRow)
    frewind(fileID);
    dataArrayBlock = textscan(fileID, formatSpec, endRow(block)-startRow(block)+1, 'Delimiter', delimiter, 'TextType', 'string', 'HeaderLines', startRow(block)-1, 'ReturnOnError', false, 'EndOfLine', '\r\n');
    for col=1:length(dataArray)
        dataArray{col} = [dataArray{col};dataArrayBlock{col}];
    end
end
%
fclose(fileID);
%
Constraints0m = table(dataArray{1:end-1}, 'VariableNames', {'VarName9','VarName10'});
% This only outputs one single 2-column table when I need 'm' 2-column variables
%
clearvars filename delimiter startRow endRow formatSpec fileID block dataArrayBlock col dataArray ans;

I'm guessing 'starRow' and 'endRow' will need to be generated first based on column 4 with a 'textscan' loop, then I need to create a new variable or index for each chunk of information. I will need to know 'm' as an output because the rest of my code runs loops based on the size of m (which I've hard coded for the time being but will need to automate) .

Answer 1

[Siddharth Bhutiya]

Yes, you are right. You can start by reading the entire data using textscan. After that, you can use something similar to the code snippet below to obtain the row indices where the output is '177'.

signalValues = mydata{:,4}; % here we get the 4th column from the data
idx = find(ismember(signalValues, '177'));

You can get more information about "ismember" and "find" functions using the links below

https://www.mathworks.com/help/matlab/ref/ismember.html

https://www.mathworks.com/help/matlab/ref/find.html

Once you have these you can generate the vectors for start and end row indices and divide your data. Also, you can index the entire data to get the subparts instead of using multiple textscans inside the for loop. For example, to get rows from start_index to end_index and columns m to n, you can index the data as follows,

mydata(start_index:end_index, m:n);