MatLab isn't reading the entirety of my data?

Hello,

I've been trying to get MatLab to cycle through a folder that's full of subfolders which are all full of .txt files which all have 1.5 million samples. The order is Folder > Subfolders > Data points (.txt files) > 1.5 million samples.

My current code is embedded below. If I run the code as is, it will produce a table with columns of all the variables (16) and rows of the 1.5 million samples. But I want it to do this for ALL of my data points in every folder because I'm going to be finding the Delta value of the column of pressure (which is supposed to remain mostly the same for reliable results) for every data point.

cd 'C:\Users\Lucia\Desktop\Ames\Acoustics\Useful things\Converted Data\Runs095_125\';
allruns = {'Run095', 'Run096', 'Run097', 'Run098', 'Run099', 'Run100',...
    'Run101', 'Run102', 'Run103', 'Run104', 'Run105', 'Run106', ...
    'Run107', 'Run108', 'Run109', 'Run110', 'Run111', 'Run112', ...
    'Run113', 'Run115', 'Run116', 'Run117', 'Run118', ...
    'Run119', 'Run120', 'Run121', 'Run122', 'Run123', 'Run124', 'Run125'};
TDS = tabularTextDatastore(allruns, 'NumHeaderLines', 13, 'ReadSize','file');
TDS.SelectedVariableNames = {'Var2','Var3'};
alldata = read(TDS);

[Guillaume]

I assume this is a continuation of how-do-i-use-matlab-to-loop-through-many-folders-and-extract-columns-of-data-from-txt-files-in-thos

[per isakson]

Doc says:

If the files are not in the current folder, then location must 
be full or relative paths.

Thus, are all files in the current directory?

[Lucia Lang]

To Guillaume, yes! I just didn't want to flood one question with several other topics once I've figured out my original issue.

To isakson, yes, I've changed the directory to the location of a parent folder containing all of my subfolders and ONLY those folders.

[per isakson]

"parent folder containing all of my subfolders and ONLY those folders" then you have to provide the relative path, e.g. something like './my/subfolder/Run103'

[Lucia Lang]

Do I need to provide the path separately for every run? Is there not a way I can make a list of folder names and have Matlab cycle through that list, like in a for loop?

Answer 1

[per isakson]

AFAIK: dir is the only function that takes wildcards in folder names.

Doc on dir says:

[...] To search through folders and subfolders on the path recursively, use wildcards in the path name. For example, dir */*.txt lists all files with a txt extension exactly one folder under the current folder, and dir **/*.txt lists all files with a txt extension zero or more folders under the current folder. Characters next to a ** wildcard must be file separators.

Assumption: all files matching, 'Run*.txt', "under" the current folder shall be imported.

Thus, I think this will work (not tested)

cd 'C:\Users\Lucia\Desktop\Ames\Acoustics\Useful things\Converted Data\Runs095_125\';
sad =  dir( './**/Run*.txt' );  % (.txt files) 
allruns = fullfile( {sad.folder}, {sad.name} );
TDS = tabularTextDatastore( allruns, 'NumHeaderLines', 13, 'ReadSize','file');
TDS.SelectedVariableNames = {'Var2','Var3'};
alldata = read(TDS);

Both

sad =  dir( '.\**\Run*.txt' );
sad =  dir( './**/Run*.txt' );

works on Windows, which surprised me.