Pull together files of the same name from different folders

I am reading in several files of the same name from all different folders based on the folder name. Here I have gathered a list of the paths for these files:

axialList = dir('*_A_*');
for i = 1:length(axialList)
    folders{i} = strcat(axialList(i).folder,axialList(i).name);
end
for i = 1:length(folders)
    axialFiles{i} = strcat(folders{i},'\specimen.dat');
end

This gives me cells with all the file paths I want to read. Then I'm trying to put these names in a loop to read it. This worked before I put it in the loop:

FormatString = repmat('%f',1,7);
for i = 1:length(axialFiles)
    fileID = fopen(axialFiles{i});
    SortHeader = textscan(fileID,'%s',5,'delimiter','\n');
    Data{i} = textscan(fileID,FormatString,'delimiter','\t');
    Data{i} = cell2mat(Data{i});
    fclose(fileID)
end

Now that I have it reading in fileID in the loop, I get the error 'Invalid file identifier. Use fopen to generate a valid file identifier'. If instead I just put the file names in like here:

FormatString = repmat('%f',1,7);
for i = 1:length(axialFiles)
%     fileID = fopen(axialFiles{i});
    SortHeader = textscan(axialFiles{i},'%s',5,'delimiter','\n');
    Data{i} = textscan(axialFiles{i},FormatString,'delimiter','\t');
    Data{i} = cell2mat(Data{i});
%     fclose(fileID)
end

I get an empty cell array. Please help!

Answer 1

[Carolyn]

Here's how i ended up doing it. Some of my cell references were a bit off, and the fullfile command helped.

axialList = dir('*_A_*');
for i = 1:length(axialList)
    folders{i} = fullfile(axialList(i).folder,'\',axialList(i).name);
end
axialFiles = cell(length(axialList),1);
for i = 1:length(folders)
    axialFiles{i} = fullfile(folders{i},'\specimen.dat');
end

Data = cell(1,length(axialFiles));
for i = 1:length(axialFiles)
    fileID = fopen(axialFiles{i});
    SortHeader = textscan(fileID,'%s',5,'delimiter','\n');
    FormatString = repmat('%f',1,7);
    Data{i} = textscan(fileID,FormatString,'delimiter','\t');
    Data{i} = cell2mat(Data{i});
    fclose(fileID);
end

[Stephen23]

"...the fullfile command helped."

Note that you did not use fullfile in any particularly useful way. The whole point of fullfile is that you do NOT need to explicitly handle the file separator characters, which means it is pointless to use add '\' yourself here, because that character just gets ignored:

fullfile(axialList(i).folder,'\',axialList(i).name)

and actually all you need is this (the file separators are added for you!):

fullfile(axialList(i).folder,axialList(i).name)

Why do you use fullfile in the first loop, but use strcat in the second? Instead of this:

   strcat(folders{i},'\specimen.dat')

the separator is handled perfectly by fullfile, so you just need:

   fullfile(folders{i},'specimen.dat')

Note that you can easily combine the first two loops into one:

axialList  = dir('*_A_*');
axialFiles = cell(numel(axialList),1);
for k = 1:numel(axialList)
    axialFiles{k} = fullfile(axialList(k).folder, axialList(k).name, 'specimen.dat');
end

I recommend that you avoid using length: use numel or size instead. I recommend that you obtain both output arguments from fopen and use the assert statement, as my answer shows, because this is a very simple check that will aid you whenever the file cannot be opened: it gives a much more informative message.

Answer 2

[Stephen23]

I recommend that you use fullfile rather than concatenating strings, and that you explicitly preallocate the cell arrays before the loops.

S = dir('*_A_*');
X = [S.isdir];
assert(any(X),'No folders matched the DIR search pattern: fix the pattern!')
F = {S(X).name}; % only folders
P = {S(X).folder};
N = numel(F);
D = cell(1,N);
fmt = repmat('%f',1,7);
for k = 1:N
    fnm = fullfile(P{k},F{k},'specimen.dat');
    [fid,msg] = fopen(fnm,'rt');
    assert(fid>=3,msg)
    D{k} = textscan(fid,fmt,'delimiter','\t');
    fclose(fid);
end

Alternatively you could try dlmread.