MATLAB Answers

Luffy
0

How to read text files from different sub folders in a folder ?

Asked by Luffy
on 30 Sep 2015
Latest activity Edited by Stephen Cobeldick on 16 Sep 2019
I have a collection of .txt files which are an output of a simulation software.The software puts the text files in different sub-folders within a folder.Is it possible to read all of those ?
For example:- In a single folder named top, there are 500 sub-folders(each subfolder is of format TC_SYS_01,TC_SYS_02,.....TC_SYS_500) with a text-file in each of them.I want to read those text files from each of those folders & do some operations.
I'm thinking of changing the simulation software code itself so that it outputs txt files within a folder(no sub-folders) & use this code:
F = dir('*.txt');
for i = 1:length(F)
text = fileread(F(i).name) ;
% Do further operations
end
but can i do it without changing the simulation software code and this : http://in.mathworks.com/matlabcentral/newsreader/view_thread/301929 doesn't help me

  0 Comments

Sign in to comment.

4 Answers

Answer by Image Analyst
on 30 Sep 2015
 Accepted Answer

Not sure which code you used in that link, but you can use genpath() to generate a list of all subfolders, then use a loop to recurse into each of them. In the innermost for loop is where you'll process your text file. Try it - just copy and paste and it should work. Here is the code:
% Start with a folder and get a list of all subfolders.
% Finds and prints names of all text files in
% that folder and all of its subfolders.
% Similar to imageSet() function in the Computer Vision System Toolbox: http://www.mathworks.com/help/vision/ref/imageset-class.html
clc; % Clear the command window.
workspace; % Make sure the workspace panel is showing.
format long g;
format compact;
% Define a starting folder wherever you want
start_path = fullfile(matlabroot, '\toolbox');
% Ask user to confirm or change.
topLevelFolder = uigetdir(start_path);
if topLevelFolder == 0
return;
end
% Get list of all subfolders.
allSubFolders = genpath(topLevelFolder);
% Parse into a cell array.
remain = allSubFolders;
listOfFolderNames = {};
while true
[singleSubFolder, remain] = strtok(remain, ';');
if isempty(singleSubFolder)
break;
end
listOfFolderNames = [listOfFolderNames singleSubFolder];
end
numberOfFolders = length(listOfFolderNames)
% Process all text files in those folders.
for k = 1 : numberOfFolders
% Get this folder and print it out.
thisFolder = listOfFolderNames{k};
fprintf('Processing folder %s\n', thisFolder);
% Get filenames of all TXT files.
filePattern = sprintf('%s/*.txt', thisFolder);
baseFileNames = dir(filePattern);
numberOfFiles = length(baseFileNames);
% Now we have a list of all text files in this folder.
if numberOfFiles >= 1
% Go through all those text files.
for f = 1 : numberOfFiles
fullFileName = fullfile(thisFolder, baseFileNames(f).name);
fprintf(' Processing text file %s\n', fullFileName);
end
else
fprintf(' Folder %s has no text files in it.\n', thisFolder);
end
end

  4 Comments

Show 1 older comment
@RC: so don't use uigetdir: use a hard-coded directory path, or pass the path as a function input argument, or get it from your UI, or whatever suits your operations.
This is brilliant code thank you, as this code answers alot of my questions about looping through subfolders and extracting information. However this code does not work on my macbook pro but will work nicely on my Windows 10 work computer, is there a reason why it may not work on Macs?
You don't need this code anymore since R2016b. You can use dir() with two asterisks, or use fileDatastore. See attached demo and see if it works with your Mac. If not, post your code in a new question.

Sign in to comment.


Answer by Meade
on 2 Mar 2017

You can try the code below. It will do the same thing as Image Analyst's post, but is a little cleaner if you don't need any of the nested folder information, i.e. you only want the file paths.
mainFolder = uigetdir(); % Select your Main folder
[~,message,~] = fileattrib([mainFolder,'\*']);
fprintf('\n There are %i total files & folders.\n',numel(message));
allExts = cellfun(@(s) s(end-2:end),{message.Name},'uni',0);% Get file ext
TXTidx = ismember(allExts,'txt');% Search extensions for "CSV" at the end
TXT_filepaths = {message(TXTidx).Name}; % Use idx of TXTs to list paths.
fprintf('There are %i files with *.txt file ext.\n',numel(TXT_filepaths));
for ii = 1:numel(TXT_filepaths)
% Do import here
end

  2 Comments

Do you know a way of using this if the file name has multiple . in it? the code runs well but reads the extension after the frist . so i have a few file names such as 000.111.222.csv so the ext is read as .111 any advice is appreciated!
"the code runs well but reads the extension after the frist "
There is nothing in this code that detects the period character, so it doesn't even know where the first period character is.
Note that the code is buggy as it incorrectly assumes that all file extensions have three characters: this can easily be fixed using fileparts.
"any advice is appreciated!"
Write simpler, more robust code using dir, fullfile, and fileparts.

Sign in to comment.


Answer by KSSV
on 30 Sep 2015

Hi
You can get the names of the sub folders inside a folder using the following piece of code.
dirinfo = dir(pathfolder);
dirinfo(~[dirinfo.isdir]) = []; %remove non-directories
dirinfo(1:2) = []; % Remove the first two fields as they are . and ..
dirinfo is a structure with all the information. You can run a loop along this structure, go to the specific folder and then you can select the text files in it.
Cheers
Srinivas

  1 Comment

Thanks for the code.The above code gives the directory information in dirinfo variable. Now i know the names of all sub-folders. To access those text files i have to go inside each of those sub-folder,do text operations & go back to root directory.
For this i've tried:
length = numel(dirinfo);
for ii = 1:length
% folder name = dirinfo(ii).name
cd dirinfo(ii).name
% file operations
%
end
But i throws up an error:- Error using cd; name is non-existent or not a directory*

Sign in to comment.


Answer by Thorsten
on 30 Sep 2015

filenames = getAllFiles('.', '*.txt', 1);
and then loop through the filenames and work on them; you do not need to cd to the directories.

  0 Comments

Sign in to comment.