How to import files in bulk and loop over them

Hi,

I am working with a large data set with csv files for different participants.

I have a multilayered issue I am looking to solve. First, I have the following code snippets which I would like to combine into one script. Unfortunately I have no luck creating loops which is why I am looking for help.

1) I need to import multiple participant data files (csv files) into Matlab. For each of these files I would like to import columns column 7 - 27 (21 columns in total). This should result in a matrix of 21 columns for each participant.

2) Next, I need to run the following code over all columns (column_n) of each participants imported matrix:

block_size = 512;
N = block_size*ceil(numel(column_n)/block_size);
column_n (end+1:N) = NaN;
windows_1n = reshape(column_n,512,[])

This should yield a several variables (windows) for each participant (for participant 1: "windows_1A", "...", "windows_1n").

3) Then, I would like to run the following code over each of those variables:

[m,n] = size(ctpz_windows);
F = zeros(n,1);
for i = 1:n
    r = windows_n(:,i);
    F(i,1) = dfaedit(r,0,0,0);
end
H_all_n = [H_all_n,F]

This way I should receive a matrix variable for each participant (for participant 1: "H_all_1", "..." , "H_all_n").

I hope this isnt too much. Any help with automating this process would be very much appreciated.

Thank you!

[Stephen23]

"Any help with automating this process would be very much appreciated."

https://www.mathworks.com/help/matlab/import_export/process-a-sequence-of-files.html

[lil brain]

Thanks!

Answer 1

[Mathieu NOE]

hello

to start... an example below to show how to loop over directories and load in natural sorting order the files in each folder

you can easily adapt the code to your own specific needs

clc
clearvars

%% define path
yourpath = pwd; % or your specific path
list=dir(yourpath);  %get info of files/folders in current directory
isfile=~[list.isdir]; %determine index of files vs folders
dirnames={list([list.isdir]).name}; % directories names (including . and ..)
dirnames=dirnames(~(strcmp('.',dirnames)|strcmp('..',dirnames))); % remove . and .. directories names from list

%% demo for excel files
sheet = 1; % specify which sheet to be processed (my demo) - if needed

%% Loop on each folder
for ci = 1:length(dirnames) % 
    fileDir = char(dirnames(ci)); % current directory name
    S = dir(fullfile(fileDir,'*.xlsx')); % get list of data files in directory according to name structure 'Sheeta*.xlsx' 
    S = natsortfiles(S); % sort file names into natural order (what matlab does not) , see FEX : 
               %(https://fr.mathworks.com/matlabcentral/fileexchange/47434-natural-order-filename-sort)

    %% Loop inside folder
    for k = 1:length(S) % read data in specified sheet
        data = xlsread(fullfile(fileDir, S(k).name),sheet); % or use a structure (S(k).data ) to store the full data structure
        % your own code here for data processing. this is just for my demo
        % for now :
        title_str = [fileDir ' / ' S(k).name ' / sheet : ' num2str(sheet)];
        figure,plot(data),title(title_str);
    end
        
end

[lil brain]

Thank you! This looks very helpful!