How to load and perform calculations on multiple .mat-files?

Hi,

We are new to Matlab and we would need help with the following problem: We have 100 .mat files in a folder consisting of a matrix with several column vector variables. The aim is to have an automated process which reads in all the files and performs certain calculations on all of them. In more detail it should do the following: A specific function file (already existing) is called to perform a calculation on each file and should then somehow record the value (probably store the values in a new vector?!) of one of the function outcome variables as we need to do further calculations on them. We would need to calculate the mean of these values. Can anybody show us how a script for our problem would look like? We read something about Batch-programs and loops, but we are not sure how exactly to solve the problem. Can it be done with either of those (batch vs. loop) or do we need to combine both? So would a loop allow me to read in multiple files and perform a function on each one? If so, how do we have to write such a loop? And there is another problem: For the function file to work, certain function input variables need to be predefined in the M-file. Do we have to define them within the loop? And for the Batch version we already created one file with all files included. Here a shortened version of it:

clc

global G

G='7000-XXX-1Out4';
'GrandAveOUT4_3'

G='7000-XXY-1Out4';
'GrandAveOUT4_3'

G='7000-XXZ-1Out4';
'GrandAveOUT4_3'

We furthermore used these commands for the main M-File to “connect” it to the other Batch-file, but it somehow doesn’t work:

%%Set path
cd ('N:\backups\Folder5\HR data analysis')

thePath.data = fullfile(pwd,'subject data','PTSDdata','to be added');
thePath.programs = fullfile(pwd,'programs');

addpath(genpath(thePath.programs))
addpath(genpath(thePath.data))

%%Select any Out4 file and open it
cd (thePath.data)               

%cd (pwd)

global F

display(F);

If using the Batch solution for our problem, how could we connect the files with one another?

Any kind of help would be highly appreciated. Thank you!

Answer 1

[per isakson]

You could try something like

    struct_array_of_matfiles = transpose( dir( 'folder_in_question\*.mat' ) );
    for sa = struct_array_of_matfiles 
      struct_of_data = load( fullfile( folder_in_question, sa.name ) );
    end

What do you mean by "connect the files with one another" ?

[Jennifer]

Thank you for your answer!

By "connecting the files" I meant we created one file (BATCH-file) containing the following commands:

clc
global G 

G='7000-XXX-1Out4';
'GrandAveOUT4_3'

G='7000-XXY-1Out4';
'GrandAveOUT4_3'

G='7000-XXZ-1Out4';
'GrandAveOUT4_3'

It contains the names of the datafiles to be analyzed.

We also created a second file (main M-file) which includes the commands for the various calculations to be carried out. This is the code we came up with to tell the program/file that the calculations should be done on each of the datafiles mentioned in the BATCH-file:

%%Set path
cd ('N:\backups\Folder5\HR data analysis')
thePath.data = fullfile(pwd,'subject data','PTSDdata','to be added');
thePath.programs = fullfile(pwd,'programs');
addpath(genpath(thePath.programs))
addpath(genpath(thePath.data))

cd (thePath.data)               
  %cd (pwd)
  global F
  display(F);

How do I tell this main M-file to do the calculations on each datafile mentioned in the other file, the "BATCH"-file? Any suggestions what code might be missing or what could be wrong?

Thank you so much!

[Matt Kindig]

I'm not sure I understand the contents of the "BATCH" file. Is this just a list of files that you want to process? If so, this doesn't need to contain Matlab files at all, just a listing of filenames, separated with some type of separator (perhaps ;). That is, something like this:

=== BATCH file (e.g. batch.txt)=========
File1;  File2;  File3;  File4;  File5;  ....

You can then read the contents of this file into Matlab to extract the filenames, and run your main M-file on each file:

===main.m ======================
RunFiles = regexp(fileread('batch.txt'), ';', 'split');  
%RunFiles is now a cell array of file names

for k=1:length(RunFiles),
   load(fullfile(...., RunFiles{k}));  %load *.mat data
   %your code to run here per file.
end

Or is the problem more complicated than this and I am just unclear as to what you are doing?

[per isakson]

The documentation of Matlab use the word, script, to mean what you call BATCH-file.

In your code

    clc
    global G 
    G='7000-XXX-1Out4';
    'GrandAveOUT4_3'
    G='7000-XXY-1Out4';
    'GrandAveOUT4_3'
    G='7000-XXZ-1Out4';
    'GrandAveOUT4_3'

the variable, G, is in turn assigned three different string values. The two first values are overwritten.

Proposal: replace this code by

    file_name_list = {'7000-XXX-1Out4','7000-XXY-1Out4','7000-XXZ-1Out4'};

I cannot see why you need to manipulate the Matlab path.

Next I propose something like

    data_folder = 'N:\backups\Folder5\HR data analysis';

    for jj = 1 : length( file_name_list )
        sas( 1, jj ) = load( fullfile( data_folder, [file_name_list{jj},'.mat'] );
    end