MATLAB Answers

How can I conduct statistical analysis of runs that collate into one variable?

1 view (last 30 days)
Elizabeth Hearn
Elizabeth Hearn on 4 Aug 2019
Commented: dpb on 5 Aug 2019
I am conducting an experiment where all my model runs collate into one variable. i.e. my runs are: Temp001 Temp002 Temp003 etc. To view the data for each of these runs I have to load Temp001 into the workspace as variable Temp and it displays the data. If I want to view the data for Temp002, I have to do the same and it loads the data for Temp002 into Temp as well. I cannot view the data for Temp001 and Temp003 at the same time as they are always loaded into the single variable 'Temp'.
For example:
Temp001: 4 5 6 7
Temp002: 1 2 3 4
Temp003: 9 8 7 6
load Temp001
display(Temp): 4 5 6 7
load Temp002
display(Temp): 1 2 3 4 % I can no longer view the data for Temp001, only for Temp002
I am conducting hundreds of runs and want to conduct statistical analysis on the runs (median, percentiles etc). How do I do this without having to save each individual run (of which there are hundreds) as separate variables?
For example:I want to find the median of each column over the 3 runs. Is there a way I can do this without having to type (the below) hundreds of times?:
load Temp001
load Temp002
load Temp003
Please help me, it will take forever otherwise!

  1 Comment

dpb on 4 Aug 2019
What does a "model run" consist of? The problem begins there in that you're not creating a way to easily handle multiple cases there.
Show us how that is done and it's probably trivial (or nearly so) to fix...
A big klew one isn't doing things properly in Matlab is when start generating sequential variables with subsequent letter or number as above...

Sign in to comment.

Answers (1)

Image Analyst
Image Analyst on 4 Aug 2019
For example, it might be close to this:
% Specify the folder where the files live.
myFolder = 'C:\Users\yourUserName\Documents\My Data';
% Check to make sure that folder actually exists. Warn user if it doesn't.
if ~isdir(myFolder)
errorMessage = sprintf('Error: The following folder does not exist:\n%s', myFolder);
% Get a list of all files in the folder with the desired file name pattern.
filePattern = fullfile(myFolder, '*.mat'); % Change to whatever pattern you need.
theFiles = dir(filePattern);
for k = 1 : length(theFiles)
baseFileName = theFiles(k).name;
fullFileName = fullfile(myFolder, baseFileName);
fprintf(1, 'Now reading %s\n', fullFileName);
% Now do whatever you want with this file name,
% such as taking the median
s = load(fullFileName); % Load contents of file into structure s.
thisData = s.Temp;
theMedians(k) = median(thisData);
theMeans(k) = mean(thisData);


Show 1 older comment
Elizabeth Hearn
Elizabeth Hearn on 5 Aug 2019
Unfortunately I cannot rewrite the code as it hasnt come from me.
This does something but it doesnt quite work. Each run is in the current folder under 'Temp001, Temp002, Temp003 etc' but the variable 'Temp' is in the working directory. Therefore in this part:
filePattern = fullfile(myFolder, '*.mat'); % Change to whatever pattern you need.
when I replace the * with 'Temp' it doesn't recognise it as it isn't Temp that is in the folder path, it is the 'Temp00x'. For example it gives this output:
theFiles =
0×1 empty struct array with fields:
Image Analyst
Image Analyst on 5 Aug 2019
If you can't change the code, then you're stuck with what you have. However if you don't change anything, then the situation will remain as it is.
dpb on 5 Aug 2019
Well, even if you didn't write the actual application, you should be able to write a wrapper routine that calls it in a loop...
The pattern IA means is to match whatever is the needed filename extension and wildcard to match the naming convention your files use-- load uses '.mat' by default so that is the pattern he illustrated and is probably the correct one. I'd probably have been a little more selective and written 'Temp*.mat', just in case there are any other .mat files in this subdirectory, to exclude anything but the specific ones wanted.
The subdirectory string needs to be the other argument to fullfile() where IA gave you the hint to put it by using "myFolder" as the placeholder.
The result of executing fullfile(myFolder, '*.mat') needs to give you a proper either fully-or partially qualified filename wildcard string that refers to the desired folder.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!