Access varying variable names in loop

Hi,

i set up an 8x19 array with numbers varying from 4 to 152, including zeros. (numbering)

I now want to fill a cell-array with data (currently stored in struct array). These struct arrays are named something_numbering, where the number is one of those from 4 to 152.

I tried the following (which obviously did not work):

for m=1:8
    for n=1:19
        if numbering ~= 0 %check if 8x19 array with numbers is nonzero
            data_vector{m,n} = something_('%03d',numbering(m,n)).something.something;
        end
    end
end

The problem is to include the varying numbering and to maintain the structure of the numbering array.

To make things more precise let me provide you with a minimal example:

numbering = [1 5 23; 0 7 24; 0 9 0]
% i now want the data-array to be
data_vector = [something_001 something_005 something_023;
               0 something_007 something_024;
               0 something_009 0]

Happy for any help! Cheers

[Stephen23]

"The problem is to include the varying numbering and to maintain the structure of the numbering array."

Yes, that is a problem.

"Happy for any help!"

Don't do that:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

So far you have not told us the most important information: how did you get all of those structures into the workspace? Most likely you did not sit and write out all of their names by hand, so you must have either imported them or created them by code. That is the best place to fix this bad data design.

For example, instead of LOADing directly into the workspace you should always LOAD into an output variable:

S = load(..);

S is a scalar structure, you can easily and efficiently loop over its fieldnames:

https://www.mathworks.com/help/matlab/ref/fieldnames.html

https://www.mathworks.com/help/matlab/matlab_prog/generate-field-names-from-variables.html

Even better would be to avoid forcing meta-data into variable names or fieldnames: meta-data is data, and should be stored in arrays, not in field/variable names. For example, if that data had been sensibly stored in one array then your task would be easy and efficient using some basic indexing. In contrast, bad data design requires slow, complex code to process it.

[Oskar Kilgus]

Hey Stephen, thanks for the reply!

".. how did you get all of those structures into the workspace?"

addpath '/Users/.../...'

load('idx.mat')
filenames = cell(91,1);
for i=1:91
filenames{i,:} = sprintf('data_%03d.mat', idx(i));
end

folder_path = '/Users/.../folder_with_stored_data';
file_list = dir(folder_path);
file_list = file_list(3:end);

for i=1:length(filenames)
    file_path=[folder_path, file_list(i).name];
    data = load(file_path);
    eval([filenames{i}, '=data;']);
end

As i´m not too much into programming i just figured some way of loading all the data.

Although the eval function or my code in general might make you cry, is there any workaround to avoid doing it all over again? It doesnt have to be robust or a beauty of code by any means...

Thanks in advance.

[Stephen23]

"It doesnt have to be robust or a beauty of code by any means..."

... and is leading to the difficulties that you are facing.

Note that the code you show does not check the order of FILENAMES against the results returned by DIR: the files returned by DIR could be competely different or in a different order and your code would keep on using the same variable names in the same order. This makes your intent unclear: what are you actually trying to achieve? Posting broken code is nowhere a good as your explaining what you are trying to do (note: this means explaining the situation and goal, not your approach to it https://xyproblem.info/ ). Are you trying to rename files? Are you just trying to process some data (if so, how?).

There are a few other bugs, such as assuming that the first two elements returned by DIR are always dot directory names.

And then there is the EVAL and making things magically appear in the workspace. Best avoided. Makes your life harder.

[Oskar Kilgus]

Thanks again,

i´ll try to make things clear:

The data i want to process is stored in .mat files containing struct arrays with 10 fields (data-vectors themselves) in the "second layer". Every .mat file represents one test-run of a system i gathered sensor-data from. I changed certain parameters of this system after a couple test-runs. Therefore i have between 2 and 8 test-runs with the same parameteres, hence 2 to 8 .mat files representing a setting of parameters. (19 parameter sets)

My intent is to load all these .mat files in a way, that i can easily access the vectors in the "second layer" of the struct-arrays in, lets say a for loop. This way i can ease my data-analysis (e.g. FFT analysis)

I tried to do so in loading all the .mat files in the workspace with according numbering and then store thes data in an array with columnwise parameter settings. (e.g. first column has parameter set A, second B and so forth...) and rowwise test-runs (e.g. first row stores data of one test-run, second of the next, up until max. 8 per column)

I hope you understand what i´m trying to say..

The problem is, that there are 91 data-sets (.mat files) and i want to find a way of storing them clearly arranged and without typing 91 lines of code for every data-vector i want to access.

edit: another problem is, that the .mat files are all named with numbers in front so i cant import them directly can i?

[Stephen23]

"another problem is, that the .mat files are all named with numbers in front so i cant import them directly can i?"

I don't see why that would be a problem, this is very common. I routinely import many thousands of data files which have all kinds of numbers in the filenames. Filenames are rather irrelevant to what you can import.

"My intent is to load all these .mat files in a way, that i can easily access the vectors in the "second layer" of the struct-arrays in, lets say a for loop. This way i can ease my data-analysis (e.g. FFT analysis)"

I strongly recommend to import them into one array. From your description, a non-scalar structure might be a good option. That would make it eas to keep all of the imported data together along with the corresponding filenames, etc.

From the non-scalar structure you can simply loop over the elements, etc, and use convenience syntaxes:

https://www.mathworks.com/matlabcentral/answers/1656435-tutorial-comma-separated-lists-and-how-to-use-them

Hopefully all of the structures saved within the MAT files have the same name (simpler, more robust, more efficient to process) and not a different name in each MAT file (... ugh, that would be very... unfortunate).

Hopefully the structure fields also have the same names, then merging them together will be easier.

Please upload some sample MAT files by clicking the paperclip button.

In lieu of that, as far as I can tell you have something like this (structures saved in MAT files):

S = struct('X',1:3,'Y','A'); save test1.mat S
S = struct('X',4:6,'Y','B'); save test2.mat S
S = struct('X',7:9,'Y','C'); save test3.mat S

which is easy to import in a loop:

P = '.'; % absolute/relative path to where the files are saved
S = dir(fullfile(P,'*.mat'));
for k = 1:numel(S)
    F = fullfile(P,S(k).name);
    D = load(F);
    D.S.name = F;
    S(k).data = D.S;
end
D = [S.data] % all file data

D = 1×3 struct array with fields:
    X
    Y
    name

vertcat(D.X) % all X data in one matrix

ans = 3×3

1 2 3 4 5 6 7 8 9

{D.name}.' % corresponding filenames

ans = 3×1 cell array
    {'./test1.mat'}
    {'./test2.mat'}
    {'./test3.mat'}

Simplify the data design and you make processing it much easier and more efficient. The name "MATLAB" comes from "MATrix LABoratory". MATLAB is designed so that working with matrices and arrays is the best approach (not splitting up the data into lots of separate variables and making the data harder to work with).

[Steven Lord]

edit: another problem is, that the .mat files are all named with numbers in front so i cant import them directly can i?

If you're calling load on a MAT-file whose name starts with numbers in front that's no problem. If you're calling load to read data from a text file in a format load can handle (created with save -ascii for example) then MATLAB would need to create a variable whose name is a modified version of the text file name that is a valid MATLAB identifier.

cd(tempdir)
x = 1:10;
y = x.^2;
save('123mat.mat', 'x', 'y')

The data was stored as expected in the MAT-file.

whos -file 123mat.mat

  Name      Size            Bytes  Class     Attributes

  x         1x10               80  double              
  y         1x10               80  double              

Calling load to read it into a struct array works fine.

data = load('123mat.mat')

data = struct with fields:
    x: [1 2 3 4 5 6 7 8 9 10]
    y: [1 4 9 16 25 36 49 64 81 100]

Calling load without an output argument risks overwriting variables that already exist in the workspace. You might expect z to be [42; 3+4i] in the code below; it's not. [If 123mat.mat had not contained variables named x or y it would have.]

x = 42;
y = 3+4i;
load('123mat.mat')
z = [x; y]

z = 2×10

1 2 3 4 5 6 7 8 9 10 1 4 9 16 25 36 49 64 81 100

Now let's look at a text file. The data looks like it was written correctly in the file (the format of the numbers is a bit different, but the values are the same as we'll see at the end looking at the variable q.)

save('123mat.txt', '-ascii', 'x', 'y')
type 123mat.txt

   1.0000000e+00   2.0000000e+00   3.0000000e+00   4.0000000e+00   5.0000000e+00   6.0000000e+00   7.0000000e+00   8.0000000e+00   9.0000000e+00   1.0000000e+01
   1.0000000e+00   4.0000000e+00   9.0000000e+00   1.6000000e+01   2.5000000e+01   3.6000000e+01   4.9000000e+01   6.4000000e+01   8.1000000e+01   1.0000000e+02

If we call load with no outputs MATLAB doesn't create a variable named 123mat in the workspace. It can't since that's not a valid variable name. But it creates something close.

clear all
load 123mat.txt
whos

  Name         Size            Bytes  Class     Attributes

  X123mat      2x10              160  double              

Where does that variable name come from? See the Algorithms section of the load documentation page.

But if we call load with an output that works fine.

q = load('123mat.txt')

q = 2×10

1 2 3 4 5 6 7 8 9 10 1 4 9 16 25 36 49 64 81 100

Answer 1

[Jacob Mathew]

Hi Oskar,

I can point you towards a simple example and hope you can build from it. Conside the code below:

x = 'hello';  % Dynamic content
myStruct = struct('hello', 'world');  % Struct with a field 'name'
fieldValue = myStruct.(x)

fieldValue = 'world'