How to Call multiple Database files and store the variables in them separately using loops?

I have around 1200 database .mat (say the database file name being 12c1.mat,12c2.mat and so on) files each containing 15 variables of same name (example : amx , ff ). I am trying to store them in a systematic manner by making a separate database ( named amx , which will have amx12c1, which carries the value of variable amx from database 12c1). I need to make a loop to call them out and assign them in a new variable (amx12c1) and then storing them in a database ( amx.mat) .

Kindly help.

[Stephen23]

"I need to make a loop to call them out and assign them in a new variable"

Putting meta-data into variable names is not recommended.

Accessing variable names dynamically is one way that beginners force themselves into writing slow, complex, obfuscated, inefficient code. Read this to know why:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

Rather than creating lots of differently-named variables or using awkward dynamic fieldnames, you should just use indexing, e.g. using a cell array or non-scalar structure:

https://www.mathworks.com/help/matlab/matlab_prog/access-multiple-elements-of-a-nonscalar-struct-array.html

[Guillaume]

Yes as Stephen says, your idea of creating variable names with metadata is only going to lead you down a rabbit holes of problems. Do not do that.

It may be better to refocus this question on finding the optimal storage method for your database. So please, give us an overview of what you're doing. Assuming that actually using a proper database system (mysql or whatever) is overkill for you, the closest data structure in matlab akin to databases is the table (which implements the join, outerjoin and innerjoin of the database world).

Answer 1

[Stephen23]

% Fake data:
amx = 1;
ff  = 11;
save('12c1.mat','amx','ff')
amx = 2;
ff  = 22;
save('12c2.mat','amx','ff')
clear
% Main code, collect data into one structure:
S = dir('12c*.mat');
for ii = 1:numel(S)
    T = load(S(ii).name);
    F = fieldnames(T);
    for jj = 1:numel(F);
        S(ii).(F{jj}) = T.(F{jj});
    end
end
% At this point you could easily save S, it contains all of your data!
% Not recommended (better to use indexing):
[~,N] = cellfun(@fileparts,{S.name},'uni',0);
for jj = 1:numel(F)
    T = cell2struct({S.(F{jj})},strcat(F{jj},N),2)
    save(sprintf('%s.mat',F{jj}),'-struct','T')
end
% Checking:
>> Z = load('ff.mat')
Z =
  scalar structure containing the fields:
    ff12c1 =  11
    ff12c2 =  22

[Guillaume]

In my opinion, using matlab structures as database is not a good idea. But as I said in my comment, we need more information about what the OP is doing to know for sure. I suspect a flatter storage may be more useful.

[Kennedy Mtandavari]

Why do you think its not a good idea?

[Stephen23]

@Kennedy Mtandavari: because meta-data is data, and data should be stored in variables (and not in variable names or fieldnames). Forcing meta-data into variable/fieldnames makes accessing data fragile, slow, and complex.

For example, filenames (meta-data) should be stored simply as data in an array (e.g. like the structure array returned by DIR), which is robust because it works for all filenames, and is simple because the array is trivial to loop over, etc. However consider the popular attempt to use filenames as a structure fieldname:

[~,F,~] = fileparts('test.csv')

F = 'test'

S.(F) = 2;

Well, that seemed to work... but is it robust, will it always work? Answer: no. There are many many many many characters that are valid in filenames that are not valid in structure fieldnames, e.g.:

[~,F,~] = fileparts('test-new.csv')

F = 'test-new'

S.(F) = 3;

Invalid field name: 'test-new'.

Nope, in general that code is buggy. And yet can be avoided so easily.