Concatenate same fields in multiple structures in a loop using field names

I have several .mat files and each is a structure with the same fields. For each field, I'd like to create an array that has the name of that field and is the concatenation of all structures with that field name. I can provide a psuedo-code, but not sure how to write it out for real:

firststruct has field names 'one' and 'two'

secondstruct has field names 'one' and 'two'

for i = 1:number of structure files I have (the .mat files)

get the field names

for j = 1:number of field names

'this would be the name of the first field (so 'one')' = this would create a vector that concatenates all .one fields;

'this would be the name of the second field (so 'two')' = this would create a vector that concatenates all .two fields;

end

end

To try to be clearer, if firststruct.one = [1,2] and secondstruct.one = [3,4], the result I'm hoping for is one = [1,2,3,4].

[Stephen23]

"I'd like to create an array that has the name of that field ..."

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

[Daniel]

I see what you're saying. I guess I'm just trying to avoid typing as much by eliminating the structure format and just having a bunch of arrays that have the names of the fields. Is there something like struct2array that does essentially that? If so, could that be combined with your answer below to make a structure that concatenates the same fields from multiple structures and then converts that structure to a bunch of arrays?

[Stephen23]

There are two possible situations:

1- you know all of the fieldnames in advance, in which case it is simple to explicitly allocate them to variables:

one = S.one;
two = S.two;

e.g. https://www.mathworks.com/matlabcentral/fileexchange/26216-structure-fields-to-variables

2- you don't know the fieldnames, in which case see previous comment. Note that having an unknown number of variables with unknown names in the workspace makes them difficult to process, increases the risk of bugs (e.g. overwriting a variable without any warning), and makes debugging difficult.

Personally I would just leave them in the structure: that little bit of extra typing makes the data easy to track and the code easy to maintain.

Answer 1

[Stephen23]

Because all of the structures contain the same fieldnames they should be stored as one non-scalar structure:

https://www.mathworks.com/help/matlab/matlab_prog/access-multiple-elements-of-a-nonscalar-struct-array.html

>> S(1).one = [1,2];
>> S(1).two = NaN;
>> S(2).one = [3,4];
>> S(2).two = Inf;

Then the concatenation can be done in just a few lines:

>> C = struct2cell(S);
>> C = cellfun(@(c)[c{:}],num2cell(C,2:3),'uni',0);
>> Z = cell2struct(C,fieldnames(S),1);
>> Z.one
ans =
   1   2   3   4
>> Z.two
ans =
   NaN   Inf

EDIT: to load multiple mat files, where each contains one structure of unknown name:

for k = 1:numel(...)
    F = ... filename
    T = struct2cell(load(F));
    S(k) = T{1};
end

[Daniel]

My structures have different names. Is there a way to loop through them from the directory they're in or otherwise load them without typing every name and then do this?

[Daniel]

I think this is in the right direction, but I'm not getting quite what I'd hoped for. Here's my code:

files = dir('PATH\*.mat');
for i = 1:length(files)
    S(i) = load(files(i).name); 
end
C = struct2cell(S);
C = cellfun(@(c)[c{:}],num2cell(C,2:3),'uni',0);
Z = cell2struct(C,fieldnames(S),1);

I have 6 .mat files. With this code, S is a 1x6 struct, but each of those six is a structure as well. So now, to get to the values I actually want, I have something like Z.originalstructure.fieldIwant, and this does not appear to be the concatenation of all 'fieldIwant' from all the originalstructure's because it has the same length instead of six times the length (they're all the same length).

[Stephen23]

"My structures have different names."

Then you should fix this when they are loaded into MATLAB. There are several ways to achieve this, assuming that each mat file contains exactly one variable (i.e. one structure), e.g.:

for k = 1:numel(...)
    F = ... filename
    T = struct2cell(load(F));
    S(k) = T{1};
end

Note that all of the structures must have exactly the same fieldnames.

In future keep in mind that processing multiple mat files is simpler when the variable names are the same in every file.

[Daniel]

OK. Not sure I've understood all this, but at some point I guess it's fastest just to go with the type heavy way. I'm doing this:

files = dir('PATH\*.mat');
% preallocate all fields
fldone = [];
% etc
for i = 1:length(files)
    load(files(i).name); % load structures with known structure and field names
    fldone = [fldone;currentstructure.fieldone];
    % etc
end

[Stephen23]

"but at some point I guess it's fastest just to go with the type heavy way"

That increases the risk of latent, almost undetectable errors in your data: consider what will happen if one of the files does not contain one of those variables (e.g. spelling error): the data from the previous iteration will be used and you will never know. Writing code assuming that data files are perfect is just asking for trouble...

"I'm doing this:"

What you wrote is pseudo-code: if the structure names are different (as you wrote) then you would also need to include some inefficient code to access each structure name dynamically, which also adds to complexity, increases the chance of bugs, and makes debugging harder. It probably isn't much less typing in the end either, perhaps even more than what I showed in my answer. Try it and see.

[Daniel]

Hey, I know you're trying to help and I appreciate it, I really do, but we're not all in this to become expert coders. I tried your way and it didn't do what I wanted. That's probably because of something else I did, but there's more than one good enough way to do this. I think I may have misled you on structure vs .mat files having the same names. The .mat files have different names (of course), but the structures are all named the same. Since I'm looping through them, if one of them is different, the code will err. I would say that writing code with extra steps that you're pretty sure you don't need is inefficient. After searching, trying some things, and then trying what you wrote, I realized that I knew a good enough way that would be faster than spending anymore time figuring out a code that saved me some typing. In the end, I think I pretty much did option one in your answer at the top except in a loop, didnt' I?

[Stephen23]

"I think I may have misled you .... but the structures are all named the same"

That is okay, the main thing is, that we get there in the end :)

"Since I'm looping through them, if one of them is different, the code will err"

Not with the code that you have shown here, it would just continue and use the previous loop iteration's data. But you can easily write it in a more robust way (by load-ing into an output variable, e.g. T):

D = 'absolute/relative path to where the files are saved';
S = dir(fullfile(D,'*.mat'));
for k = 1:numel(S)
    T = load(S(k).name);
    S(k).one = T.structurename.fieldone;
end

Note that if the filenames contain numbers then your code could easily load them (and concatenate them) in an unexpected order. You should also read about array preallocation:

https://www.mathworks.com/help/matlab/matlab_prog/preallocating-arrays.html