MATLAB Answers

Concatenate Structures: select structures only if not empty.

9 views (last 30 days)
ThorBarra on 4 May 2019
Commented: Jos (10584) on 6 May 2019
I'd appreciate help on merging data from different sources. After importing the data, I store them in structures. One structure for each data source, each structure has the same fields. So, concatenating the structures into one is straight forward:
DataMerged = [Data1; Data2; Data3; Data4];
Now, the tricky part: For some data sets (experiments), one or several of these structures can be empty, i.e. contain not even any fields. How would I need to modify the code above to concatenate only those structures that are not empty? Currently, I use this work-around:
if ~isempty(test) && ~isempty(test2) && ~isempty(test3) && ~isempty(test4)
listFiles = [test; test2; test3; test4];
disp('Concatenate 1-4')
if ~isempty(test) && ~isempty(test2) && ~isempty(test3) && isempty(test4)
listFiles = [test; test2; test3];
disp('Concatenate 1-3')
and so on, for all combinations. Would you know a more elegant way of doing this?Thanks, for any help and suggestions,


Sign in to comment.

Accepted Answer

Jos (10584)
Jos (10584) on 4 May 2019
This clearly shows the drawback of naming your variables dynamically, like A1, A2, A3, A4. If you change, for instance, the way you import variables into your workspace, this would be a trivial, non-exisiting, problem.
Let's assume that you load the data like this or something similar:
Data1 = import('Exp1.dat')
Data2 = import('Exp2.dat')
Change this to a loop, and life becomes much measier:
Files = {'Exp1.dat', 'Exp2.dat'}
for k=1:numel(Files)
Data{k} = import(Files{k}) ;
tf = ~cellfun(@isempty, Data)
AllData = [Data{tf}] ;


ThorBarra on 5 May 2019
Thanks, Jos. Works like a charm. Here is the final code:
%% List specific files
% in directory and sub-directory
% IGOR format
TmpFileList{1} = dir([dataPath_XPS filesep '**' filesep 'N*.ibw']);
% h5 format
TmpFileList{2} = dir([dataPath_NEXAFS filesep '**' filesep 'N*.h5']);
% Photos
TmpFileList{3} = dir([dataPath_Photo filesep '**' filesep 'A*.bmp']);
% Photos
TmpFileList{4} = dir([dataPath_Photo filesep '**' filesep '201*.bmp']);
Do things to each list.....
%% Merge lists into one
% get index of non-empty structures
tf =~cellfun(@isempty,TmpFileList);
% merge the lists into one structure (cat(1, A, B) is the same as [A; B].)
ListFiles = cat(1,TmpFileList{tf});
% delete all other variables
clearvars -except ListFiles
Jos (10584)
Jos (10584) on 6 May 2019
You're welcome :-)
Another suggestion: use fullfile to create filenames:
fullfile(dataPath_XPS, '**', 'N*.ibw')

Sign in to comment.

More Answers (1)

Walter Roberson
Walter Roberson on 4 May 2019
Create a template that has all of the fields but no contents, such as
template_struct = struct('filename', [], 'is_loaded', [], 'SNR', []);
If you check with size() and fieldnames() you will see that this will be a struct of size 0 but which has the right fields.
if isempty(test); test = template_struct; end
if isempty(test2); test2 = template_struct; end
if isempty(test3); test3 = template_struct; end
if isempty(test4); test4 = template_struct; end
DataMerged = [test1; test2; test3; test4];

  1 Comment

Walter Roberson
Walter Roberson on 4 May 2019
Or just use:
DataMerged = [];
if ~isempty(test); DataMerged = [DataMerged; test]; end
if ~isempty(test2); DataMerged = [DataMerged; test2]; end

Sign in to comment.




Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!