Tall arrays are not allowed to contain data of type struct

I am having trouble with tall arrays in the following workflow,

• Process data in a loop. In each iteration, generate structure containing results and save structure in a *.mat file
• Read all *.mat files into a datastore
• Turn datastore into a tall array. Gives the error "Tall arrays are not allowed to contain data of type struct"
• Turn tall array into a timetable

Here's an example, which I generated by modifying Edric Ellis's answer here. Can you help me eliminate the error?

%% Setup
nRowLimit = 100; % Number of records in each file
nCalculation = 20; % Number of calculation iterations and also mat files
tStart = datetime(2013, 11, 1, 8, 0, 0); % For later use in generating datetime array

%% Prepare to save matlab files
iFileToSave = 0;
dirSave = 'c:\temp';

%%
for iCalc = 1:nCalculation % Loop through iterations of the calculation
    clear StrucResults; % Structure to hold results of calculations
    
    % Generate structure containing pretend calculation results
    for iRow = 1:nRowLimit
        % Generate example datetime to fill a datetime array
        StrucResults.date(iRow) = tStart + 1000*iFileToSave + iRow;
        % Generate example character to fill a cell array
        StrucResults.label{iRow} = char(round(rand(1)*100));
        % Generate example value to fill a numerical matrix
        StrucResults.white(iRow, :) = rand(1, 5);
    end
    
    %% Save structure containing results to small mat file 
    iFileToSave = iFileToSave + 1;
    % Make a distinct file name
    fileSave = fullfile(dirSave, sprintf('FileContainingStruct_%05d.mat', iFileToSave));
    save(fileSave, 'StrucResults');
end

%% Read the data back in as a tall array
% First create a datastore from all the mat files
fds = fileDatastore(fullfile(dirSave, '*.mat'), ... % Files to load
    'ReadFcn', @(x)struct2table(load(x)), ... % Try to convert structure to table to avoid error in next code line
    'UniformRead', true);
% Read into a tall array
tallData = tall(fds); % Throws error "Tall arrays are not allowed to contain data of type struct"
% Change to a tall timetable
tallTimeTable = table2timetable(tallData);

It's fine to eliminate the structure in favor of storing the results in individual variables, but I couldn't get that to work either.

Answer 1

[Edric Ellis]

Your code was so nearly working. I made two tiny changes, marked with %###:

%% Setup
nRowLimit = 100; % Number of records in each file
nCalculation = 20; % Number of calculation iterations and also mat files
tStart = datetime(2013, 11, 1, 8, 0, 0); % For later use in generating datetime array

%% Prepare to save matlab files
iFileToSave = 0;
dirSave = fullfile(tempdir(), '809910');
if ~exist(dirSave, 'dir')
    mkdir(dirSave);
end

%%
for iCalc = 1:nCalculation % Loop through iterations of the calculation
    clear StrucResults; % Structure to hold results of calculations
    
    % Generate structure containing pretend calculation results
    %### Ensure vector fields are columns
    for iRow = 1:nRowLimit
        % Generate example datetime to fill a datetime array
        StrucResults.date(iRow, 1) = tStart + 1000*iFileToSave + iRow;
        % Generate example character to fill a cell array
        StrucResults.label{iRow, 1} = char(round(rand(1)*100));
        % Generate example value to fill a numerical matrix
        StrucResults.white(iRow, :) = rand(1, 5);
    end
    
    %% Save structure containing results to small mat file 
    iFileToSave = iFileToSave + 1;
    % Make a distinct file name
    fileSave = fullfile(dirSave, sprintf('FileContainingStruct_%05d.mat', iFileToSave));
    %### Use "save -struct" to ensure the fields of StrucResults are saved into the file directly.
    save(fileSave, '-struct', 'StrucResults');
end

%% Read the data back in as a tall array
% First create a datastore from all the mat files
fds = fileDatastore(fullfile(dirSave, '*.mat'), ... % Files to load
    'ReadFcn', @(x)struct2table(load(x)), ... % Try to convert structure to table to avoid error in next code line
    'UniformRead', true);
% Read into a tall array
tallData = tall(fds); % Throws error "Tall arrays are not allowed to contain data of type struct"
% Change to a tall timetable
tallTimeTable = table2timetable(tallData);

% Check the results
head(tallTimeTable)

ans =

  8×2 tall timetable

            date            label                             white                          
    ____________________    _____    ________________________________________________________

    02-Nov-2013 08:00:00    {''}     0.24734     0.82558     0.41452      0.3178     0.41834
    03-Nov-2013 08:00:00    {'B'}     0.49954     0.61133     0.76577     0.67823     0.50137
    04-Nov-2013 08:00:00    {''}    0.099728     0.28918     0.42864     0.62252     0.61956
    05-Nov-2013 08:00:00    {'H'}     0.67335     0.32173     0.25268    0.057868     0.17846
    06-Nov-2013 08:00:00    {''}     0.82981    0.059187     0.34086     0.31046     0.25538
    07-Nov-2013 08:00:00    {'@'}     0.53214     0.78883     0.30166     0.77258     0.68763
    08-Nov-2013 08:00:00    {'Y'}     0.37458     0.19037     0.75301     0.62862     0.85332
    09-Nov-2013 08:00:00    {'8'}     0.27491     0.22561     0.12215     0.46316     0.66067

The first change was simply to ensure that the vectors in the struct were columns.

The second more important change was to use save -struct to get rid of the extra layer of "struct" in the MAT file.

[KAE]

This help is so appreciated!