Cache values from loop (with differing lengths) to save in 1 variable / matrix

I want to calculate the median for time to an event. The 'time to event' is calculated per file in a for loop (so the loop is needed).

Per file there can be multiple 'time to event' values, and the number of values can differ per file.

So, subsequently I want to pool all the time to event values after all files have been processed and to calculate the median for all files.

For example:

file1 = [2;4;6;8]; -> median is 5

file2 = [1;5;10;15]; -> median is 7,5

combined = [2;4;6;8;1;5;10;15]; -> 5,5

for i = 1:length(files)
    time_to_event(i) = dataset.time_to_event; 
end

median_time_to_event = nanmedian(time_to_event) % use ~NaN

it tells me that the indices of the left side are not compatible with the right side

So, I need a way to combine the generated time to events to in the loop (although they may have different lengths) and then calculate the median.

I was thinking maybe add the time of events of the seperate files in seperate colums and fill the difference in row length with NaN?

But I havent been able to.

Hopefully anybody can help, thanks!

Answer 1

[Voss]

To handle vectors of different lengths, you can make time_to_event a cell array:

time_to_event = cell(numel(files),1);
for ii = 1:numel(files)
    % dateset = data from the ii-th file
    time_to_event{ii} = dataset.time_to_event; 
end

Then the median of all elements of all time_to_event vectors can be found like this (assuming each one is a column vector like you show in your examples):

median_time_to_event = nanmedian(vertcat(time_to_event{:}));

And the median of each element of time_to_event (i.e., each time-to-event vector) can be found like this:

median_each = cellfun(@nanmedian,time_to_event)

Here's running a concrete case not based on any data:

time_to_event = cell(4,1);
for ii = 1:4
    time_to_event{ii} = rand(randi(10),1);
end
disp(time_to_event)

    { 2×1 double}
    {10×1 double}
    { 7×1 double}
    { 3×1 double}

median_time_to_event = nanmedian(vertcat(time_to_event{:}))

median_time_to_event = 0.4235

median_each = cellfun(@nanmedian,time_to_event)

median_each = 4×1

0.5675 0.3538 0.3109 0.4629

[SRRellum]

Thanks for your solution. I saw the above answer of Walter Roberson first and found a similar answer to yours. But comparing mine to yours: yours is way cleaner:)

Answer 2

[Walter Roberson]

for i = 1:length(files)
    dataset = load something from files(i)
    time_to_event{i} = dataset.time_to_event(:); 
end

individual_medians = cellfun(@nanmedian, time_to_event);
median_time_to_event = nanmedian(individual_medians);

[SRRellum]

Thanks for you answer.

However, this returned the median of all medians. Which can differ from the median if all values are first taken.

I changed it to the following and works. Maybe not the pretiest solution, but thanks for your help in the proces.

for i = 1:length(files)
    dataset.time_to_event = calculating time from alarm to event in files(i)
    time_to_event{i} = dataset.time_to_event(:);
end

all_time_to_event = cell2mat({vertcat(time_to_event{:})}); % concatenate different cells + transform cells to double structure
median_time_to_event = nanmedian(all_time_to_event,'all');