How to process a table with time, x,y,z coordinate, particle mass? dividing the table based on the time value, then process them piece by piece and sum it up.

I tried to extract the mass out for each time interval, like 36.138-36.148s.

it works.

First: this code seems clumsy. is there any better way to do the same thing?

Next step: I would like to show an animation of particle mass. I need to check first whether there are particles appearing at the same locations (same y, z coordinates). I didn't figure that out.

Do I need to divide this table into table sets based on time points? kind of making a database, which I am not familiar with? I have found some materials dealing with only 2 variables ie mass vs time; what I would like to show is particle parcel Mass vs ( y, z) changing with time, after summing particle parcel mass up at same (y,z).

I have done part of the work by using this code:

Could you please give me some suggestions to deal with the data (8GB) more efficiently, and manipute it more conveniently?

----------------

x_5cm = readtable("D:\SPOC\tulip-shape-burner-combustor-v2\particle-tracking\particle_number\dpm_output_2s\x=0.05m.dpm", opts);

B_5cm = table2array(x_5cm);

C_5cm = sortrows(B_5cm,13);

%sample(C_5cm(:,2))

%Y_5cm = sortrows(B_5cm,2);

%histogram(C_5cm(:,13));

[N_5cm,edges_5cm] = histcounts(C_5cm(:,13));

figure(1)

t = 0:pi/500:2*pi;

xt1 = 0.0275.*sin(t);

yt1 = 0.0275.*cos(t);

zt1 = 0.*(sin(t).*sin(t)+cos(t).^2);

stem3(C_5cm(1:22880,2), C_5cm(1:22880,3), C_5cm(1:22880,9));

grid on

hold on;

plot3(xt1, yt1, zt1);

xlabel('Y','FontSize',18,'FontName','times New Roman');

ylabel('Z','FontSize',18,'FontName','times New Roman');

zlabel('particle mass','FontSize',18,'FontName','times New Roman');

----------------------------------

This code is to get the total particle parcel mass at certain time.

x_5cm = readtable("x=0.05m.dpm", opts);

B_5cm = table2array(x_5cm);

C_5cm = sortrows(B_5cm,13);

histogram(C_5cm(:,13))

[N_5cm,edges_5cm] = histcounts(C_5cm(:,13));

%nbins_5cm = round((C_5cm(end,13)-C_5cm(1,13))/0.001+1);

%histogram(C_5cm(:,13),nbins_5cm)

%[N_5cm,edges_5cm] = histcounts(C_5cm(:,13),nbins_5cm);

j=1;

for i = 1:length(N_5cm)-1

if i==1

P_5cm_M(j,1) = edges_5cm(i);

P_5cm_M(j,2) = sum(C_5cm(1:N_5cm(i),9));

else

j=j+1;

P_5cm_M(j,1) = edges_5cm(i);

P_5cm_M(j,2) = sum(C_5cm(sum(N_5cm(1:i-1)):sum(N_5cm(1:i)),9));

end

end

Answer 1

[Siraj]

Hi!

It is my understanding that you have a table on which you want to perform the following:

1. Extract unique “flow_time” values from the table.
2. For each unique “flow_time” value, find and isolate the subset of rows in the table that have that specific “flow_time”.
3. Within each group (created based on the same “flow_time”), identify rows that have same “y” and “z” coordinates. For these rows, sum up the “parcel_mass” values to calculate the total mass for each unique set of coordinates at that specific “flow_time”.
4. Create a separate scatter plot for each unique “flow_time” value. On these plots, display the cumulative “parcel_mass” on the vertical axis against the “y” and “z” coordinates on the horizontal axes.

This can be achieved by leveraging the different ways of indexing a table. Refer to the following link to see a summary of table indexing syntaxes.

https://www.mathworks.com/help/matlab/matlab_prog/access-data-in-a-table.html

To help clarify the steps needed to accomplish the task, I've prepared a simple example. This example demonstrates a straightforward method for achieving the goal without switching between tables and matrices or sorting the data. You can refer to the provided code to understand how to efficiently process the table.

%% Creating a dummy table 
% Define the number of rows for the table
numRows = 10;

% Create column vectors with the specified characteristics
X = repmat(0.05, numRows, 1); % X column with constant value 0.05
y = rand(numRows, 1) * 0.01; % y column with random double values around 0.0053
z = rand(numRows, 1) * 0.01; % z column with random double values similar to y
parcel_mass = rand(numRows, 1) * 1e-10 + 1e-11; % parcel_mass with values like 2.6202e-11

% Define four unique flow_time values
unique_flow_times = rand(4, 1) * 100; % Generate four unique flow_time values

% Assign these unique values to the flow_time column, ensuring at least two rows share the same value
flow_time = repmat(unique_flow_times(1), numRows, 1); % Start with all rows having the same flow_time
flow_time(3:4) = unique_flow_times(2); % Assign the second unique value to the 3rd and 4th rows
flow_time(5:6) = unique_flow_times(3); % Assign the third unique value to the 5th and 6th rows
flow_time(7:end) = unique_flow_times(4); % Assign the fourth unique value to the remaining rows

% Manually assign identical values to y and z for some rows to ensure repetition
% For example, set rows 1 and 2 to have the same y and z values
duplicate_y = rand(1) * 0.01;
duplicate_z = rand(1) * 0.01;
y(1:2) = duplicate_y;
z(1:2) = duplicate_z;

% For additional repetition, set rows from 8 to have another set of identical y and z values
y(8:end) = duplicate_y + 0.001; % Slightly different to ensure it's a unique pair
z(8:end) = duplicate_z + 0.001;

% Create the table
dataTable = table(X, y, z, parcel_mass, flow_time);

% Display the table
disp(dataTable);

     X          y            z        parcel_mass    flow_time
    ____    _________    _________    ___________    _________

    0.05    0.0056177    0.0064576    8.0218e-11      1.6603  
    0.05    0.0056177    0.0064576    7.2133e-11      1.6603  
    0.05    0.0024865    0.0020868    3.8539e-11      25.473  
    0.05    0.0048963    0.0023159    1.0374e-11      25.473  
    0.05    0.0082402    0.0039981    4.5019e-11      25.206  
    0.05    0.0029215    0.0064237    3.2815e-11      25.206  
    0.05    0.0077417    0.0014296    1.9603e-11      88.868  
    0.05    0.0066177    0.0074576    4.1622e-11      88.868  
    0.05    0.0066177    0.0074576    1.0733e-10      88.868  
    0.05    0.0066177    0.0074576    6.6131e-11      88.868  

%% Algorithm 
% Get the unique flow_time values
uniqueFlowTimes = unique(dataTable.flow_time);

% Initialize a cell array to store the cumulative parcel_mass for each unique flow_time
cumulativeParcelMass = cell(length(uniqueFlowTimes), 1);

% Loop through each unique flow_time to process the data
for i = 1:length(uniqueFlowTimes)
    % Find the rows that have the current unique flow_time
    flowTimeRows = dataTable(dataTable.flow_time == uniqueFlowTimes(i), :);
    
    % Group rows by 'y' and 'z' and sum the 'parcel_mass' for each group
    sumTable = varfun(@sum, flowTimeRows, 'InputVariables', 'parcel_mass', ...
        'GroupingVariables', {'y', 'z'});
    
    % Store the results in the cell array
    cumulativeParcelMass{i} = sumTable;
end

% Plotting the graph for each unique flow_time
for i = 1:length(uniqueFlowTimes)
    % Extract the data for the current unique flow_time
    flowTimeData = cumulativeParcelMass{i};
    
    if(isempty(flowTimeData))
        continue
    end
    
    % Create a 3D scatter plot for cumulative parcel mass vs (y, z)
    figure; % Create a new figure for each unique flow_time
    scatter3(flowTimeData.y, flowTimeData.z, flowTimeData.sum_parcel_mass, 'filled');
    title(sprintf('Cumulative Parcel Mass vs. (y,z) for Flow Time = %.2f', uniqueFlowTimes(i)));
    xlabel('y');
    ylabel('z');
    zlabel('Cumulative Parcel Mass');
    grid on;
end

In the above code, first I have created a dummy table and then performed the following:

1. Extracted all unique “flow_time” values and initialized a cell array to store cumulative “parcel_mass” data.
2. Processed the data by finding rows with matching “flow_time”, grouping them by “y” and “z”, summing “parcel_mass” for each group, and storing the results.
3. Plotted a 3D scatter plot for each unique “flow_time”, showing cumulative “parcel_mass” versus (“y”, “z”) coordinates, and labelled each plot with the corresponding “flow_time” value.

Refer to the following links to learn about "cell array" and "varfun" in MATLAB.

https://www.mathworks.com/help/matlab/ref/cell.html

https://www.mathworks.com/help/matlab/ref/table.varfun.html

Hope this helps.

[LEI Li]

Thank you for your help. I will test this method later.

[LEI Li]

sumTable = varfun(@sum, flowTimeRows, 'InputVariables', 'parcel_mass', ...

'GroupingVariables', {'y', 'z'});

this code is brilliant. Thank you, Siraj.

[LEI Li]

is it possible to make a video by combining the figures at different time point?

[Siraj]

Yes you can make a video by combining different figures. To achieve this you can use the "VideoWriter" function. Please refer to the following link to learn more about it.

https://www.mathworks.com/help/matlab/ref/videowriter.html

For a detailed example you can refer to the following MATLAB Answer:

https://www.mathworks.com/matlabcentral/answers/384848-get-figures-and-use-them-to-build-a-video-avi