Matlab sensing the dynamic data changes and automatically plotting

Hello guys,

This is my plot. In the plot, you can see 6 periods where big dynamic data changes happened. I don't want an overall normal conventional plot.

I want the Matlab to sense these 6 periods and generate 6 plots.

load = 'newdata3.csv';
data = readtable(load);
data = sortrows(data,'Var1','ascend');
timetable(data.Var1, data.Var2);
plot(data.Var1,data.Var2)

j1 = find(diff([0; data.Var2]) > 10);
ss = data(j1,:);
plot(ss.Var1,ss.Var2)

This is the plot I got after using your code. But I want to generate 6 plots. Can you help me?

Answer 1

[Star Strider]

Just the code this time —

data = readtable('https://www.mathworks.com/matlabcentral/answers/uploaded_files/1175778/newdata3.csv');
data.Var1.Format = 'HH:mm:ss.SSS'

data = 126000×2 table
        Var1         Var2  
    ____________    _______

    00:14:54.000    -3.3268
    00:14:54.000    -4.1778
    00:14:54.000    -4.6337
    00:14:54.000     -4.729
    00:14:54.000    -4.9851
    00:14:54.000    -4.2736
    00:14:54.000    -3.2882
    00:14:54.000     -2.257
    00:14:54.000    -1.5046
    00:14:54.000    -1.6617
    00:14:54.000    -1.4103
    00:14:54.000    -2.4553
    00:14:54.000    -3.3668
    00:14:54.000     -4.193
    00:14:54.000    -5.1044
    00:14:54.000     -5.133

TS = sortrows(data,1);
TS.Var1 = TS.Var1 + 0.001*seconds(0:size(TS,1)-1).';

[envh,envl] = envelope(TS.Var2, 130, 'peaks');

% figure
% plot(TS{:,1}, TS{:,2})
% grid
% xlabel('x')
% ylabel('y')
% xline(TS.Var1(1,1)+seconds(0:60:3600))
% xline(TS.Var1(1,1)+seconds(0:300:3600),'-r', 'LineWidth',2)

figure
plot(TS{:,1}, TS{:,2}, 'DisplayName','Data')
hold on
plot(TS{:,1}, envh, '-r', 'DisplayName','Upper Envelope')
plot(TS{:,1}, envl, '-g', 'DisplayName','Lower Envelope')
hold off
grid
xlabel('x')
ylabel('y')
legend('Location','best')

figure
plot(TS{:,1}, TS{:,2}, 'DisplayName','Data')
hold on
plot(TS{:,1}, envh, '-r', 'DisplayName','Upper Envelope')
plot(TS{:,1}, envl, '-g', 'DisplayName','Lower Envelope')
grid
xlabel('x')
ylabel('y')
legend('Location','best')
% xline(TS.Var1(1,1)+seconds(0:60:3600))
% xline(TS.Var1(1,1)+seconds(0:300:3600),'-r', 'LineWidth',2)
xlim([TS.Var1(1,1) TS.Var1(9600,1)])

Lv = envh > 30 & envl < -30;

stidx = strfind([false; Lv].', [0 1])-1;
enidx = strfind([false; Lv].', [1 0]);
s = enidx - stidx;

for k = 1:numel(s)
    ixr = stidx(k):enidx(k);
    T{k} = TS{ixr,1};
    S{k} = TS{ixr,2};
end

figure
hold on
for k = 1:numel(T)
    plot(T{k}, S{k})
end
hold off
xlabel('x')
ylabel('y')

.

[uzzi]

Dear Star Strider, you saved my day. Thank you very much for your answer. I didn't know the envelope function so I'm learning about it. It was very helpful.

I would like to know how did you determine the X limit for the zoomed graph.

xlim([TS.Var1(1,1) TS.Var1(9600,1)])

Can you explain it to me?

[Star Strider]

As always, my pleasure!

That limit was initially from the first figure result (as well as the figure plotted in the commented-out data), then I fine-tuned it to get the interval I wanted. I zoomed it to see the details that are not otherwise visible in the plot of the entire data set, so that I could set the envelope parameters appropriately. (The envelope funciton is quite useful for problems such as this.)

I could not get a combination of the amplitude threshold and time difference logic to work correctly, so I decided to use the envelope function instead.

[uzzi]

Can you explain me how did you decide to go with 130 peaks?

[envh,envl] = envelope(TS.Var2, 130, 'peaks');

[Star Strider]

I could not figure out how to make the minute values of the time aggregate with my previous code, so I developed and entirely new approach, based on thresholding the absolute amplitudes of the signal and thresholding the time (actually index) differences.

data = readtable('https://www.mathworks.com/matlabcentral/answers/uploaded_files/1175778/newdata3.csv');
data.Var1.Format = 'HH:mm:ss';
TS = sortrows(data,1)

TS = 126000×2 table
      Var1        Var2  
    ________    ________

    00:02:07    0.063494
    00:02:07     0.13935
    00:02:07     0.11501
    00:02:07     0.10228
    00:02:07     0.15616
    00:02:07    0.094422
    00:02:07     0.13998
    00:02:07    0.084701
    00:02:07     0.16303
    00:02:07    0.067169
    00:02:07     0.14234
    00:02:07    0.088202
    00:02:07     0.10777
    00:02:07     0.13575
    00:02:07      0.1072
    00:02:07     0.14045

thidx = abs(TS.Var2) >= 20;                         % Absolute Amplitude Threshold
idxn = find(thidx);                                 % Index Vector
didxn = diff([0; idxn]) > 150;                      % Time Difference Threshold
thidxv = [1; idxn(didxn); size(TS,1)];              % Index Vector

for k = 1:numel(thidxv)-1                           % Segmentation Loop
    idxr = thidxv(k) : thidxv(k+1)-1;               % Time Indices
    idxre = idxr(abs(TS{idxr,2}) > 20);             % Absolute Amplitude Threshold
    T{k} = TS{idxre,1};                             % Time Segments
    S{k} = TS{idxre,2};                             % Signal Segments
end


figure
hold on
for k = 1:numel(T)
    plot(T{k}, S{k})
end
hold off
grid
xlabel('x')
ylabel('y')

This also appears to be shorter and more efficient.

.

[uzzi]

This works much better!!

[Star Strider]

Thank you!

[uzzi]

Dear Star Strider,

I am doing this code with another file and I am getting a lot of figures where there is no big data changes occurr. I don't know why. I attached the file here. Can you take a look into it when you are free?

[Star Strider]

These data are not at all like the previous data. I have no idea what to do with them.

Here, the envelope approach would likely be best, although it is going to be difficult as well, because the baselines are not constant between the segments.

The sampling interval is not constant, however resampling it allows digital filtering, and whil that eliminates the wandering baseline, it does not improve the signal itself.

Stopping here with this signal.

Uz = unzip('https://www.mathworks.com/matlabcentral/answers/uploaded_files/1180033/aggff.zip');
T1 = readtable(Uz{1})

T1 = 700880×2 table
      tt_agg         Var1   
    ___________    _________

    {'51:40.9'}     -0.10611
    {'51:40.9'}    -0.035732
    {'51:40.9'}     -0.13328
    {'51:40.9'}    -0.030485
    {'51:40.9'}     -0.10945
    {'51:40.9'}    -0.048296
    {'51:41.0'}     -0.11417
    {'51:41.0'}    -0.021363
    {'51:41.0'}     -0.12879
    {'51:41.0'}    -0.022544
    {'51:41.0'}     -0.12789
    {'51:41.0'}    -0.037168
    {'51:41.0'}     -0.11063
    {'51:41.0'}    -0.036606
    {'51:41.0'}     -0.11115
    {'51:41.0'}    -0.047595

tv = datetime(T1{:,1}, 'InputFormat','mm:ss.S','Format','mm:ss.S');

[tv,sidx] = sort(tv);
dv = T1{sidx,2};





figure
plot(tv, dv)
grid

idx = [373000 : 376000];
figure
plot(tv(idx), dv(idx))
grid
xlim([min(tv(idx)) max(tv(idx))])
title('Zoomed')

idx = [373000 : 376000];
figure
plot(tv(idx), abs(dv(idx)))
grid
xlim([min(tv(idx)) max(tv(idx))])
title('Zoomed Absolute Values')

Ts = mean(diff(tv));
Tsd = std(diff(tv));
ts_min = minutes(Tsd)

ts_min = 0.0030

t_mins = minutes(Ts)

t_mins = 8.5604e-05

% 1/t_mins

Fs = 1E+4;
[dvr,tvr] = resample(dv, tv, Fs);
Fn = Fs/2;
L = numel(dvr)

L = 35999027

NFFT = 2^nextpow2(L)

NFFT = 67108864

FTdvr = fft(dvr-mean(dvr), NFFT)/L;
Fv = linspace(0, 1, NFFT/2+1)*Fn;
Iv = 1:numel(Fv);

figure
semilogy(Fv, abs(FTdvr(Iv))*2)
grid
xlabel('Frequency')
ylabel('Magnitude')

% figure
% semilogy(Fv, abs(FTdvr(Iv))*2)
% grid
% xlim([0 250])
% xlabel('Frequency')
% ylabel('Magnitude')

dvrf = highpass(dvr, 0.1, Fs, 'ImpulseResponse','iir');

figure
plot(tvr, dvrf)
xlabel('Time')
ylabel('Amplitude')

.

[uzzi]

Thank you very much for helping me. I am figuring it out how to understand that data itself. Your final figure looks like very much well sorted out. Thanks again for helping me.

[Star Strider]

As always, my pleasure!

I worked for about an hour to do something with that last file, and could not.

.