'For ' loop for every day for one minut fequency sea level data

1 view (last 30 days)
(01). I have 12 years of 1 min frequency sea level data (six million rows). I need to save min and max within a every day (60*24) as a different text files using all 12 years. (02). Same thing within 14 days to save min and max values as different files in 12 years (60*24*14). Drafted code here with. It is working but mean value for min and max shows wrong. The each min and max files only consisted 30 values, but it have 1440 rows. 07/01/2018 00:00,1.027 07/01/2018 00:01,1.034 07/01/2018 00:02,1.047 07/01/2018 00:03,1.026 07/01/2018 00:04,1.031 07/01/2018 00:05,1.057

Accepted Answer

Are Mjaavatten
Are Mjaavatten on 9 Sep 2018
Your problem seems to be the selection of data to calculate min and max values. The code below should help you on your way. Consider also reading in the timestamps and using datetime to get flexible tick labels the x axis.
load july2018.txt
x=july2018(:,3);
timespan = length (x); %43201
day = 24*60;
time = (1:timespan)'/day;
noon = (12*60 + day*(0:29))/day;
MAXT=[]; MINT=[];
for jj=1:day:timespan-day;
t=x(jj+(0:day-1));
maxt=max(t);
mint=min(t);
MAXT=[MAXT;maxt];
MINT=[MINT;mint];
end
plot(time,x,noon,MAXT,'o',noon,MINT,'*');
xlabel('days')
legend('Level','Daily max','Daily min')
  3 Comments
Are Mjaavatten
Are Mjaavatten on 10 Sep 2018
Edited: Are Mjaavatten on 10 Sep 2018
I now realize that my original answer has some shortcomings: There are some missing data in your file, and this must be taken into account by reading the time stamps as well as the x data. Akira Agata does this very elegantly in his answer. However, my old Matlab version (2014b) lacks the table2timetable and retime commands, so I have used a simple workaround.
I also now plot stair plots of MAXT and MINT, and write those data to file.
The modified code looks like this:
T = readtable('july2018.txt', 'Format', '%{MM/dd/yyyy HH:mm}D%f');
t = table2array(T(:,1));
x = table2array(T(:,2));
% Add data for midnight on day 1:
t = [datetime('2018-07-01 00:00');t];
x = [NaN;x];
midnight = find(t.Minute == 0 & t.Hour == 0); % Index of midnight (00:00)
ndays = length(midnight)-1;
MAXT=zeros(ndays,1); MINT=zeros(ndays,1);
for i = 1:ndays
MAXT(i)= max(x(midnight(i):midnight(i+1)));
MINT(i)= min(x(midnight(i):midnight(i+1)));
end
plot(t,x,'b') % Full time series
% To plot MAXT and MINT as step functions, I would normally user "stairs"
% but this seems not to work with datetime, so I must do it the hard way:
plot_midnight = [midnight(2:end-1),midnight(2:end-1)]';
plot_midnight = [midnight(1);plot_midnight(:);midnight(end)];
plot_max = [MAXT(1:end),MAXT(1:end)]';
plot_max = plot_max(:);
plot_min = [MINT(1:end),MINT(1:end)]';
plot_min = plot_min(:);
hold on
plot(t(plot_midnight),plot_max,'r',t(plot_midnight),plot_min,'k');
hold off
legend('Level','Daily max','Daily min')
% Now write to file:
days = t(midnight(1:end-1));
Year = days.Year;
Month = days.Month;
Day = days.Day;
Daily_max = MAXT;
Daily_min = MINT;
maxmin = table(Year,Month,Day,Daily_max,Daily_min);
writetable(maxmin,'Daily_max_min.txt')
indika kathaluwa weligamage
Thank you it is working very useful for me. I have to find mean min and max during 10 years of time series.

Sign in to comment.

More Answers (1)

Akira Agata
Akira Agata on 9 Sep 2018
Edited: Akira Agata on 9 Sep 2018
I would recommend storing your data in timetable variable, and apply retime function to do your task. Following is an example.
% Read data file and convert to timetable
T = readtable('july2018.txt', 'Format', '%{MM/dd/yyyy HH:mm}D%f');
T.Properties.VariableNames = {'time','value'};
TT = table2timetable(T);
% min and max within a day
Tmin_1d = retime(TT,'daily','min');
Tmax_1d = retime(TT,'daily','max');
% Same thing within 14 days to save min and max
t1 = TT.time(1):days(14):TT.time(end);
Tmin_14d = retime(TT,t1,'min');
Tmax_14d = retime(TT,t1,'max');
  3 Comments
Akira Agata
Akira Agata on 12 Sep 2018
That's a relatively easy part. MATLAB has writetable function for this task. To save the result in file, please try following.
Tmin_1d.time.Format = 'MM/dd/yyyy';
writetable(timetable2table(Tmin_1d),'Tmin_1d.csv');
And to plot the result, you can use plot function, such as:
figure
plot(Tmin_1d.time,Tmin_1d.value,'o-')
indika kathaluwa weligamage

THANK YOU. I SOLVE MY PROBLEM USING YOU AND MIJAAATTTAN 'S ANSWER'S . THANK YOU SO MUCH. YOU HAVE IDEA FINDING OF TIDE RANGERS. HEGEST SPRIN,NEAP, LOWESTSPRING NEAP, SATRONOMICAL MEAN HEIGHST AND LOWEST. IS THERE ANY PROGARAM.

Sign in to comment.

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!