How do I save a matrix and prevent overwriting from a for loop?

I have a matrix (390000,5) with column1=year, column2=month, column3=day, column4=hour, column5=rainfall. I would like to get the daily mean for each day by averaging from hour=1:24 for the same day. My loop manage to calculate the mean but when the month changes, it overwrites the previous matrix. I would like to have a new matrix with only 4 column (year, month, day, Daily_mean). How do I stack this matrix so it will run from month 1:12, day 1:31? Below is my attempt to solve this. Thanks.

station = 122333;
m = dlmread([num2str(station,'%i') '.txt'],'',1,0); 
mrain = [m(:,1:4) m(:,end)];
for j=1:12 %loop over month
  for i=1:31 %loop over days
    idx = find (mrain(:,1)==1968 & mrain(:,2)==j & mrain(:,3)==i); %find index
    a = mrain(idx,:); 
    meanHr= nanmean(mrain(idx,5)); %find the mean daily rainfall for each day
    meandata(i,:)=[1968 j i meanHr]; %store in a new matrix
  end
end

[Stephen23]

If you simply uploaded your datafile then we could show you how this could be achieved. It is difficult writing code for data that we cannot see. You can use the paperclip button to upload files (do not forget to press both buttons: Choose file and Attach file).

[Sandy Rich]

Hi Stephen,

Thank you for your reply. I had attached a file containing 3 years of hourly data. I wanted to loop through so that I can find the daily mean from year 1 to year 3. Haven't done the year loop yet as the month loop overwrites the previous matrix. The output matrix should look like this form too [year month day rain]. Please do suggest. Thanks!

Answer 1

[dpb]

Don't need any loops --

pavg=mean(reshape(m(:,end),24,[])).'; % arrange by columns of 24-hr days and average
new=[m(1:24:end,[1:3] pavg];          % first hour of day row and augment with average

Write the new array to a file. Caveat--presumes complete days and also I note the first record in your sample datafile has a NaN for the data which is going to propagate thru unless use nanmean or substitute in a value.

The "trick" in the averaging is to remember storage order is column-major in Matlab so that reshape -ing by 24 puts each day's values in a 2D array of the number of days in the file columns. mean by default works down columns so we now have a row vector of 1 by nDays columns which we turn back to a single column by the .' transpose operator. All that's then left is to pick one row from each day for the time data and graft on the averages as the last column.

ADDENDUM

On reflection, it doesn't seem like it makes any sense to average over an individual day's hourly observations...it seems that you would want/need to sum those for the daily totals and then accumulate, perhaps, averages for a given month/day across years...but that's not what you say you want as an end result in length above. I'd change mean above to sum.