Thread Subject: Data Subsampling

Hi
I have a relatively simple problem that I can't figure out an elegant way to solve.

I have a data set consisting of:

Column 1: a series of days numbered 1 to 365 or 366 for each year from 1974 until now
Column 2: average daily temperature on each day

What I would like to do is to produce a matrix of years (x), days (y) and temperature (z). So far I have been able to do this for Day 1 for all years using a combination of row and column indices but I am unsure of how to this for all days across all years. Any help would be greatly appreciated.

Thanks.

"Billy Bob" <jforwin@gmail.com> wrote in message <ich2vm$mbl$1@fred.mathworks.com>...
> Hi
> I have a relatively simple problem that I can't figure out an elegant way to solve.
>
> I have a data set consisting of:
>
> Column 1: a series of days numbered 1 to 365 or 366 for each year from 1974 until now
> Column 2: average daily temperature on each day
>
> What I would like to do is to produce a matrix of years (x), days (y) and temperature (z). So far I have been able to do this for Day 1 for all years using a combination of row and column indices but I am unsure of how to this for all days across all years. Any help would be greatly appreciated.
>
> Thanks.

It sounds like ACCUMARRAY might be what you want. Feed it your days/years as subs, [days years], and the temperatures as vals, [T]. It'll then place all of the T's in their respective positions.

"Billy Bob" <jforwin@gmail.com> wrote in message <ich2vm$mbl$1@fred.mathworks.com>...
> Hi
> I have a relatively simple problem that I can't figure out an elegant way to solve.
>
> I have a data set consisting of:
>
> Column 1: a series of days numbered 1 to 365 or 366 for each year from 1974 until now
> Column 2: average daily temperature on each day
>
> What I would like to do is to produce a matrix of years (x), days (y) and temperature (z). So far I have been able to do this for Day 1 for all years using a combination of row and column indices but I am unsure of how to this for all days across all years. Any help would be greatly appreciated.
>
> Thanks.
- - - - - - - - - - -
 Let A be your present data set array.

 B = [cumsum([false;diff(A(:,1))<0])+1974,A];

Roger Stafford

"Roger Stafford" <ellieandrogerxyzzy@mindspring.com.invalid> wrote in message <ich5cg$22r$1@fred.mathworks.com>...
> "Billy Bob" <jforwin@gmail.com> wrote in message <ich2vm$mbl$1@fred.mathworks.com>...
> > Hi
> > I have a relatively simple problem that I can't figure out an elegant way to solve.
> >
> > I have a data set consisting of:
> >
> > Column 1: a series of days numbered 1 to 365 or 366 for each year from 1974 until now
> > Column 2: average daily temperature on each day
> >
> > What I would like to do is to produce a matrix of years (x), days (y) and temperature (z). So far I have been able to do this for Day 1 for all years using a combination of row and column indices but I am unsure of how to this for all days across all years. Any help would be greatly appreciated.
> >
> > Thanks.
> - - - - - - - - - - -
> Let A be your present data set array.
>
> B = [cumsum([false;diff(A(:,1))<0])+1974,A];
>
> Roger Stafford

Thanks for this Roger and Sean
So what I have now is each day correctly labeled with the year (Columns 1 (Year) and 2 (Day) ) and temperature in Column 3.

What I would like to do is separate out each year with it's associated temperature data for the 37 year data record, I guess using the year as marker (to take into account Leap Years) ? Any advice on how to do this?

Thanks for the help on what are undoubtedly pretty silly questions!

"Billy Bob" <jforwin@gmail.com> wrote in message <ich6cc$6j9$1@fred.mathworks.com>...
> "Roger Stafford" <ellieandrogerxyzzy@mindspring.com.invalid> wrote in message <ich5cg$22r$1@fred.mathworks.com>...
> > "Billy Bob" <jforwin@gmail.com> wrote in message <ich2vm$mbl$1@fred.mathworks.com>...
> > > Hi
> > > I have a relatively simple problem that I can't figure out an elegant way to solve.
> > >
> > > I have a data set consisting of:
> > >
> > > Column 1: a series of days numbered 1 to 365 or 366 for each year from 1974 until now
> > > Column 2: average daily temperature on each day
> > >
> > > What I would like to do is to produce a matrix of years (x), days (y) and temperature (z). So far I have been able to do this for Day 1 for all years using a combination of row and column indices but I am unsure of how to this for all days across all years. Any help would be greatly appreciated.
> > >
> > > Thanks.
> > - - - - - - - - - - -
> > Let A be your present data set array.
> >
> > B = [cumsum([false;diff(A(:,1))<0])+1974,A];
> >
> > Roger Stafford
>
> Thanks for this Roger and Sean
> So what I have now is each day correctly labeled with the year (Columns 1 (Year) and 2 (Day) ) and temperature in Column 3.
>
> What I would like to do is separate out each year with it's associated temperature data for the 37 year data record, I guess using the year as marker (to take into account Leap Years) ? Any advice on how to do this?
>
> Thanks for the help on what are undoubtedly pretty silly questions!

data(:,1) = data(:,1)-min(data(:,1))+1; %normalize so 1974 is 1

new_data_matrix = accumarray(data(:,[1 2]),data(:,3));

each row of the new matrix will be the year (1:37) and the columns will be the 366 days.

You could also do:
new_data_matrix = zeros(37,366);
new_data_matrix(sub2ind([37 366],data(:,1),data(:,2))) = data(:,3);

"Sean de " <sean.dewolski@nospamplease.umit.maine.edu> wrote in message <ich6v4$flh$1@fred.mathworks.com>...
> "Billy Bob" <jforwin@gmail.com> wrote in message <ich6cc$6j9$1@fred.mathworks.com>...
> > "Roger Stafford" <ellieandrogerxyzzy@mindspring.com.invalid> wrote in message <ich5cg$22r$1@fred.mathworks.com>...
> > > "Billy Bob" <jforwin@gmail.com> wrote in message <ich2vm$mbl$1@fred.mathworks.com>...
> > > > Hi
> > > > I have a relatively simple problem that I can't figure out an elegant way to solve.
> > > >
> > > > I have a data set consisting of:
> > > >
> > > > Column 1: a series of days numbered 1 to 365 or 366 for each year from 1974 until now
> > > > Column 2: average daily temperature on each day
> > > >
> > > > What I would like to do is to produce a matrix of years (x), days (y) and temperature (z). So far I have been able to do this for Day 1 for all years using a combination of row and column indices but I am unsure of how to this for all days across all years. Any help would be greatly appreciated.
> > > >
> > > > Thanks.
> > > - - - - - - - - - - -
> > > Let A be your present data set array.
> > >
> > > B = [cumsum([false;diff(A(:,1))<0])+1974,A];
> > >
> > > Roger Stafford
> >
> > Thanks for this Roger and Sean
> > So what I have now is each day correctly labeled with the year (Columns 1 (Year) and 2 (Day) ) and temperature in Column 3.
> >
> > What I would like to do is separate out each year with it's associated temperature data for the 37 year data record, I guess using the year as marker (to take into account Leap Years) ? Any advice on how to do this?
> >
> > Thanks for the help on what are undoubtedly pretty silly questions!
>
> data(:,1) = data(:,1)-min(data(:,1))+1; %normalize so 1974 is 1
>
> new_data_matrix = accumarray(data(:,[1 2]),data(:,3));
>
> each row of the new matrix will be the year (1:37) and the columns will be the 366 days.
>
> You could also do:
> new_data_matrix = zeros(37,366);
> new_data_matrix(sub2ind([37 366],data(:,1),data(:,2))) = data(:,3);

Thanks Sean - what I get with the first approach is a matrix of zeros from Row 0 to 1973 by 366 columns and then the data appears. Is there a way to get Matlab to ignore Rows 0 to 1973? Thanks for your help.

> > data(:,1) = data(:,1)-min(data(:,1))+1; %normalize so 1974 is 1
> >
> > new_data_matrix = accumarray(data(:,[1 2]),data(:,3));
> >
> > each row of the new matrix will be the year (1:37) and the columns will be the 366 days.
> >
> > You could also do:
> > new_data_matrix = zeros(37,366);
> > new_data_matrix(sub2ind([37 366],data(:,1),data(:,2))) = data(:,3);
>
> Thanks Sean - what I get with the first approach is a matrix of zeros from Row 0 to 1973 by 366 columns and then the data appears. Is there a way to get Matlab to ignore Rows 0 to 1973? Thanks for your help.

That was the point of this line. It makes the year 1974 = 1, thus making it the first row.

data(:,1) = data(:,1)-min(data(:,1))+1; %normalize so 1974 is 1

"Sean de " <sean.dewolski@nospamplease.umit.maine.edu> wrote in message <ich8p9$fgj$1@fred.mathworks.com>...
>
> > > data(:,1) = data(:,1)-min(data(:,1))+1; %normalize so 1974 is 1
> > >
> > > new_data_matrix = accumarray(data(:,[1 2]),data(:,3));
> > >
> > > each row of the new matrix will be the year (1:37) and the columns will be the 366 days.
> > >
> > > You could also do:
> > > new_data_matrix = zeros(37,366);
> > > new_data_matrix(sub2ind([37 366],data(:,1),data(:,2))) = data(:,3);
> >
> > Thanks Sean - what I get with the first approach is a matrix of zeros from Row 0 to 1973 by 366 columns and then the data appears. Is there a way to get Matlab to ignore Rows 0 to 1973? Thanks for your help.
>
> That was the point of this line. It makes the year 1974 = 1, thus making it the first row.
>
> data(:,1) = data(:,1)-min(data(:,1))+1; %normalize so 1974 is 1


That's great. Thanks for your help Sean.