Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
sum for two time series dataset???

Subject: sum for two time series dataset???

From: Kuo-Hsien

Date: 11 Jun, 2012 15:44:07

Message: 1 of 7

Dear all,

I like to sum up the observations of two time series datasets when both of them have observations based on YEAR and DOY. I also like to skip the sum when either dataset has 0 and only do the sum for the maximum observation in that DOY.

Thanks for your solutions.
Michael

Here is the example: data_1 and data_2.

% data_1
% YEAR DOY OBS_1
1994 109 0.42
1994 110 0.73
1994 111 0.69
1994 113 0.8
1994 114 0.43
1994 115 0.75
1994 123 0.6
1994 127 0.2
1994 131 0.44
1994 131 0.43
1994 131 0.63
1994 132 0.99
1994 132 0.51
1994 133 0.71
1994 133 0.99
1994 134 0.65
1994 134 0.69
1994 134 0.97
1994 134 0.03
1994 134 0
1994 134 0
1994 135 0.68
1994 135 0.72
1994 136 1.22
1994 136 0
1994 136 0
1994 136 1.28
1994 136 1.34

% data_2
% YEAR DOY OBS_2
1994 110 0.92
1994 111 0.34
1994 113 0.42
1994 114 0.37
1994 115 0.38
1994 122 0.22
1994 127 0.32
1994 131 0.34
1994 131 0.2
1994 132 0.51
1994 132 0.43
1994 132 0.4
1994 133 0.4
1994 134 0.32
1994 134 0.39
1994 135 0.35
1994 135 0.38
1994 135 0.34
1994 135 1.83
1994 135 0.22
1994 135 0.36
1994 135 0.39
1994 135 0.24
1994 135 0.39
1994 136 0.42
1994 136 0.29
1994 136 0.3
1994 136 0.4
1994 136 0.54
1994 136 0.4

Subject: sum for two time series dataset???

From: dpb

Date: 12 Jun, 2012 12:44:15

Message: 2 of 7

On 6/11/2012 10:44 AM, Kuo-Hsien wrote:
> Dear all,
>
> I like to sum up the observations of two time series datasets when both
> of them have observations based on YEAR and DOY. I also like to skip the
> sum when either dataset has 0 and only do the sum for the maximum
> observation in that DOY.
...
> % data_1
> % YEAR DOY OBS_1
> 1994 109 0.42
> 1994 110 0.73
> 1994 111 0.69
> 1994 113 0.8
...
> 1994 135 0.72
> 1994 136 1.22
> 1994 136 0
> 1994 136 0
> 1994 136 1.28
> 1994 136 1.34
>
> % data_2
> % YEAR DOY OBS_2
> 1994 110 0.92
> 1994 111 0.34
> 1994 113 0.42
> 1994 114 0.37
> 1994 115 0.38
...
> 1994 136 0.54
> 1994 136 0.4


[x,ia,ib]=intersect(d1,d2); % DOY 1&2
t=0;
for ix=1:length(ia)
   s1=x1(x1==x1(ia(ix))); % sets that match
   s2=x2(x2==x2(ib(ix)));
   if (all(s1) & all(s2)) % no zeros in either set
     t=t+max(s1,s2);
   end
end

NB--air code; check carefully... :)

One might be able to get fancy w/ bsxfun() (and/or arrayfun() ??).

Bruno???? :)

--

Subject: sum for two time series dataset???

From: Bruno Luong

Date: 12 Jun, 2012 17:34:08

Message: 3 of 7

dpb <none@non.net> wrote in message <jr7div$pc$1@speranza.aioe.org>...
> On 6/11/2012 10:44 AM, Kuo-Hsien wrote:

>
> Bruno???? :)
>

Good to be sollicitated. I'm not quite 100% I understand the task though (the maximum part is confusing to me).

Bruno

Subject: sum for two time series dataset???

From: dpb

Date: 12 Jun, 2012 17:56:00

Message: 4 of 7

On 6/12/2012 12:34 PM, Bruno Luong wrote:
> dpb <none@non.net> wrote in message <jr7div$pc$1@speranza.aioe.org>...
>> On 6/11/2012 10:44 AM, Kuo-Hsien wrote:
>
>>
>> Bruno???? :)
>>
>
> Good to be sollicitated. I'm not quite 100% I understand the task though
> (the maximum part is confusing to me).

I think that would be spelled as solicited in "'Murrican English" anyway...

Not having either bsxfun nor arrayfun I've not had opportunity to learn
by trying and haven't even attempted to do so by simply the paper
exercise from the documentation.

My interpretation/solution takes the maximum of all the data from the
combined set for a given day (excluding any day w/ a null value) and
then sums that value over the days. Now whether that's the actual
intended result only OP can tell us, but that's what I thought was asked
for. :)

--

Subject: sum for two time series dataset???

From: dpb

Date: 12 Jun, 2012 22:12:38

Message: 5 of 7

On 6/12/2012 7:44 AM, dpb wrote:
....

> [x,ia,ib]=intersect(d1,d2); % DOY 1&2
> t=0;
> for ix=1:length(ia)
> s1=x1(x1==x1(ia(ix))); % sets that match
> s2=x2(x2==x2(ib(ix)));
> if (all(s1) & all(s2)) % no zeros in either set
> t=t+max(s1,s2);
> end
> end
>
> NB--air code; check carefully... :)
...

Woops...tested for wrong equality...

for ix=1:length(ia)
   s1=x1(d1==d1(ia(end))); % sets that match
   s2=x2(d2==d2(ib(end)));
   if (all(s1) & all(s2)) % no zeros in either set
     t=t+max(s1,s2);
   end
end

or, slightly more concise but harder to get on the fly... :)

for ix=1:length(ia)
   s=[x1(d1==d1(ia(end))); x2(d2==d2(ib(end)))];
   if all(s)
     t=t+max(s);
   end
end

Note I use x1/2 for the data in the two cases and d1/2 for the DOY. You
can go back to the proper column index in the original arrays; I just
used a simple variable to eliminate the other level of addressing in
writing the logical indexing--to add the (:,1), etc., makes keeping
track of parentheses, etc., tougher when writing air code... :)

--

Subject: sum for two time series dataset???

From: Bruno Luong

Date: 13 Jun, 2012 06:45:07

Message: 6 of 7

I'm still not quite sure what OP wants.

I see no need to process both data array separately. So I group them.

data = [data1; data2];
doy = data(:,1:2);
obs = data(:,3);
obs(obs == 0) = NaN; % mark as not-valid data
[doy_sorted, ~, J] = unique(doy,'rows');
maxobs = accumarray(J, obs, [], @max); % max observation corresponds to doy_sorted
s = sum(~isnan(maxobs)) % sum-only obs with valid doy

You should have accumarray in your MATLAB dpb, it has been around for a while.

% Bruno

Subject: sum for two time series dataset???

From: dpb

Date: 13 Jun, 2012 12:51:10

Message: 7 of 7

On 6/13/2012 1:45 AM, Bruno Luong wrote:
> I'm still not quite sure what OP wants.
>
> I see no need to process both data array separately. So I group them.
>
> data = [data1; data2];
> doy = data(:,1:2);
> obs = data(:,3);
> obs(obs == 0) = NaN; % mark as not-valid data
> [doy_sorted, ~, J] = unique(doy,'rows');
> maxobs = accumarray(J, obs, [], @max); % max observation corresponds to
> doy_sorted
> s = sum(~isnan(maxobs)) % sum-only obs with valid doy
>
> You should have accumarray in your MATLAB dpb, it has been around for a
> while.

Yeah, there's no real need to wait until the selection to group.

 >> help accumarray

accumarray.m not found.

 >> accumarray
??? Undefined function or variable 'accumarray'.

 >>

Probably in R14; not in R12 which is when I updated when went out on own
and subsequently retired so haven't had the resources coming in to
justify updating since.

--

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us