Thread Subject: effective way to extract data according to commen dates?

Dear all,

I have a big cell array DATA with size 1*3000. The longest vector (single cell) has 5042*1 elements with their corresponding dates which are stored in another cell array with the exact same size. ( 1*3000 for the cell array and 5042*1 for each cell)

At each date, I need to for a combination including 3 random selected values from the 3000 cells. And keep doing this process 5042 times since I have 5042 dates at most.

my problem is, for each cell the starting and ending time is different, how could I do this selection in an effective way?

eg:

dates =

  Columns 1 through 6

    [3674x1 double] [2319x1 double] [5042x1 double] [5042x1 double] [1929x1 double] [4099x1 double]
................ till columns 3000

DATA =

  Columns 1 through 6

    [3674x1 double] [2319x1 double] [5042x1 double] [5042x1 double] [1929x1 double] [4099x1 double]
................ till columns 3000

many thanks in advance

I am wondering to create the time series (TS) for each date and data cell, then find the intersections for each time series, i.e. for one TS I will have 2999 other intersections at most.

Then I start the random selection process among the 2999*3000 intersections....
Huge computation time... I am afraid is there an more effective way to do that?

Thanks in advance

"Skirt Zhang" <silence_qunzi@hotmail.com> wrote in message <j067rq$bo3$1@newscl01ah.mathworks.com>...
> Dear all,
>
> I have a big cell array DATA with size 1*3000. The longest vector (single cell) has 5042*1 elements with their corresponding dates which are stored in another cell array with the exact same size. ( 1*3000 for the cell array and 5042*1 for each cell)
>
> At each date, I need to for a combination including 3 random selected values from the 3000 cells. And keep doing this process 5042 times since I have 5042 dates at most.
>
> my problem is, for each cell the starting and ending time is different, how could I do this selection in an effective way?
>
> eg:
>
> dates =
>
> Columns 1 through 6
>
> [3674x1 double] [2319x1 double] [5042x1 double] [5042x1 double] [1929x1 double] [4099x1 double]
> ................ till columns 3000
>
> DATA =
>
> Columns 1 through 6
>
> [3674x1 double] [2319x1 double] [5042x1 double] [5042x1 double] [1929x1 double] [4099x1 double]
> ................ till columns 3000
>
> many thanks in advance

On Jul 20, 9:53 pm, "Skirt Zhang" <silence_qu...@hotmail.com> wrote:
> I am wondering to create the time series (TS) for each date and data cell, then find the intersections for each time series, i.e. for one TS I will have 2999 other intersections at most.
>
> Then I start the random selection process among the 2999*3000 intersections....
> Huge computation time... I am afraid is there an more effective way to do that?
>
> Thanks in advance
>
>
>
>
>
>
>
> "Skirt Zhang" <silence_qu...@hotmail.com> wrote in message <j067rq$bo...@newscl01ah.mathworks.com>...
> > Dear all,
>
> > I have a big cell array DATA with size 1*3000. The longest vector (single cell) has 5042*1 elements with their corresponding dates which are stored in another cell array with the exact same size. ( 1*3000 for the cell array and 5042*1 for each cell)
>
> > At each date, I need to for a combination including 3 random selected values from the 3000 cells. And keep doing this process 5042 times since I have 5042 dates at most.
>
> > my problem is, for each cell the starting and ending time is different, how could I do this selection in an effective way?
>
> > eg:
>
> > dates =
>
> >   Columns 1 through 6
>
> >     [3674x1 double]    [2319x1 double]    [5042x1 double]    [5042x1 double]    [1929x1 double]    [4099x1 double]
> > ................ till columns 3000
>
> > DATA =
>
> >   Columns 1 through 6
>
> >     [3674x1 double]    [2319x1 double]    [5042x1 double]    [5042x1 double]    [1929x1 double]    [4099x1 double]
> > ................ till columns 3000
>
> > many thanks in advance

This seems to be the most important sentence:
"At each date, I need to for a combination including 3 random selected
values from the 3000 cells. "
but it's nonsense. What does "to for a combination" mean?
You need to slow down and explain yourself better.

Also, don't top post. It makes the thread hard to follow.
Put replies UNDERNEATH.

TideMan <mulgor@gmail.com> wrote in message <abc255be-3181-4c00-ae04-a5db3c164cc4@gv8g2000vbb.googlegroups.com>...
> On Jul 20, 9:53 pm, "Skirt Zhang" <silence_qu...@hotmail.com> wrote:
> > I am wondering to create the time series (TS) for each date and data cell, then find the intersections for each time series, i.e. for one TS I will have 2999 other intersections at most.
> >
> > Then I start the random selection process among the 2999*3000 intersections....
> > Huge computation time... I am afraid is there an more effective way to do that?
> >
> > Thanks in advance
> >
> >
> >
> >
> >
> >
> >
> > "Skirt Zhang" <silence_qu...@hotmail.com> wrote in message <j067rq$bo...@newscl01ah.mathworks.com>...
> > > Dear all,
> >
> > > I have a big cell array DATA with size 1*3000. The longest vector (single cell) has 5042*1 elements with their corresponding dates which are stored in another cell array with the exact same size. ( 1*3000 for the cell array and 5042*1 for each cell)
> >
> > > At each date, I need to for a combination including 3 random selected values from the 3000 cells. And keep doing this process 5042 times since I have 5042 dates at most.
> >
> > > my problem is, for each cell the starting and ending time is different, how could I do this selection in an effective way?
> >
> > > eg:
> >
> > > dates =
> >
> > >   Columns 1 through 6
> >
> > >     [3674x1 double]    [2319x1 double]    [5042x1 double]    [5042x1 double]    [1929x1 double]    [4099x1 double]
> > > ................ till columns 3000
> >
> > > DATA =
> >
> > >   Columns 1 through 6
> >
> > >     [3674x1 double]    [2319x1 double]    [5042x1 double]    [5042x1 double]    [1929x1 double]    [4099x1 double]
> > > ................ till columns 3000
> >
> > > many thanks in advance
>
> This seems to be the most important sentence:
> "At each date, I need to for a combination including 3 random selected
> values from the 3000 cells. "
> but it's nonsense. What does "to for a combination" mean?
> You need to slow down and explain yourself better.
>
> Also, don't top post. It makes the thread hard to follow.
> Put replies UNDERNEATH.

Hi,

I think I need to rephrase the question:
I have two cell arrays both have the same size (1*6)

dates =
 
> > > Columns 1 through 6
 
  [3674x1 double] [2319x1 double] [5042x1 double] [5042x1 double] [1929x1 double] [4099x1 double]

DATA=
 
> > > Columns 1 through 6
 
  [3674x1 double] [2319x1 double] [5042x1 double] [5042x1 double] [1929x1 double] [4099x1 double]


In each cell I have the most 5042 elements, named Pt. the starting and ending time for each Pt cells are different along cells 1 to 6.
At each common date for 3 Pts selected from the 6 cells, I need to create (n!) / (k!(n-k)!) combinations, where n=6 and k=3.

So I am wondering to build up the time series for each cell and find intersections for each cell first. Or I should draw the combinations at each date step?

Is there an effective way to do this?

"Skirt Zhang" <silence_qunzi@hotmail.com> wrote in message <j06f29$svs$1@newscl01ah.mathworks.com>...

>
> Hi,
>
> I think I need to rephrase the question:
> I have two cell arrays both have the same size (1*6)
>
> dates =
>
> > > > Columns 1 through 6
>
> [3674x1 double] [2319x1 double] [5042x1 double] [5042x1 double] [1929x1 double] [4099x1 double]
>
> DATA=
>
> > > > Columns 1 through 6
>
> [3674x1 double] [2319x1 double] [5042x1 double] [5042x1 double] [1929x1 double] [4099x1 double]
>
>
> In each cell I have the most 5042 elements, named Pt. the starting and ending time for each Pt cells are different along cells 1 to 6.
> At each common date for 3 Pts selected from the 6 cells, I need to create (n!) / (k!(n-k)!) combinations, where n=6 and k=3.
>

What if the date is not found in 6 cells ? Do we have to consider them ?

Bruno

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <j06gkt$41i$1@newscl01ah.mathworks.com>...
> "Skirt Zhang" <silence_qunzi@hotmail.com> wrote in message <j06f29$svs$1@newscl01ah.mathworks.com>...
>
> >
> > Hi,
> >
> > I think I need to rephrase the question:
> > I have two cell arrays both have the same size (1*6)
> >
> > dates =
> >
> > > > > Columns 1 through 6
> >
> > [3674x1 double] [2319x1 double] [5042x1 double] [5042x1 double] [1929x1 double] [4099x1 double]
> >
> > DATA=
> >
> > > > > Columns 1 through 6
> >
> > [3674x1 double] [2319x1 double] [5042x1 double] [5042x1 double] [1929x1 double] [4099x1 double]
> >
> >
> > In each cell I have the most 5042 elements, named Pt. the starting and ending time for each Pt cells are different along cells 1 to 6.
> > At each common date for 3 Pts selected from the 6 cells, I need to create (n!) / (k!(n-k)!) combinations, where n=6 and k=3.
> >
>
> What if the date is not found in 6 cells ? Do we have to consider them ?
>
> Bruno

Hi Bruno,

The date is corresponding to the Pt, where t is the time index. I will create 6 time series combing the data and dates.

"Skirt Zhang" <silence_qunzi@hotmail.com> wrote in message
news:j06h7l$5m2$1@newscl01ah.mathworks.com...
> "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message
> <j06gkt$41i$1@newscl01ah.mathworks.com>...
>> "Skirt Zhang" <silence_qunzi@hotmail.com> wrote in message
>> <j06f29$svs$1@newscl01ah.mathworks.com>...

*snip*

It's still not clear (at least to me) what you're trying to do.

Write some code that creates a SMALL example (each cell array containing 4
cells, each cell of which contains no more than 5-6 elements.) Then with
that data, show exactly what results you expect and explain (in words) how
you obtained those results from that sample data. It may be easier for us to
understand the specific example and use that to figure out the general case.
[Remember to include both the code that creates the sample data and the
explanation of the results you expect.]

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

On 7/20/2011 8:24 AM, Steven_Lord wrote:
...

> It's still not clear (at least to me) what you're trying to do.
...

What, _your_ crystal ball is in the shop, too, Steve???? :)

--

"Steven_Lord" <slord@mathworks.com> wrote in message <j06ku4$gkt$1@newscl01ah.mathworks.com>...
>
>
> "Skirt Zhang" <silence_qunzi@hotmail.com> wrote in message
> news:j06h7l$5m2$1@newscl01ah.mathworks.com...
> > "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message
> > <j06gkt$41i$1@newscl01ah.mathworks.com>...
> >> "Skirt Zhang" <silence_qunzi@hotmail.com> wrote in message
> >> <j06f29$svs$1@newscl01ah.mathworks.com>...
>
> *snip*
>
> It's still not clear (at least to me) what you're trying to do.
>
> Write some code that creates a SMALL example (each cell array containing 4
> cells, each cell of which contains no more than 5-6 elements.) Then with
> that data, show exactly what results you expect and explain (in words) how
> you obtained those results from that sample data. It may be easier for us to
> understand the specific example and use that to figure out the general case.
> [Remember to include both the code that creates the sample data and the
> explanation of the results you expect.]
>
> --
> Steve Lord
> slord@mathworks.com
> To contact Technical Support use the Contact Us link on
> http://www.mathworks.com

dates is in cell :

dates =
 Columns 1 through 6

 [3674x1 double] [2319x1 double] [5042x1 double] [5042x1 double]
[30-05-1995 [27-02-1992 [03-01-1990 [03-01-1990

....... ....... .......
....... ....... .......
29-12-2009] 02-05-2001] 31-12-2009] 31-12-2009]


Data is in cell :

Data =
 Columns 1 through 6

 [3674x1 double] [2319x1 double] [5042x1 double] [5042x1 double]
[o.o4 [0.87 [0.27 [0.77

....... ....... ....... .......
....... ....... ....... .......
0.89] 0.76] 0.47 ] 1]

Call them

Data1 Data2 Data3 Data4
First I want to generate timeseries
[30-05-1995 o.o4 [27-02-1992 0.87 [03-01-1990 0.27 [03-01-1990 0.77

....... ....... ....... .......
....... ....... ....... .......
29-12-2009 0.89] 02-05-2001 0.76 ] 31-12-2009 0.47] 31-12-2009 1]


I call the above
 TS1 TS2 TS3 TS4

then I want to calculate the product of three elements of data selected from TS1 to 4.

For example, if these three are selected from TS1, 2 and 3 respectively, I will only have the intersected period 30-05-1995----02-05-2001 this gives me the calculating period. For each date during this period, I calculate the product of the three values, defined as "a".

If I select 3 from TS4, 2 and 3, then 27-02-1992 ---------02-05-2001 will be my calculation period.


In the end I want to generate a new time series, for which the time stamp is the longest one:

 [03-01-1990 A1

 

 31-12-2009 Aend]

and A1 equals to the average value of the 3 selected elements' product, i.e. a, at the corresponding date. A2....Aend follows the same procedure.

Hope this time it's more clear?

"Skirt Zhang" <silence_qunzi@hotmail.com> wrote in message
news:j06ocb$sah$1@newscl01ah.mathworks.com...
> "Steven_Lord" <slord@mathworks.com> wrote in message
> <j06ku4$gkt$1@newscl01ah.mathworks.com>...
>>
>>
>> "Skirt Zhang" <silence_qunzi@hotmail.com> wrote in message
>> news:j06h7l$5m2$1@newscl01ah.mathworks.com...
>> > "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message
>> > <j06gkt$41i$1@newscl01ah.mathworks.com>...
>> >> "Skirt Zhang" <silence_qunzi@hotmail.com> wrote in message
>> >> <j06f29$svs$1@newscl01ah.mathworks.com>...
>>
>> *snip*
>>
>> It's still not clear (at least to me) what you're trying to do.
>>
>> Write some code that creates a SMALL example (each cell array containing
>> 4 cells, each cell of which contains no more than 5-6 elements.) Then
>> with that data, show exactly what results you expect and explain (in
>> words) how you obtained those results from that sample data. It may be
>> easier for us to understand the specific example and use that to figure
>> out the general case. [Remember to include both the code that creates the
>> sample data and the explanation of the results you expect.]
>>
>> --
>> Steve Lord
>> slord@mathworks.com
>> To contact Technical Support use the Contact Us link on
>> http://www.mathworks.com
>
> dates is in cell :
>
> dates =
> Columns 1 through 6
>
> [3674x1 double] [2319x1 double] [5042x1 double] [5042x1
> double]

*snip*

Stop. You didn't do what I asked.

> Hope this time it's more clear?

No. Post code that people can RUN and use to experiment on possible
solutions.

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

"dpb" <none@non.net> wrote in message news:j06nvn$989$2@speranza.aioe.org...
> On 7/20/2011 8:24 AM, Steven_Lord wrote:
> ...
>
>> It's still not clear (at least to me) what you're trying to do.
> ...
>
> What, _your_ crystal ball is in the shop, too, Steve???? :)

It's working on a different project; I may switch it back to CSSM mode soon,
though.

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

"Steven_Lord" <slord@mathworks.com> wrote in message <j06oqj$jd$1@newscl01ah.mathworks.com>...
>
>
> "dpb" <none@non.net> wrote in message news:j06nvn$989$2@speranza.aioe.org...
> > On 7/20/2011 8:24 AM, Steven_Lord wrote:
> > ...
> >
> >> It's still not clear (at least to me) what you're trying to do.
> > ...
> >
> > What, _your_ crystal ball is in the shop, too, Steve???? :)
>
> It's working on a different project; I may switch it back to CSSM mode soon,
> though.
>
> --
> Steve Lord
> slord@mathworks.com
> To contact Technical Support use the Contact Us link on


Here is the code to generate the time series:

for i=1:6
    Data{i}=rand(300+10*i,1);
    Date{i}= ( 728809-i:1:(728809+299+10*i-i)) ;
end


%% for betas
for i=1:6
ts{i} = timeseries(Data{i},Date{i}');

end