Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Speed Performance

Subject: Speed Performance

From: Ashish Gupta

Date: 11 Apr, 2011 13:48:04

Message: 1 of 9

This is one of the part of my program .And its taking huge time .. I m not understanding how to improve the performance by any means . All matrix are preallocated. I m not able to understand the vectorization in nested loops. I am a newbie .
R Hstar, tr, are all calculated before..



for n=2:10000,
    for j=12:-1:1,
        for p=1:32,
            for k=1:50,
                for l=1:50,
                    Bkplj{j}(p,k,l)=multiplier*R(p,k,l)*Hstar(k,l);
                    probx=tr{1,j+1};
                     for q=1:32,
                        Fd(q)=Max{j+1}(l,q);
                        Multiply(q)= probx(q)*Fd(q);
                        Summation=sum(Multiply);
                     end
                     Acc{j}(p,k,l)= Bkplj{j}(p,k,l)+ Summation;
                     Max{j} =(max(Acc{j},[],3))';
                     [Trap{j} , stmax{j}]= max(Acc{j},[],3);
                end
            end
        end
    end
    for k=1:50,
        for q=1:32,
            Fd(q)=Max{j}(k,q);
            probx= tr{1,j};
            Multiply(q)=probx(q)*Fd(q);
            Summation=sum(Multiply);
        end
        F{n}(k)=Summation;
    end
    for k=1:15,
        if all((F{n}(k)-F{n-1}(k))/F{n}(k)<0.001)
            disp(' CONVERGENCE DONE')
            return
        end
    end
    Max{13}=Max{1};
end
 

Subject: Speed Performance

From: Robert Cumming

Date: 11 Apr, 2011 13:53:05

Message: 2 of 9

"Ashish Gupta" <fidrizersashish@gmail.com> wrote in message <inv0qk$qs2$1@fred.mathworks.com>...
> This is one of the part of my program .And its taking huge time .. I m not understanding how to improve the performance by any means . All matrix are preallocated. I m not able to understand the vectorization in nested loops. I am a newbie .
> R Hstar, tr, are all calculated before..
>
>
>
> for n=2:10000,
> for j=12:-1:1,
> for p=1:32,
> for k=1:50,
> for l=1:50,
> Bkplj{j}(p,k,l)=multiplier*R(p,k,l)*Hstar(k,l);
> probx=tr{1,j+1};
> for q=1:32,
> Fd(q)=Max{j+1}(l,q);
> Multiply(q)= probx(q)*Fd(q);
> Summation=sum(Multiply);
> end
> Acc{j}(p,k,l)= Bkplj{j}(p,k,l)+ Summation;
> Max{j} =(max(Acc{j},[],3))';
> [Trap{j} , stmax{j}]= max(Acc{j},[],3);
> end
> end
> end
> end
> for k=1:50,
> for q=1:32,
> Fd(q)=Max{j}(k,q);
> probx= tr{1,j};
> Multiply(q)=probx(q)*Fd(q);
> Summation=sum(Multiply);
> end
> F{n}(k)=Summation;
> end
> for k=1:15,
> if all((F{n}(k)-F{n-1}(k))/F{n}(k)<0.001)
> disp(' CONVERGENCE DONE')
> return
> end
> end
> Max{13}=Max{1};
> end
>
Have you used the profiler to check where the code spends most time?

profile on

then run your code

profile viewer

will provide you with a html page which you can investigate your code.

Subject: Speed Performance

From: Florin Neacsu

Date: 11 Apr, 2011 17:26:04

Message: 3 of 9

"Ashish Gupta" <fidrizersashish@gmail.com> wrote in message <inv0qk$qs2$1@fred.mathworks.com>...
> This is one of the part of my program .And its taking huge time .. I m not understanding how to improve the performance by any means . All matrix are preallocated. I m not able to understand the vectorization in nested loops. I am a newbie .
> R Hstar, tr, are all calculated before..
>
>
>
> for n=2:10000,
> for j=12:-1:1,
> for p=1:32,
> for k=1:50,
> for l=1:50,
> Bkplj{j}(p,k,l)=multiplier*R(p,k,l)*Hstar(k,l);
> probx=tr{1,j+1};
> for q=1:32,
> Fd(q)=Max{j+1}(l,q);
> Multiply(q)= probx(q)*Fd(q);
> Summation=sum(Multiply);
> end
> Acc{j}(p,k,l)= Bkplj{j}(p,k,l)+ Summation;
> Max{j} =(max(Acc{j},[],3))';
> [Trap{j} , stmax{j}]= max(Acc{j},[],3);
> end
> end
> end
> end
> for k=1:50,
> for q=1:32,
> Fd(q)=Max{j}(k,q);
> probx= tr{1,j};
> Multiply(q)=probx(q)*Fd(q);
> Summation=sum(Multiply);
> end
> F{n}(k)=Summation;
> end
> for k=1:15,
> if all((F{n}(k)-F{n-1}(k))/F{n}(k)<0.001)
> disp(' CONVERGENCE DONE')
> return
> end
> end
> Max{13}=Max{1};
> end
>

Hi,

you are doing more than
32*50*50*30*12*10000

ans =

  2.8800e+011

operations, and you are also using max on each of those operations.

Maybe you could explain what you are trying to do and then someone could suggest another approach.

Regards,
Florin
 

Subject: Speed Performance

From: Ashish Gupta

Date: 11 Apr, 2011 18:02:05

Message: 4 of 9

"Florin Neacsu" <fneacsu2@gmail.com> wrote in message <invdjc$of4$1@fred.mathworks.com>...
> "Ashish Gupta" <fidrizersashish@gmail.com> wrote in message <inv0qk$qs2$1@fred.mathworks.com>...
> > This is one of the part of my program .And its taking huge time .. I m not understanding how to improve the performance by any means . All matrix are preallocated. I m not able to understand the vectorization in nested loops. I am a newbie .
> > R Hstar, tr, are all calculated before..
> >
> >
> >
> > for n=2:10000,
> > for j=12:-1:1,
> > for p=1:32,
> > for k=1:50,
> > for l=1:50,
> > Bkplj{j}(p,k,l)=multiplier*R(p,k,l)*Hstar(k,l);
> > probx=tr{1,j+1};
> > for q=1:32,
> > Fd(q)=Max{j+1}(l,q);
> > Multiply(q)= probx(q)*Fd(q);
> > Summation=sum(Multiply);
> > end
> > Acc{j}(p,k,l)= Bkplj{j}(p,k,l)+ Summation;
> > Max{j} =(max(Acc{j},[],3))';
> > [Trap{j} , stmax{j}]= max(Acc{j},[],3);
> > end
> > end
> > end
> > end
> > for k=1:50,
> > for q=1:32,
> > Fd(q)=Max{j}(k,q);
> > probx= tr{1,j};
> > Multiply(q)=probx(q)*Fd(q);
> > Summation=sum(Multiply);
> > end
> > F{n}(k)=Summation;
> > end
> > for k=1:15,
> > if all((F{n}(k)-F{n-1}(k))/F{n}(k)<0.001)
> > disp(' CONVERGENCE DONE')
> > return
> > end
> > end
> > Max{13}=Max{1};
> > end
> >
>
> Hi,
>
> you are doing more than
> 32*50*50*30*12*10000
>
> ans =
>
> 2.8800e+011
>
> operations, and you are also using max on each of those operations.
>
> Maybe you could explain what you are trying to do and then someone could suggest another approach.
>
> Regards,
> Florin
> I am applyin Stochastic Dynamic Programming on reservoir operation . and its basically used to check convergence . I think my coding is inefficient .So i was askin may be some loops can be taken out in order to reduce time or somethin else .

Subject: Speed Performance

From: Ashish Gupta

Date: 11 Apr, 2011 18:06:05

Message: 5 of 9

"Robert Cumming" <robert.cumming@airbus.com> wrote in message <inv141$30a$1@fred.mathworks.com>...
> "Ashish Gupta" <fidrizersashish@gmail.com> wrote in message <inv0qk$qs2$1@fred.mathworks.com>...
> > This is one of the part of my program .And its taking huge time .. I m not understanding how to improve the performance by any means . All matrix are preallocated. I m not able to understand the vectorization in nested loops. I am a newbie .
> > R Hstar, tr, are all calculated before..
> >
> >
> >
> > for n=2:10000,
> > for j=12:-1:1,
> > for p=1:32,
> > for k=1:50,
> > for l=1:50,
> > Bkplj{j}(p,k,l)=multiplier*R(p,k,l)*Hstar(k,l);
> > probx=tr{1,j+1};
> > for q=1:32,
> > Fd(q)=Max{j+1}(l,q);
> > Multiply(q)= probx(q)*Fd(q);
> > Summation=sum(Multiply);
> > end
> > Acc{j}(p,k,l)= Bkplj{j}(p,k,l)+ Summation;
> > Max{j} =(max(Acc{j},[],3))';
> > [Trap{j} , stmax{j}]= max(Acc{j},[],3);
> > end
> > end
> > end
> > end
> > for k=1:50,
> > for q=1:32,
> > Fd(q)=Max{j}(k,q);
> > probx= tr{1,j};
> > Multiply(q)=probx(q)*Fd(q);
> > Summation=sum(Multiply);
> > end
> > F{n}(k)=Summation;
> > end
> > for k=1:15,
> > if all((F{n}(k)-F{n-1}(k))/F{n}(k)<0.001)
> > disp(' CONVERGENCE DONE')
> > return
> > end
> > end
> > Max{13}=Max{1};
> > end
> >
> Have you used the profiler to check where the code spends most time?
>
> profile on
>
> then run your code
>
> profile viewer
>
> will provide you with a html page which you can investigate your code.
Well problem is that in order to check the results from profiler u have to run the code first fully. It took more then 20 hrs for n=2:150

Subject: Speed Performance

From: Florin Neacsu

Date: 11 Apr, 2011 19:20:24

Message: 6 of 9

"Ashish Gupta" <fidrizersashish@gmail.com> wrote in message <invfud$8mq$1@fred.mathworks.com>...
> "Robert Cumming" <robert.cumming@airbus.com> wrote in message <inv141$30a$1@fred.mathworks.com>...
> > "Ashish Gupta" <fidrizersashish@gmail.com> wrote in message <inv0qk$qs2$1@fred.mathworks.com>...
> > > This is one of the part of my program .And its taking huge time .. I m not understanding how to improve the performance by any means . All matrix are preallocated. I m not able to understand the vectorization in nested loops. I am a newbie .
> > > R Hstar, tr, are all calculated before..
> > >
> > >
> > >
> > > for n=2:10000,
> > > for j=12:-1:1,
> > > for p=1:32,
> > > for k=1:50,
> > > for l=1:50,
> > > Bkplj{j}(p,k,l)=multiplier*R(p,k,l)*Hstar(k,l);
> > > probx=tr{1,j+1};
> > > for q=1:32,
> > > Fd(q)=Max{j+1}(l,q);
> > > Multiply(q)= probx(q)*Fd(q);
> > > Summation=sum(Multiply);
> > > end
> > > Acc{j}(p,k,l)= Bkplj{j}(p,k,l)+ Summation;
> > > Max{j} =(max(Acc{j},[],3))';
> > > [Trap{j} , stmax{j}]= max(Acc{j},[],3);
> > > end
> > > end
> > > end
> > > end
> > > for k=1:50,
> > > for q=1:32,
> > > Fd(q)=Max{j}(k,q);
> > > probx= tr{1,j};
> > > Multiply(q)=probx(q)*Fd(q);
> > > Summation=sum(Multiply);
> > > end
> > > F{n}(k)=Summation;
> > > end
> > > for k=1:15,
> > > if all((F{n}(k)-F{n-1}(k))/F{n}(k)<0.001)
> > > disp(' CONVERGENCE DONE')
> > > return
> > > end
> > > end
> > > Max{13}=Max{1};
> > > end
> > >
> > Have you used the profiler to check where the code spends most time?
> >
> > profile on
> >
> > then run your code
> >
> > profile viewer
> >
> > will provide you with a html page which you can investigate your code.
> Well problem is that in order to check the results from profiler u have to run the code first fully. It took more then 20 hrs for n=2:150

Well,

32*50*50*30*12*250*0.000017/3600

ans =

    34

0.000017 is the time for accessing a matrix and performing a simple operation on my system. I know it's not the most conclusive example, but it explains your 20h+ execution time.

You can run your code for fever iterations and still notice where the bottle neck is. But my guess is you need to change your approach. Sorry, I don't have any suggestions on how you should do it ...

Regards,
Florin

Subject: Speed Performance

From: Ashish Gupta

Date: 11 Apr, 2011 20:41:05

Message: 7 of 9

"Florin Neacsu" <fneacsu2@gmail.com> wrote in message <invk9o$p7v$1@fred.mathworks.com>...
> "Ashish Gupta" <fidrizersashish@gmail.com> wrote in message <invfud$8mq$1@fred.mathworks.com>...
> > "Robert Cumming" <robert.cumming@airbus.com> wrote in message <inv141$30a$1@fred.mathworks.com>...
> > > "Ashish Gupta" <fidrizersashish@gmail.com> wrote in message <inv0qk$qs2$1@fred.mathworks.com>...
> > > > This is one of the part of my program .And its taking huge time .. I m not understanding how to improve the performance by any means . All matrix are preallocated. I m not able to understand the vectorization in nested loops. I am a newbie .
> > > > R Hstar, tr, are all calculated before..
> > > >
> > > >
> > > >
> > > > for n=2:10000,
> > > > for j=12:-1:1,
> > > > for p=1:32,
> > > > for k=1:50,
> > > > for l=1:50,
> > > > Bkplj{j}(p,k,l)=multiplier*R(p,k,l)*Hstar(k,l);
> > > > probx=tr{1,j+1};
> > > > for q=1:32,
> > > > Fd(q)=Max{j+1}(l,q);
> > > > Multiply(q)= probx(q)*Fd(q);
> > > > Summation=sum(Multiply);
> > > > end
> > > > Acc{j}(p,k,l)= Bkplj{j}(p,k,l)+ Summation;
> > > > Max{j} =(max(Acc{j},[],3))';
> > > > [Trap{j} , stmax{j}]= max(Acc{j},[],3);
> > > > end
> > > > end
> > > > end
> > > > end
> > > > for k=1:50,
> > > > for q=1:32,
> > > > Fd(q)=Max{j}(k,q);
> > > > probx= tr{1,j};
> > > > Multiply(q)=probx(q)*Fd(q);
> > > > Summation=sum(Multiply);
> > > > end
> > > > F{n}(k)=Summation;
> > > > end
> > > > for k=1:15,
> > > > if all((F{n}(k)-F{n-1}(k))/F{n}(k)<0.001)
> > > > disp(' CONVERGENCE DONE')
> > > > return
> > > > end
> > > > end
> > > > Max{13}=Max{1};
> > > > end
> > > >
> > > Have you used the profiler to check where the code spends most time?
> > >
> > > profile on
> > >
> > > then run your code
> > >
> > > profile viewer
> > >
> > > will provide you with a html page which you can investigate your code.
> > Well problem is that in order to check the results from profiler u have to run the code first fully. It took more then 20 hrs for n=2:150
>
> Well,
>
> 32*50*50*30*12*250*0.000017/3600
>
> ans =
>
> 34
>
> 0.000017 is the time for accessing a matrix and performing a simple operation on my system. I know it's not the most conclusive example, but it explains your 20h+ execution time.
>
> You can run your code for fever iterations and still notice where the bottle neck is. But my guess is you need to change your approach. Sorry, I don't have any suggestions on how you should do it ...
>
> Regards,
> Florin

I guess i have to try another approach . Although running this program on the clock for fewer iteration i found that in these two lines its taking most of the time .
        Multiply(q)=probx(q)*Fd(q);
        Summation=sum(Multiply);
So i removed the probx=tr{1,j+1}; and in this line i used
        Multiply(q)=tr{1,j+1}(q)*Fd(q); It produces same result.with 10% faster.
Although if i change further to
 Summation=sum(tr{1,j+1}(q)*Fd(q);
it produces entirely different results but reduces times by 55%. i am not able to understand why its happening .
tr is 1*j+1 cell each cell having q values.
Regards
Ashish

Subject: Speed Performance

From: dpb

Date: 11 Apr, 2011 23:14:22

Message: 8 of 9

On 4/11/2011 1:02 PM, Ashish Gupta wrote:
...

>> I am applyin Stochastic Dynamic Programming on reservoir operation .
>> and its basically used to check convergence . I think my coding is
>> inefficient .So i was askin may be some loops can be taken out in
>> order to reduce time or somethin else .

To recast the question -- can you write the actual computation as a set
of analytical equations rather than making folks try to interpret the
end result from the code?

Knowing that, some may well recognize well known algebraic "tricks" that
are much easier to apply if one can work on the formulation instead of
trying to do so so from an implementation.

--

Subject: Speed Performance

From: Rune Allnor

Date: 12 Apr, 2011 07:07:18

Message: 9 of 9

On Apr 11, 8:02 pm, "Ashish Gupta" <fidrizersash...@gmail.com> wrote:

> >  I am applyin Stochastic Dynamic Programming  on reservoir operation . and its basically  used to check convergence . I think my coding is inefficient .So i was askin may be some loops can be taken out in order to reduce time or somethin else .

In that case you are in *way* over your head. The first
question you need to get to grips with, is if you are
ready to play with the really big boys: This kind of thing
is at the forefront of industrial research.

If you think you are, then

1) Learn efficient programming. Forget about matlab
   or java, maybe even C++. This is the kind of thing
   where you need to get close to 'the metal', meaning
   C or even assembler language.

2) learn how to use supercomputers. To have any chance
   of succeeding, you will need to learn how to take
   advantage of the present state of the art in computers.

3) Get access to a supercomputer and apply what you
   learned under items 1) and 2).

If you start from scratch, as it seems you does, be prepared
to spend 15-20 years learning all the necessary tricks and
strategies required to get you close to your goal.

Rune

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us