Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
computation time of function lsqnonlin

Subject: computation time of function lsqnonlin

From: Toan Cao

Date: 13 Sep, 2013 03:38:06

Message: 1 of 13

Hi,
I write an objective function in both m-file (fileA1.m) and C mex file (fileA2.c). Given an input to this objective function, computation time in C mex file is 15 times faster than that of m- file. However, when i use the objective function in function lsqnonlin of Matlab, computation time which lsqnonlin gets a optimal solution is almost the same for both files (~101 minutes for C mex file and ~104 minutes for m-file).
I hope when i use C mex file, i will save more time for optimization process, but it is not.
Can you help me explain why lsqnonlin spends almost the same computation time?
I would like to understand more how lsqnonlin of Matlab works to reduce its time?
Thanks in advance!

Toan

Subject: computation time of function lsqnonlin

From: Rakesh Kumar

Date: 13 Sep, 2013 14:25:07

Message: 2 of 13

"Toan Cao" <toancv3010@gmail.com> wrote in message <l0u1au$ei4$1@newscl01ah.mathworks.com>...
> Hi,
> I write an objective function in both m-file (fileA1.m) and C mex file (fileA2.c). Given an input to this objective function, computation time in C mex file is 15 times faster than that of m- file. However, when i use the objective function in function lsqnonlin of Matlab, computation time which lsqnonlin gets a optimal solution is almost the same for both files (~101 minutes for C mex file and ~104 minutes for m-file).
> I hope when i use C mex file, i will save more time for optimization process, but it is not.
> Can you help me explain why lsqnonlin spends almost the same computation time?
> I would like to understand more how lsqnonlin of Matlab works to reduce its time?
> Thanks in advance!
>
> Toan

Hi Toan,
This is unexpected.
First, I would suggest to compare the #iterations and #function evaluations in two scenarios (just to make sure we have same implementation in MATLAB and C). If these numbers are similar, then I would profile and see where the time is spent in two scenarios. Can you report back here with your findings? Thanks!

Rakesh

Subject: computation time of function lsqnonlin

From: Steven_Lord

Date: 13 Sep, 2013 14:50:15

Message: 3 of 13



"Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message
news:l0v783$po$1@newscl01ah.mathworks.com...
> "Toan Cao" <toancv3010@gmail.com> wrote in message
> <l0u1au$ei4$1@newscl01ah.mathworks.com>...
>> Hi,
>> I write an objective function in both m-file (fileA1.m) and C mex file
>> (fileA2.c). Given an input to this objective function, computation time
>> in C mex file is 15 times faster than that of m- file. However, when i
>> use the objective function in function lsqnonlin of Matlab, computation
>> time which lsqnonlin gets a optimal solution is almost the same for both
>> files (~101 minutes for C mex file and ~104 minutes for m-file).
>> I hope when i use C mex file, i will save more time for optimization
>> process, but it is not.
>> Can you help me explain why lsqnonlin spends almost the same computation
>> time?
>> I would like to understand more how lsqnonlin of Matlab works to reduce
>> its time?
>> Thanks in advance!
>>
>> Toan
>
> Hi Toan,
> This is unexpected. First, I would suggest to compare the #iterations and
> #function evaluations in two scenarios (just to make sure we have same
> implementation in MATLAB and C). If these numbers are similar, then I
> would profile and see where the time is spent in two scenarios. Can you
> report back here with your findings? Thanks!

In addition, make sure you're comparing apples and oranges. Do fileA1.m and
fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given
the exact same inputs? Even a small difference in the return values could
affect the path in the solution space the optimization routines follow, if I
remember correctly.

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

Subject: computation time of function lsqnonlin

From: Toan Cao

Date: 13 Sep, 2013 18:59:06

Message: 4 of 13

"Steven_Lord" <slord@mathworks.com> wrote in message <l0v8n7$so5$1@newscl01ah.mathworks.com>...
>
>
> "Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message
> news:l0v783$po$1@newscl01ah.mathworks.com...
> > "Toan Cao" <toancv3010@gmail.com> wrote in message
> > <l0u1au$ei4$1@newscl01ah.mathworks.com>...
> >> Hi,
> >> I write an objective function in both m-file (fileA1.m) and C mex file
> >> (fileA2.c). Given an input to this objective function, computation time
> >> in C mex file is 15 times faster than that of m- file. However, when i
> >> use the objective function in function lsqnonlin of Matlab, computation
> >> time which lsqnonlin gets a optimal solution is almost the same for both
> >> files (~101 minutes for C mex file and ~104 minutes for m-file).
> >> I hope when i use C mex file, i will save more time for optimization
> >> process, but it is not.
> >> Can you help me explain why lsqnonlin spends almost the same computation
> >> time?
> >> I would like to understand more how lsqnonlin of Matlab works to reduce
> >> its time?
> >> Thanks in advance!
> >>
> >> Toan
> >
> > Hi Toan,
> > This is unexpected. First, I would suggest to compare the #iterations and
> > #function evaluations in two scenarios (just to make sure we have same
> > implementation in MATLAB and C). If these numbers are similar, then I
> > would profile and see where the time is spent in two scenarios. Can you
> > report back here with your findings? Thanks!
>
> In addition, make sure you're comparing apples and oranges. Do fileA1.m and
> fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given
> the exact same inputs? Even a small difference in the return values could
> affect the path in the solution space the optimization routines follow, if I
> remember correctly.
>
> --
> Steve Lord
> slord@mathworks.com
> To contact Technical Support use the Contact Us link on
> http://www.mathworks.com

Hi Rakesh,

When i set just for 5 iteration , following outputs of optimization process make me more confusion:
-----------------------------------------------------------------------------------------
m-file:
                                                    First-Order Norm of
 Iteration Func-count Residual optimality Lambda step
     0 1 9.02706 1.1 0.02
     1 4 8.15864 0.775 2 0.715913
     2 6 7.12331 0.592 20 0.19392
     3 7 6.6927 0.546 2 0.147436
     4 9 6.4763 0.513 20 0.124842
     5 10 6.37183 0.486 2 0.112678
     6 12 6.28601 0.465 20 0.106625
Elapsed time is 3240.325396 seconds.

C mex file:

                                                    First-Order Norm of
 Iteration Func-count Residual optimality Lambda step
     0 1 9.02706 1.1 0.02
     1 4 8.15864 0.775 2 0.715913
     2 6 7.12331 0.592 20 0.19392
     3 7 6.6927 0.546 2 0.147436
     4 9 6.4763 0.513 20 0.124842
     5 10 6.37183 0.486 2 0.112678
     6 12 6.28601 0.465 20 0.106625
Elapsed time is 8275.102153 seconds.
-----------------------------------------------------------------------------------------
Uhmmmm, as you see every parameters are similar but it is terrible when elapsed time for C mex file is even longer than that of m-file for above case.

Hi Steve Lord,
There are some things relating to what you say. In Jacobian matrix returned by the objective function, when i use Matlab to compare element by element between these two matrices of 2 files, Matlab indicates some different values among elements.
However, if i run comparison two times of the same output Jacobian matrices , locations of difference at each running time are not the same. Then, i check some locations , some values of 2 elements are different at last digit as Matlab indicates. (for example: 4.559150742630094 vs 4.559150742630095), some values are the same (for example: 1.641766402933731 vs 1.641766402933731).
Therefore, i think this is due to machine error between values returned by c mex file and m-file. But i can not still explain where is the problem relating computation time of lsqnonlin.

Looking forward to your reply.
Toan

Subject: computation time of function lsqnonlin

From: Rakesh Kumar

Date: 13 Sep, 2013 19:09:06

Message: 5 of 13

"Toan Cao" <toancv3010@gmail.com> wrote in message <l0vn9q$8pp$1@newscl01ah.mathworks.com>...
> "Steven_Lord" <slord@mathworks.com> wrote in message <l0v8n7$so5$1@newscl01ah.mathworks.com>...
> >
> >
> > "Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message
> > news:l0v783$po$1@newscl01ah.mathworks.com...
> > > "Toan Cao" <toancv3010@gmail.com> wrote in message
> > > <l0u1au$ei4$1@newscl01ah.mathworks.com>...
> > >> Hi,
> > >> I write an objective function in both m-file (fileA1.m) and C mex file
> > >> (fileA2.c). Given an input to this objective function, computation time
> > >> in C mex file is 15 times faster than that of m- file. However, when i
> > >> use the objective function in function lsqnonlin of Matlab, computation
> > >> time which lsqnonlin gets a optimal solution is almost the same for both
> > >> files (~101 minutes for C mex file and ~104 minutes for m-file).
> > >> I hope when i use C mex file, i will save more time for optimization
> > >> process, but it is not.
> > >> Can you help me explain why lsqnonlin spends almost the same computation
> > >> time?
> > >> I would like to understand more how lsqnonlin of Matlab works to reduce
> > >> its time?
> > >> Thanks in advance!
> > >>
> > >> Toan
> > >
> > > Hi Toan,
> > > This is unexpected. First, I would suggest to compare the #iterations and
> > > #function evaluations in two scenarios (just to make sure we have same
> > > implementation in MATLAB and C). If these numbers are similar, then I
> > > would profile and see where the time is spent in two scenarios. Can you
> > > report back here with your findings? Thanks!
> >
> > In addition, make sure you're comparing apples and oranges. Do fileA1.m and
> > fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given
> > the exact same inputs? Even a small difference in the return values could
> > affect the path in the solution space the optimization routines follow, if I
> > remember correctly.
> >
> > --
> > Steve Lord
> > slord@mathworks.com
> > To contact Technical Support use the Contact Us link on
> > http://www.mathworks.com
>
> Hi Rakesh,
>
> When i set just for 5 iteration , following outputs of optimization process make me more confusion:
> -----------------------------------------------------------------------------------------
> m-file:
> First-Order Norm of
> Iteration Func-count Residual optimality Lambda step
> 0 1 9.02706 1.1 0.02
> 1 4 8.15864 0.775 2 0.715913
> 2 6 7.12331 0.592 20 0.19392
> 3 7 6.6927 0.546 2 0.147436
> 4 9 6.4763 0.513 20 0.124842
> 5 10 6.37183 0.486 2 0.112678
> 6 12 6.28601 0.465 20 0.106625
> Elapsed time is 3240.325396 seconds.
>
> C mex file:
>
> First-Order Norm of
> Iteration Func-count Residual optimality Lambda step
> 0 1 9.02706 1.1 0.02
> 1 4 8.15864 0.775 2 0.715913
> 2 6 7.12331 0.592 20 0.19392
> 3 7 6.6927 0.546 2 0.147436
> 4 9 6.4763 0.513 20 0.124842
> 5 10 6.37183 0.486 2 0.112678
> 6 12 6.28601 0.465 20 0.106625
> Elapsed time is 8275.102153 seconds.
> -----------------------------------------------------------------------------------------
> Uhmmmm, as you see every parameters are similar but it is terrible when elapsed time for C mex file is even longer than that of m-file for above case.
>
> Hi Steve Lord,
> There are some things relating to what you say. In Jacobian matrix returned by the objective function, when i use Matlab to compare element by element between these two matrices of 2 files, Matlab indicates some different values among elements.
> However, if i run comparison two times of the same output Jacobian matrices , locations of difference at each running time are not the same. Then, i check some locations , some values of 2 elements are different at last digit as Matlab indicates. (for example: 4.559150742630094 vs 4.559150742630095), some values are the same (for example: 1.641766402933731 vs 1.641766402933731).
> Therefore, i think this is due to machine error between values returned by c mex file and m-file. But i can not still explain where is the problem relating computation time of lsqnonlin.
>
> Looking forward to your reply.
> Toan

What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?

Subject: computation time of function lsqnonlin

From: Rakesh Kumar

Date: 13 Sep, 2013 19:10:07

Message: 6 of 13

"Toan Cao" <toancv3010@gmail.com> wrote in message <l0vn9q$8pp$1@newscl01ah.mathworks.com>...
> "Steven_Lord" <slord@mathworks.com> wrote in message <l0v8n7$so5$1@newscl01ah.mathworks.com>...
> >
> >
> > "Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message
> > news:l0v783$po$1@newscl01ah.mathworks.com...
> > > "Toan Cao" <toancv3010@gmail.com> wrote in message
> > > <l0u1au$ei4$1@newscl01ah.mathworks.com>...
> > >> Hi,
> > >> I write an objective function in both m-file (fileA1.m) and C mex file
> > >> (fileA2.c). Given an input to this objective function, computation time
> > >> in C mex file is 15 times faster than that of m- file. However, when i
> > >> use the objective function in function lsqnonlin of Matlab, computation
> > >> time which lsqnonlin gets a optimal solution is almost the same for both
> > >> files (~101 minutes for C mex file and ~104 minutes for m-file).
> > >> I hope when i use C mex file, i will save more time for optimization
> > >> process, but it is not.
> > >> Can you help me explain why lsqnonlin spends almost the same computation
> > >> time?
> > >> I would like to understand more how lsqnonlin of Matlab works to reduce
> > >> its time?
> > >> Thanks in advance!
> > >>
> > >> Toan
> > >
> > > Hi Toan,
> > > This is unexpected. First, I would suggest to compare the #iterations and
> > > #function evaluations in two scenarios (just to make sure we have same
> > > implementation in MATLAB and C). If these numbers are similar, then I
> > > would profile and see where the time is spent in two scenarios. Can you
> > > report back here with your findings? Thanks!
> >
> > In addition, make sure you're comparing apples and oranges. Do fileA1.m and
> > fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given
> > the exact same inputs? Even a small difference in the return values could
> > affect the path in the solution space the optimization routines follow, if I
> > remember correctly.
> >
> > --
> > Steve Lord
> > slord@mathworks.com
> > To contact Technical Support use the Contact Us link on
> > http://www.mathworks.com
>
> Hi Rakesh,
>
> When i set just for 5 iteration , following outputs of optimization process make me more confusion:
> -----------------------------------------------------------------------------------------
> m-file:
> First-Order Norm of
> Iteration Func-count Residual optimality Lambda step
> 0 1 9.02706 1.1 0.02
> 1 4 8.15864 0.775 2 0.715913
> 2 6 7.12331 0.592 20 0.19392
> 3 7 6.6927 0.546 2 0.147436
> 4 9 6.4763 0.513 20 0.124842
> 5 10 6.37183 0.486 2 0.112678
> 6 12 6.28601 0.465 20 0.106625
> Elapsed time is 3240.325396 seconds.
>
> C mex file:
>
> First-Order Norm of
> Iteration Func-count Residual optimality Lambda step
> 0 1 9.02706 1.1 0.02
> 1 4 8.15864 0.775 2 0.715913
> 2 6 7.12331 0.592 20 0.19392
> 3 7 6.6927 0.546 2 0.147436
> 4 9 6.4763 0.513 20 0.124842
> 5 10 6.37183 0.486 2 0.112678
> 6 12 6.28601 0.465 20 0.106625
> Elapsed time is 8275.102153 seconds.
> -----------------------------------------------------------------------------------------
> Uhmmmm, as you see every parameters are similar but it is terrible when elapsed time for C mex file is even longer than that of m-file for above case.
>
> Hi Steve Lord,
> There are some things relating to what you say. In Jacobian matrix returned by the objective function, when i use Matlab to compare element by element between these two matrices of 2 files, Matlab indicates some different values among elements.
> However, if i run comparison two times of the same output Jacobian matrices , locations of difference at each running time are not the same. Then, i check some locations , some values of 2 elements are different at last digit as Matlab indicates. (for example: 4.559150742630094 vs 4.559150742630095), some values are the same (for example: 1.641766402933731 vs 1.641766402933731).
> Therefore, i think this is due to machine error between values returned by c mex file and m-file. But i can not still explain where is the problem relating computation time of lsqnonlin.
>
> Looking forward to your reply.
> Toan

What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?

Subject: computation time of function lsqnonlin

From: Toan Cao

Date: 14 Sep, 2013 15:04:06

Message: 7 of 13

"Rakesh Kumar" wrote in message <l0vnuf$kp5$1@newscl01ah.mathworks.com>...
> "Toan Cao" <toancv3010@gmail.com> wrote in message <l0vn9q$8pp$1@newscl01ah.mathworks.com>...
> > "Steven_Lord" <slord@mathworks.com> wrote in message <l0v8n7$so5$1@newscl01ah.mathworks.com>...
> > >
> > >
> > > "Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message
> > > news:l0v783$po$1@newscl01ah.mathworks.com...
> > > > "Toan Cao" <toancv3010@gmail.com> wrote in message
> > > > <l0u1au$ei4$1@newscl01ah.mathworks.com>...
> > > >> Hi,
> > > >> I write an objective function in both m-file (fileA1.m) and C mex file
> > > >> (fileA2.c). Given an input to this objective function, computation time
> > > >> in C mex file is 15 times faster than that of m- file. However, when i
> > > >> use the objective function in function lsqnonlin of Matlab, computation
> > > >> time which lsqnonlin gets a optimal solution is almost the same for both
> > > >> files (~101 minutes for C mex file and ~104 minutes for m-file).
> > > >> I hope when i use C mex file, i will save more time for optimization
> > > >> process, but it is not.
> > > >> Can you help me explain why lsqnonlin spends almost the same computation
> > > >> time?
> > > >> I would like to understand more how lsqnonlin of Matlab works to reduce
> > > >> its time?
> > > >> Thanks in advance!
> > > >>
> > > >> Toan
> > > >
> > > > Hi Toan,
> > > > This is unexpected. First, I would suggest to compare the #iterations and
> > > > #function evaluations in two scenarios (just to make sure we have same
> > > > implementation in MATLAB and C). If these numbers are similar, then I
> > > > would profile and see where the time is spent in two scenarios. Can you
> > > > report back here with your findings? Thanks!
> > >
> > > In addition, make sure you're comparing apples and oranges. Do fileA1.m and
> > > fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given
> > > the exact same inputs? Even a small difference in the return values could
> > > affect the path in the solution space the optimization routines follow, if I
> > > remember correctly.
> > >
> > > --
> > > Steve Lord
> > > slord@mathworks.com
> > > To contact Technical Support use the Contact Us link on
> > > http://www.mathworks.com
> >
> > Hi Rakesh,
> >
> > When i set just for 5 iteration , following outputs of optimization process make me more confusion:
> > -----------------------------------------------------------------------------------------
> > m-file:
> > First-Order Norm of
> > Iteration Func-count Residual optimality Lambda step
> > 0 1 9.02706 1.1 0.02
> > 1 4 8.15864 0.775 2 0.715913
> > 2 6 7.12331 0.592 20 0.19392
> > 3 7 6.6927 0.546 2 0.147436
> > 4 9 6.4763 0.513 20 0.124842
> > 5 10 6.37183 0.486 2 0.112678
> > 6 12 6.28601 0.465 20 0.106625
> > Elapsed time is 3240.325396 seconds.
> >
> > C mex file:
> >
> > First-Order Norm of
> > Iteration Func-count Residual optimality Lambda step
> > 0 1 9.02706 1.1 0.02
> > 1 4 8.15864 0.775 2 0.715913
> > 2 6 7.12331 0.592 20 0.19392
> > 3 7 6.6927 0.546 2 0.147436
> > 4 9 6.4763 0.513 20 0.124842
> > 5 10 6.37183 0.486 2 0.112678
> > 6 12 6.28601 0.465 20 0.106625
> > Elapsed time is 8275.102153 seconds.
> > -----------------------------------------------------------------------------------------
> > Uhmmmm, as you see every parameters are similar but it is terrible when elapsed time for C mex file is even longer than that of m-file for above case.
> >
> > Hi Steve Lord,
> > There are some things relating to what you say. In Jacobian matrix returned by the objective function, when i use Matlab to compare element by element between these two matrices of 2 files, Matlab indicates some different values among elements.
> > However, if i run comparison two times of the same output Jacobian matrices , locations of difference at each running time are not the same. Then, i check some locations , some values of 2 elements are different at last digit as Matlab indicates. (for example: 4.559150742630094 vs 4.559150742630095), some values are the same (for example: 1.641766402933731 vs 1.641766402933731).
> > Therefore, i think this is due to machine error between values returned by c mex file and m-file. But i can not still explain where is the problem relating computation time of lsqnonlin.
> >
> > Looking forward to your reply.
> > Toan
>
> What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?

Hi Rakesh,
Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.
Profiler of optimzation using m-file:
http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg
Profiler of optimzation using C mexfile:
http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg
As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.
Is there any solution to reduce computation time for LevenbergMarquardt ?

Toan

Subject: computation time of function lsqnonlin

From: Rakesh Kumar

Date: 16 Sep, 2013 13:13:06

Message: 8 of 13

"Toan Cao" <toancv3010@gmail.com> wrote in message <l11tt6$c0q$1@newscl01ah.mathworks.com>...
> "Rakesh Kumar" wrote in message <l0vnuf$kp5$1@newscl01ah.mathworks.com>...
> > "Toan Cao" <toancv3010@gmail.com> wrote in message <l0vn9q$8pp$1@newscl01ah.mathworks.com>...
> > > "Steven_Lord" <slord@mathworks.com> wrote in message <l0v8n7$so5$1@newscl01ah.mathworks.com>...
> > > >
> > > >
> > > > "Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message
> > > > news:l0v783$po$1@newscl01ah.mathworks.com...
> > > > > "Toan Cao" <toancv3010@gmail.com> wrote in message
> > > > > <l0u1au$ei4$1@newscl01ah.mathworks.com>...
> > > > >> Hi,
> > > > >> I write an objective function in both m-file (fileA1.m) and C mex file
> > > > >> (fileA2.c). Given an input to this objective function, computation time
> > > > >> in C mex file is 15 times faster than that of m- file. However, when i
> > > > >> use the objective function in function lsqnonlin of Matlab, computation
> > > > >> time which lsqnonlin gets a optimal solution is almost the same for both
> > > > >> files (~101 minutes for C mex file and ~104 minutes for m-file).
> > > > >> I hope when i use C mex file, i will save more time for optimization
> > > > >> process, but it is not.
> > > > >> Can you help me explain why lsqnonlin spends almost the same computation
> > > > >> time?
> > > > >> I would like to understand more how lsqnonlin of Matlab works to reduce
> > > > >> its time?
> > > > >> Thanks in advance!
> > > > >>
> > > > >> Toan
> > > > >
> > > > > Hi Toan,
> > > > > This is unexpected. First, I would suggest to compare the #iterations and
> > > > > #function evaluations in two scenarios (just to make sure we have same
> > > > > implementation in MATLAB and C). If these numbers are similar, then I
> > > > > would profile and see where the time is spent in two scenarios. Can you
> > > > > report back here with your findings? Thanks!
> > > >
> > > > In addition, make sure you're comparing apples and oranges. Do fileA1.m and
> > > > fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given
> > > > the exact same inputs? Even a small difference in the return values could
> > > > affect the path in the solution space the optimization routines follow, if I
> > > > remember correctly.
> > > >
> > > > --
> > > > Steve Lord
> > > > slord@mathworks.com
> > > > To contact Technical Support use the Contact Us link on
> > > > http://www.mathworks.com
> > >
> > > Hi Rakesh,
> > >
> > > When i set just for 5 iteration , following outputs of optimization process make me more confusion:
> > > -----------------------------------------------------------------------------------------
> > > m-file:
> > > First-Order Norm of
> > > Iteration Func-count Residual optimality Lambda step
> > > 0 1 9.02706 1.1 0.02
> > > 1 4 8.15864 0.775 2 0.715913
> > > 2 6 7.12331 0.592 20 0.19392
> > > 3 7 6.6927 0.546 2 0.147436
> > > 4 9 6.4763 0.513 20 0.124842
> > > 5 10 6.37183 0.486 2 0.112678
> > > 6 12 6.28601 0.465 20 0.106625
> > > Elapsed time is 3240.325396 seconds.
> > >
> > > C mex file:
> > >
> > > First-Order Norm of
> > > Iteration Func-count Residual optimality Lambda step
> > > 0 1 9.02706 1.1 0.02
> > > 1 4 8.15864 0.775 2 0.715913
> > > 2 6 7.12331 0.592 20 0.19392
> > > 3 7 6.6927 0.546 2 0.147436
> > > 4 9 6.4763 0.513 20 0.124842
> > > 5 10 6.37183 0.486 2 0.112678
> > > 6 12 6.28601 0.465 20 0.106625
> > > Elapsed time is 8275.102153 seconds.
> > > -----------------------------------------------------------------------------------------
> > > Uhmmmm, as you see every parameters are similar but it is terrible when elapsed time for C mex file is even longer than that of m-file for above case.
> > >
> > > Hi Steve Lord,
> > > There are some things relating to what you say. In Jacobian matrix returned by the objective function, when i use Matlab to compare element by element between these two matrices of 2 files, Matlab indicates some different values among elements.
> > > However, if i run comparison two times of the same output Jacobian matrices , locations of difference at each running time are not the same. Then, i check some locations , some values of 2 elements are different at last digit as Matlab indicates. (for example: 4.559150742630094 vs 4.559150742630095), some values are the same (for example: 1.641766402933731 vs 1.641766402933731).
> > > Therefore, i think this is due to machine error between values returned by c mex file and m-file. But i can not still explain where is the problem relating computation time of lsqnonlin.
> > >
> > > Looking forward to your reply.
> > > Toan
> >
> > What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?
>
> Hi Rakesh,
> Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.
> Profiler of optimzation using m-file:
> http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg
> Profiler of optimzation using C mexfile:
> http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg
> As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.
> Is there any solution to reduce computation time for LevenbergMarquardt ?
>
> Toan

Hi Toan,
First, in your previous post you mentioned these overall time for two runs.
8275 seconds (C-mex file)
3240 seconds (MATLAB file)

The profiler result you posted shows these total time for testCM:
3249 seconds (C-mex file)
3234 seconds (MATLAB file).

They don't match. Are we looking at the same data?

In addition, the C-mex fie by itself takes very little time (1.188 seconds, 16 calls) compared to MATLAB file (18.90 in 16 calls). This is the expected speed up. However, if you look at the profiler results closely, the bottleneck appears to be the LevenbergMarquardt function and not your objective function.

In this problem, most of the time is spent in LevenbergMarquardt function (MATLAB or MEX) and the speed up you get from MEX is lost by 10% slowdown of LevenbergMarquardt (painful!). In your profiler results, I see that LevenbergMarquardt takes more time in C-mex case (~ 10% more), which I can investigate further if you can please provide an example to me.

BTW, What was your findings with 'trust-region-reflective' algorithm (which is the default algorithm choice)? Was that slower or did not work?

Thanks for posting the profiler results.
Rakesh

Subject: computation time of function lsqnonlin

From: Toan Cao

Date: 16 Sep, 2013 14:16:06

Message: 9 of 13

"Rakesh Kumar" wrote in message <l17052$78$1@newscl01ah.mathworks.com>...
> "Toan Cao" <toancv3010@gmail.com> wrote in message <l11tt6$c0q$1@newscl01ah.mathworks.com>...
> > "Rakesh Kumar" wrote in message <l0vnuf$kp5$1@newscl01ah.mathworks.com>...
> > > "Toan Cao" <toancv3010@gmail.com> wrote in message <l0vn9q$8pp$1@newscl01ah.mathworks.com>...
> > > > "Steven_Lord" <slord@mathworks.com> wrote in message <l0v8n7$so5$1@newscl01ah.mathworks.com>...
> > > > >
> > > > >
> > > > > "Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message
> > > > > news:l0v783$po$1@newscl01ah.mathworks.com...
> > > > > > "Toan Cao" <toancv3010@gmail.com> wrote in message
> > > > > > <l0u1au$ei4$1@newscl01ah.mathworks.com>...
> > > > > >> Hi,
> > > > > >> I write an objective function in both m-file (fileA1.m) and C mex file
> > > > > >> (fileA2.c). Given an input to this objective function, computation time
> > > > > >> in C mex file is 15 times faster than that of m- file. However, when i
> > > > > >> use the objective function in function lsqnonlin of Matlab, computation
> > > > > >> time which lsqnonlin gets a optimal solution is almost the same for both
> > > > > >> files (~101 minutes for C mex file and ~104 minutes for m-file).
> > > > > >> I hope when i use C mex file, i will save more time for optimization
> > > > > >> process, but it is not.
> > > > > >> Can you help me explain why lsqnonlin spends almost the same computation
> > > > > >> time?
> > > > > >> I would like to understand more how lsqnonlin of Matlab works to reduce
> > > > > >> its time?
> > > > > >> Thanks in advance!
> > > > > >>
> > > > > >> Toan
> > > > > >
> > > > > > Hi Toan,
> > > > > > This is unexpected. First, I would suggest to compare the #iterations and
> > > > > > #function evaluations in two scenarios (just to make sure we have same
> > > > > > implementation in MATLAB and C). If these numbers are similar, then I
> > > > > > would profile and see where the time is spent in two scenarios. Can you
> > > > > > report back here with your findings? Thanks!
> > > > >
> > > > > In addition, make sure you're comparing apples and oranges. Do fileA1.m and
> > > > > fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given
> > > > > the exact same inputs? Even a small difference in the return values could
> > > > > affect the path in the solution space the optimization routines follow, if I
> > > > > remember correctly.
> > > > >
> > > > > --
> > > > > Steve Lord
> > > > > slord@mathworks.com
> > > > > To contact Technical Support use the Contact Us link on
> > > > > http://www.mathworks.com
> > > >
> > > > Hi Rakesh,
> > > >
> > > > When i set just for 5 iteration , following outputs of optimization process make me more confusion:
> > > > -----------------------------------------------------------------------------------------
> > > > m-file:
> > > > First-Order Norm of
> > > > Iteration Func-count Residual optimality Lambda step
> > > > 0 1 9.02706 1.1 0.02
> > > > 1 4 8.15864 0.775 2 0.715913
> > > > 2 6 7.12331 0.592 20 0.19392
> > > > 3 7 6.6927 0.546 2 0.147436
> > > > 4 9 6.4763 0.513 20 0.124842
> > > > 5 10 6.37183 0.486 2 0.112678
> > > > 6 12 6.28601 0.465 20 0.106625
> > > > Elapsed time is 3240.325396 seconds.
> > > >
> > > > C mex file:
> > > >
> > > > First-Order Norm of
> > > > Iteration Func-count Residual optimality Lambda step
> > > > 0 1 9.02706 1.1 0.02
> > > > 1 4 8.15864 0.775 2 0.715913
> > > > 2 6 7.12331 0.592 20 0.19392
> > > > 3 7 6.6927 0.546 2 0.147436
> > > > 4 9 6.4763 0.513 20 0.124842
> > > > 5 10 6.37183 0.486 2 0.112678
> > > > 6 12 6.28601 0.465 20 0.106625
> > > > Elapsed time is 8275.102153 seconds.
> > > > -----------------------------------------------------------------------------------------
> > > > Uhmmmm, as you see every parameters are similar but it is terrible when elapsed time for C mex file is even longer than that of m-file for above case.
> > > >
> > > > Hi Steve Lord,
> > > > There are some things relating to what you say. In Jacobian matrix returned by the objective function, when i use Matlab to compare element by element between these two matrices of 2 files, Matlab indicates some different values among elements.
> > > > However, if i run comparison two times of the same output Jacobian matrices , locations of difference at each running time are not the same. Then, i check some locations , some values of 2 elements are different at last digit as Matlab indicates. (for example: 4.559150742630094 vs 4.559150742630095), some values are the same (for example: 1.641766402933731 vs 1.641766402933731).
> > > > Therefore, i think this is due to machine error between values returned by c mex file and m-file. But i can not still explain where is the problem relating computation time of lsqnonlin.
> > > >
> > > > Looking forward to your reply.
> > > > Toan
> > >
> > > What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?
> >
> > Hi Rakesh,
> > Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.
> > Profiler of optimzation using m-file:
> > http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg
> > Profiler of optimzation using C mexfile:
> > http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg
> > As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.
> > Is there any solution to reduce computation time for LevenbergMarquardt ?
> >
> > Toan
>
> Hi Toan,
> First, in your previous post you mentioned these overall time for two runs.
> 8275 seconds (C-mex file)
> 3240 seconds (MATLAB file)
>
> The profiler result you posted shows these total time for testCM:
> 3249 seconds (C-mex file)
> 3234 seconds (MATLAB file).
>
> They don't match. Are we looking at the same data?
>
> In addition, the C-mex fie by itself takes very little time (1.188 seconds, 16 calls) compared to MATLAB file (18.90 in 16 calls). This is the expected speed up. However, if you look at the profiler results closely, the bottleneck appears to be the LevenbergMarquardt function and not your objective function.
>
> In this problem, most of the time is spent in LevenbergMarquardt function (MATLAB or MEX) and the speed up you get from MEX is lost by 10% slowdown of LevenbergMarquardt (painful!). In your profiler results, I see that LevenbergMarquardt takes more time in C-mex case (~ 10% more), which I can investigate further if you can please provide an example to me.
>
> BTW, What was your findings with 'trust-region-reflective' algorithm (which is the default algorithm choice)? Was that slower or did not work?
>
> Thanks for posting the profiler results.
> Rakesh

Hi Rakesh,

The computation time of my two posts are different because i ran at two different time but the same number of iteration.
However, for the case where 8275 seconds for C-mex file and 3240 seconds for MATLAB file, i can not explain why computation time with C-mex file is bigger than although number of iteration is the same for both C mex file and m-file.
Relating to computing burden of Levenberg-Martquardt, i think Matlab is slow because of my big sparse Jacobian matrix (10000x12000) to find movement direction at each iteration.

Toan

Subject: computation time of function lsqnonlin

From: Eric Sampson

Date: 16 Sep, 2013 15:44:06

Message: 10 of 13

"Toan Cao" <toancv3010@gmail.com> wrote in message <l11tt6$c0q$1@newscl01ah.mathworks.com>...
> >
> > What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?
>
> Hi Rakesh,
> Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.
> Profiler of optimzation using m-file:
> http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg
> Profiler of optimzation using C mexfile:
> http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg
> As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.
> Is there any solution to reduce computation time for LevenbergMarquardt ?
>
> Toan

Can you post some more images from clicking on the Profiler report for the LevenbergMarquardt.m function, and its slowest lines/function calls/etc? Maybe there is a way to speed it up.

Subject: computation time of function lsqnonlin

From: Toan Cao

Date: 16 Sep, 2013 16:52:06

Message: 11 of 13

"Eric Sampson" wrote in message <l17906$ac5$1@newscl01ah.mathworks.com>...
> "Toan Cao" <toancv3010@gmail.com> wrote in message <l11tt6$c0q$1@newscl01ah.mathworks.com>...
> > >
> > > What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?
> >
> > Hi Rakesh,
> > Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.
> > Profiler of optimzation using m-file:
> > http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg
> > Profiler of optimzation using C mexfile:
> > http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg
> > As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.
> > Is there any solution to reduce computation time for LevenbergMarquardt ?
> >
> > Toan
>
> Can you post some more images from clicking on the Profiler report for the LevenbergMarquardt.m function, and its slowest lines/function calls/etc? Maybe there is a way to speed it up.

Hi Eric Sampson,
here is spending time of levenberg-marquardt function:
http://i906.photobucket.com/albums/ac264/toancv3010/levenbergTime_zps2f500238.jpg
I hope to receive some comments from you. Thank you.

Toan

Subject: computation time of function lsqnonlin

From: Rakesh Kumar

Date: 17 Sep, 2013 19:20:08

Message: 12 of 13

"Toan Cao" <toancv3010@gmail.com> wrote in message <l17cvm$558$1@newscl01ah.mathworks.com>...
> "Eric Sampson" wrote in message <l17906$ac5$1@newscl01ah.mathworks.com>...
> > "Toan Cao" <toancv3010@gmail.com> wrote in message <l11tt6$c0q$1@newscl01ah.mathworks.com>...
> > > >
> > > > What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?
> > >
> > > Hi Rakesh,
> > > Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.
> > > Profiler of optimzation using m-file:
> > > http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg
> > > Profiler of optimzation using C mexfile:
> > > http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg
> > > As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.
> > > Is there any solution to reduce computation time for LevenbergMarquardt ?
> > >
> > > Toan
> >
> > Can you post some more images from clicking on the Profiler report for the LevenbergMarquardt.m function, and its slowest lines/function calls/etc? Maybe there is a way to speed it up.
>
> Hi Eric Sampson,
> here is spending time of levenberg-marquardt function:
> http://i906.photobucket.com/albums/ac264/toancv3010/levenbergTime_zps2f500238.jpg
> I hope to receive some comments from you. Thank you.
>
> Toan

The backslash operator (where ~99% of the time is spent) has some heuristics to analyze and decide what method (and pre-processing) to use for solving the equation. It is likely that in these two cases, the backslash operator thinks the matrix structure (probably symmetry) is different and ends up with a different and expensive step. This is just a guess without looking at the actual matrices.

Rakesh

Subject: computation time of function lsqnonlin

From: Eric Sampson

Date: 17 Sep, 2013 21:24:09

Message: 13 of 13

"Rakesh Kumar" wrote in message <l1aa18$7n5$1@newscl01ah.mathworks.com>...
> "Toan Cao" <toancv3010@gmail.com> wrote in message <l17cvm$558$1@newscl01ah.mathworks.com>...
> > "Eric Sampson" wrote in message <l17906$ac5$1@newscl01ah.mathworks.com>...
> > > "Toan Cao" <toancv3010@gmail.com> wrote in message <l11tt6$c0q$1@newscl01ah.mathworks.com>...
> > > > >
> > > > > What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?
> > > >
> > > > Hi Rakesh,
> > > > Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.
> > > > Profiler of optimzation using m-file:
> > > > http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg
> > > > Profiler of optimzation using C mexfile:
> > > > http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg
> > > > As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.
> > > > Is there any solution to reduce computation time for LevenbergMarquardt ?
> > > >
> > > > Toan
> > >
> > > Can you post some more images from clicking on the Profiler report for the LevenbergMarquardt.m function, and its slowest lines/function calls/etc? Maybe there is a way to speed it up.
> >
> > Hi Eric Sampson,
> > here is spending time of levenberg-marquardt function:
> > http://i906.photobucket.com/albums/ac264/toancv3010/levenbergTime_zps2f500238.jpg
> > I hope to receive some comments from you. Thank you.
> >
> > Toan
>
> The backslash operator (where ~99% of the time is spent) has some heuristics to analyze and decide what method (and pre-processing) to use for solving the equation. It is likely that in these two cases, the backslash operator thinks the matrix structure (probably symmetry) is different and ends up with a different and expensive step. This is just a guess without looking at the actual matrices.
>
> Rakesh

I agree with Rakesh. If you want to pursue this matter further, then what I would suggest is to put a debugging breakpoint on line 178 of the LM function, run the code with the original M file objective function and then with the MEX objective function, and in each case when it stops at the breakpoint save the AugJac and AugRes variables to a .mat file. Then create a support request at The Mathworks, include the two M files, and tell them to run 'tic, AugJac \ AugRes;, toc' with the contents of each .mat file and ask for their help in explaining the difference. Good luck!

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us