MATLAB Newsgroup

Hi,

I write an objective function in both m-file (fileA1.m) and C mex file (fileA2.c). Given an input to this objective function, computation time in C mex file is 15 times faster than that of m- file. However, when i use the objective function in function lsqnonlin of Matlab, computation time which lsqnonlin gets a optimal solution is almost the same for both files (~101 minutes for C mex file and ~104 minutes for m-file).

I hope when i use C mex file, i will save more time for optimization process, but it is not.

Can you help me explain why lsqnonlin spends almost the same computation time?

I would like to understand more how lsqnonlin of Matlab works to reduce its time?

Thanks in advance!

Toan

"Toan Cao" <toancv3010@gmail.com> wrote in message <l0u1au$ei4$1@newscl01ah.mathworks.com>...

> Hi,

> I write an objective function in both m-file (fileA1.m) and C mex file (fileA2.c). Given an input to this objective function, computation time in C mex file is 15 times faster than that of m- file. However, when i use the objective function in function lsqnonlin of Matlab, computation time which lsqnonlin gets a optimal solution is almost the same for both files (~101 minutes for C mex file and ~104 minutes for m-file).

> I hope when i use C mex file, i will save more time for optimization process, but it is not.

> Can you help me explain why lsqnonlin spends almost the same computation time?

> I would like to understand more how lsqnonlin of Matlab works to reduce its time?

> Thanks in advance!

>

> Toan

Hi Toan,

This is unexpected.

First, I would suggest to compare the #iterations and #function evaluations in two scenarios (just to make sure we have same implementation in MATLAB and C). If these numbers are similar, then I would profile and see where the time is spent in two scenarios. Can you report back here with your findings? Thanks!

Rakesh

"Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message

news:l0v783$po$1@newscl01ah.mathworks.com...

> "Toan Cao" <toancv3010@gmail.com> wrote in message

> <l0u1au$ei4$1@newscl01ah.mathworks.com>...

>> Hi,

>> I write an objective function in both m-file (fileA1.m) and C mex file

>> (fileA2.c). Given an input to this objective function, computation time

>> in C mex file is 15 times faster than that of m- file. However, when i

>> use the objective function in function lsqnonlin of Matlab, computation

>> time which lsqnonlin gets a optimal solution is almost the same for both

>> files (~101 minutes for C mex file and ~104 minutes for m-file).

>> I hope when i use C mex file, i will save more time for optimization

>> process, but it is not.

>> Can you help me explain why lsqnonlin spends almost the same computation

>> time?

>> I would like to understand more how lsqnonlin of Matlab works to reduce

>> its time?

>> Thanks in advance!

>>

>> Toan

>

> Hi Toan,

> This is unexpected. First, I would suggest to compare the #iterations and

> #function evaluations in two scenarios (just to make sure we have same

> implementation in MATLAB and C). If these numbers are similar, then I

> would profile and see where the time is spent in two scenarios. Can you

> report back here with your findings? Thanks!

In addition, make sure you're comparing apples and oranges. Do fileA1.m and

fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given

the exact same inputs? Even a small difference in the return values could

affect the path in the solution space the optimization routines follow, if I

remember correctly.

--

Steve Lord

slord@mathworks.com

To contact Technical Support use the Contact Us link on

http://www.mathworks.com

"Steven_Lord" <slord@mathworks.com> wrote in message <l0v8n7$so5$1@newscl01ah.mathworks.com>...

>

>

> "Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message

> news:l0v783$po$1@newscl01ah.mathworks.com...

> > "Toan Cao" <toancv3010@gmail.com> wrote in message

> > <l0u1au$ei4$1@newscl01ah.mathworks.com>...

> >> Hi,

> >> I write an objective function in both m-file (fileA1.m) and C mex file

> >> (fileA2.c). Given an input to this objective function, computation time

> >> in C mex file is 15 times faster than that of m- file. However, when i

> >> use the objective function in function lsqnonlin of Matlab, computation

> >> time which lsqnonlin gets a optimal solution is almost the same for both

> >> files (~101 minutes for C mex file and ~104 minutes for m-file).

> >> I hope when i use C mex file, i will save more time for optimization

> >> process, but it is not.

> >> Can you help me explain why lsqnonlin spends almost the same computation

> >> time?

> >> I would like to understand more how lsqnonlin of Matlab works to reduce

> >> its time?

> >> Thanks in advance!

> >>

> >> Toan

> >

> > Hi Toan,

> > This is unexpected. First, I would suggest to compare the #iterations and

> > #function evaluations in two scenarios (just to make sure we have same

> > implementation in MATLAB and C). If these numbers are similar, then I

> > would profile and see where the time is spent in two scenarios. Can you

> > report back here with your findings? Thanks!

>

> In addition, make sure you're comparing apples and oranges. Do fileA1.m and

> fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given

> the exact same inputs? Even a small difference in the return values could

> affect the path in the solution space the optimization routines follow, if I

> remember correctly.

>

> --

> Steve Lord

> slord@mathworks.com

> To contact Technical Support use the Contact Us link on

> http://www.mathworks.com

Hi Rakesh,

When i set just for 5 iteration , following outputs of optimization process make me more confusion:

-----------------------------------------------------------------------------------------

m-file:

First-Order Norm of

Iteration Func-count Residual optimality Lambda step

0 1 9.02706 1.1 0.02

1 4 8.15864 0.775 2 0.715913

2 6 7.12331 0.592 20 0.19392

3 7 6.6927 0.546 2 0.147436

4 9 6.4763 0.513 20 0.124842

5 10 6.37183 0.486 2 0.112678

6 12 6.28601 0.465 20 0.106625

Elapsed time is 3240.325396 seconds.

C mex file:

First-Order Norm of

Iteration Func-count Residual optimality Lambda step

0 1 9.02706 1.1 0.02

1 4 8.15864 0.775 2 0.715913

2 6 7.12331 0.592 20 0.19392

3 7 6.6927 0.546 2 0.147436

4 9 6.4763 0.513 20 0.124842

5 10 6.37183 0.486 2 0.112678

6 12 6.28601 0.465 20 0.106625

Elapsed time is 8275.102153 seconds.

-----------------------------------------------------------------------------------------

Uhmmmm, as you see every parameters are similar but it is terrible when elapsed time for C mex file is even longer than that of m-file for above case.

Hi Steve Lord,

There are some things relating to what you say. In Jacobian matrix returned by the objective function, when i use Matlab to compare element by element between these two matrices of 2 files, Matlab indicates some different values among elements.

However, if i run comparison two times of the same output Jacobian matrices , locations of difference at each running time are not the same. Then, i check some locations , some values of 2 elements are different at last digit as Matlab indicates. (for example: 4.559150742630094 vs 4.559150742630095), some values are the same (for example: 1.641766402933731 vs 1.641766402933731).

Therefore, i think this is due to machine error between values returned by c mex file and m-file. But i can not still explain where is the problem relating computation time of lsqnonlin.

Looking forward to your reply.

Toan

"Toan Cao" <toancv3010@gmail.com> wrote in message <l0vn9q$8pp$1@newscl01ah.mathworks.com>...

> "Steven_Lord" <slord@mathworks.com> wrote in message <l0v8n7$so5$1@newscl01ah.mathworks.com>...

> >

> >

> > "Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message

> > news:l0v783$po$1@newscl01ah.mathworks.com...

> > > "Toan Cao" <toancv3010@gmail.com> wrote in message

> > > <l0u1au$ei4$1@newscl01ah.mathworks.com>...

> > >> Hi,

> > >> I write an objective function in both m-file (fileA1.m) and C mex file

> > >> (fileA2.c). Given an input to this objective function, computation time

> > >> in C mex file is 15 times faster than that of m- file. However, when i

> > >> use the objective function in function lsqnonlin of Matlab, computation

> > >> time which lsqnonlin gets a optimal solution is almost the same for both

> > >> files (~101 minutes for C mex file and ~104 minutes for m-file).

> > >> I hope when i use C mex file, i will save more time for optimization

> > >> process, but it is not.

> > >> Can you help me explain why lsqnonlin spends almost the same computation

> > >> time?

> > >> I would like to understand more how lsqnonlin of Matlab works to reduce

> > >> its time?

> > >> Thanks in advance!

> > >>

> > >> Toan

> > >

> > > Hi Toan,

> > > This is unexpected. First, I would suggest to compare the #iterations and

> > > #function evaluations in two scenarios (just to make sure we have same

> > > implementation in MATLAB and C). If these numbers are similar, then I

> > > would profile and see where the time is spent in two scenarios. Can you

> > > report back here with your findings? Thanks!

> >

> > In addition, make sure you're comparing apples and oranges. Do fileA1.m and

> > fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given

> > the exact same inputs? Even a small difference in the return values could

> > affect the path in the solution space the optimization routines follow, if I

> > remember correctly.

> >

> > --

> > Steve Lord

> > slord@mathworks.com

> > To contact Technical Support use the Contact Us link on

> > http://www.mathworks.com

>

> Hi Rakesh,

>

> When i set just for 5 iteration , following outputs of optimization process make me more confusion:

> -----------------------------------------------------------------------------------------

> m-file:

> First-Order Norm of

> Iteration Func-count Residual optimality Lambda step

> 0 1 9.02706 1.1 0.02

> 1 4 8.15864 0.775 2 0.715913

> 2 6 7.12331 0.592 20 0.19392

> 3 7 6.6927 0.546 2 0.147436

> 4 9 6.4763 0.513 20 0.124842

> 5 10 6.37183 0.486 2 0.112678

> 6 12 6.28601 0.465 20 0.106625

> Elapsed time is 3240.325396 seconds.

>

> C mex file:

>

> First-Order Norm of

> Iteration Func-count Residual optimality Lambda step

> 0 1 9.02706 1.1 0.02

> 1 4 8.15864 0.775 2 0.715913

> 2 6 7.12331 0.592 20 0.19392

> 3 7 6.6927 0.546 2 0.147436

> 4 9 6.4763 0.513 20 0.124842

> 5 10 6.37183 0.486 2 0.112678

> 6 12 6.28601 0.465 20 0.106625

> Elapsed time is 8275.102153 seconds.

> -----------------------------------------------------------------------------------------

> Uhmmmm, as you see every parameters are similar but it is terrible when elapsed time for C mex file is even longer than that of m-file for above case.

>

> Hi Steve Lord,

> There are some things relating to what you say. In Jacobian matrix returned by the objective function, when i use Matlab to compare element by element between these two matrices of 2 files, Matlab indicates some different values among elements.

> However, if i run comparison two times of the same output Jacobian matrices , locations of difference at each running time are not the same. Then, i check some locations , some values of 2 elements are different at last digit as Matlab indicates. (for example: 4.559150742630094 vs 4.559150742630095), some values are the same (for example: 1.641766402933731 vs 1.641766402933731).

> Therefore, i think this is due to machine error between values returned by c mex file and m-file. But i can not still explain where is the problem relating computation time of lsqnonlin.

>

> Looking forward to your reply.

> Toan

What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?

> "Steven_Lord" <slord@mathworks.com> wrote in message <l0v8n7$so5$1@newscl01ah.mathworks.com>...

> >

> >

> > "Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message

> > news:l0v783$po$1@newscl01ah.mathworks.com...

> > > "Toan Cao" <toancv3010@gmail.com> wrote in message

> > > <l0u1au$ei4$1@newscl01ah.mathworks.com>...

> > >> Hi,

> > >> I write an objective function in both m-file (fileA1.m) and C mex file

> > >> (fileA2.c). Given an input to this objective function, computation time

> > >> in C mex file is 15 times faster than that of m- file. However, when i

> > >> use the objective function in function lsqnonlin of Matlab, computation

> > >> time which lsqnonlin gets a optimal solution is almost the same for both

> > >> files (~101 minutes for C mex file and ~104 minutes for m-file).

> > >> I hope when i use C mex file, i will save more time for optimization

> > >> process, but it is not.

> > >> Can you help me explain why lsqnonlin spends almost the same computation

> > >> time?

> > >> I would like to understand more how lsqnonlin of Matlab works to reduce

> > >> its time?

> > >> Thanks in advance!

> > >>

> > >> Toan

> > >

> > > Hi Toan,

> > > This is unexpected. First, I would suggest to compare the #iterations and

> > > #function evaluations in two scenarios (just to make sure we have same

> > > implementation in MATLAB and C). If these numbers are similar, then I

> > > would profile and see where the time is spent in two scenarios. Can you

> > > report back here with your findings? Thanks!

> >

> > In addition, make sure you're comparing apples and oranges. Do fileA1.m and

> > fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given

> > the exact same inputs? Even a small difference in the return values could

> > affect the path in the solution space the optimization routines follow, if I

> > remember correctly.

> >

> > --

> > Steve Lord

> > slord@mathworks.com

> > To contact Technical Support use the Contact Us link on

> > http://www.mathworks.com

>

> Hi Rakesh,

>

> When i set just for 5 iteration , following outputs of optimization process make me more confusion:

> -----------------------------------------------------------------------------------------

> m-file:

> First-Order Norm of

> Iteration Func-count Residual optimality Lambda step

> 0 1 9.02706 1.1 0.02

> 1 4 8.15864 0.775 2 0.715913

> 2 6 7.12331 0.592 20 0.19392

> 3 7 6.6927 0.546 2 0.147436

> 4 9 6.4763 0.513 20 0.124842

> 5 10 6.37183 0.486 2 0.112678

> 6 12 6.28601 0.465 20 0.106625

> Elapsed time is 3240.325396 seconds.

>

> C mex file:

>

> First-Order Norm of

> Iteration Func-count Residual optimality Lambda step

> 0 1 9.02706 1.1 0.02

> 1 4 8.15864 0.775 2 0.715913

> 2 6 7.12331 0.592 20 0.19392

> 3 7 6.6927 0.546 2 0.147436

> 4 9 6.4763 0.513 20 0.124842

> 5 10 6.37183 0.486 2 0.112678

> 6 12 6.28601 0.465 20 0.106625

> Elapsed time is 8275.102153 seconds.

> -----------------------------------------------------------------------------------------

> Uhmmmm, as you see every parameters are similar but it is terrible when elapsed time for C mex file is even longer than that of m-file for above case.

>

> Hi Steve Lord,

> There are some things relating to what you say. In Jacobian matrix returned by the objective function, when i use Matlab to compare element by element between these two matrices of 2 files, Matlab indicates some different values among elements.

> However, if i run comparison two times of the same output Jacobian matrices , locations of difference at each running time are not the same. Then, i check some locations , some values of 2 elements are different at last digit as Matlab indicates. (for example: 4.559150742630094 vs 4.559150742630095), some values are the same (for example: 1.641766402933731 vs 1.641766402933731).

> Therefore, i think this is due to machine error between values returned by c mex file and m-file. But i can not still explain where is the problem relating computation time of lsqnonlin.

>

> Looking forward to your reply.

> Toan

What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?

"Rakesh Kumar" wrote in message <l0vnuf$kp5$1@newscl01ah.mathworks.com>...

> "Toan Cao" <toancv3010@gmail.com> wrote in message <l0vn9q$8pp$1@newscl01ah.mathworks.com>...

> > "Steven_Lord" <slord@mathworks.com> wrote in message <l0v8n7$so5$1@newscl01ah.mathworks.com>...

> > >

> > >

> > > "Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message

> > > news:l0v783$po$1@newscl01ah.mathworks.com...

> > > > "Toan Cao" <toancv3010@gmail.com> wrote in message

> > > > <l0u1au$ei4$1@newscl01ah.mathworks.com>...

> > > >> Hi,

> > > >> I write an objective function in both m-file (fileA1.m) and C mex file

> > > >> (fileA2.c). Given an input to this objective function, computation time

> > > >> in C mex file is 15 times faster than that of m- file. However, when i

> > > >> use the objective function in function lsqnonlin of Matlab, computation

> > > >> time which lsqnonlin gets a optimal solution is almost the same for both

> > > >> files (~101 minutes for C mex file and ~104 minutes for m-file).

> > > >> I hope when i use C mex file, i will save more time for optimization

> > > >> process, but it is not.

> > > >> Can you help me explain why lsqnonlin spends almost the same computation

> > > >> time?

> > > >> I would like to understand more how lsqnonlin of Matlab works to reduce

> > > >> its time?

> > > >> Thanks in advance!

> > > >>

> > > >> Toan

> > > >

> > > > Hi Toan,

> > > > This is unexpected. First, I would suggest to compare the #iterations and

> > > > #function evaluations in two scenarios (just to make sure we have same

> > > > implementation in MATLAB and C). If these numbers are similar, then I

> > > > would profile and see where the time is spent in two scenarios. Can you

> > > > report back here with your findings? Thanks!

> > >

> > > In addition, make sure you're comparing apples and oranges. Do fileA1.m and

> > > fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given

> > > the exact same inputs? Even a small difference in the return values could

> > > affect the path in the solution space the optimization routines follow, if I

> > > remember correctly.

> > >

> > > --

> > > Steve Lord

> > > slord@mathworks.com

> > > To contact Technical Support use the Contact Us link on

> > > http://www.mathworks.com

> >

> > Hi Rakesh,

> >

> > When i set just for 5 iteration , following outputs of optimization process make me more confusion:

> > -----------------------------------------------------------------------------------------

> > m-file:

> > First-Order Norm of

> > Iteration Func-count Residual optimality Lambda step

> > 0 1 9.02706 1.1 0.02

> > 1 4 8.15864 0.775 2 0.715913

> > 2 6 7.12331 0.592 20 0.19392

> > 3 7 6.6927 0.546 2 0.147436

> > 4 9 6.4763 0.513 20 0.124842

> > 5 10 6.37183 0.486 2 0.112678

> > 6 12 6.28601 0.465 20 0.106625

> > Elapsed time is 3240.325396 seconds.

> >

> > C mex file:

> >

> > First-Order Norm of

> > Iteration Func-count Residual optimality Lambda step

> > 0 1 9.02706 1.1 0.02

> > 1 4 8.15864 0.775 2 0.715913

> > 2 6 7.12331 0.592 20 0.19392

> > 3 7 6.6927 0.546 2 0.147436

> > 4 9 6.4763 0.513 20 0.124842

> > 5 10 6.37183 0.486 2 0.112678

> > 6 12 6.28601 0.465 20 0.106625

> > Elapsed time is 8275.102153 seconds.

> > -----------------------------------------------------------------------------------------

> > Uhmmmm, as you see every parameters are similar but it is terrible when elapsed time for C mex file is even longer than that of m-file for above case.

> >

> > Hi Steve Lord,

> > There are some things relating to what you say. In Jacobian matrix returned by the objective function, when i use Matlab to compare element by element between these two matrices of 2 files, Matlab indicates some different values among elements.

> > However, if i run comparison two times of the same output Jacobian matrices , locations of difference at each running time are not the same. Then, i check some locations , some values of 2 elements are different at last digit as Matlab indicates. (for example: 4.559150742630094 vs 4.559150742630095), some values are the same (for example: 1.641766402933731 vs 1.641766402933731).

> > Therefore, i think this is due to machine error between values returned by c mex file and m-file. But i can not still explain where is the problem relating computation time of lsqnonlin.

> >

> > Looking forward to your reply.

> > Toan

>

> What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?

Hi Rakesh,

Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.

Profiler of optimzation using m-file:

http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg

Profiler of optimzation using C mexfile:

http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg

As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.

Is there any solution to reduce computation time for LevenbergMarquardt ?

Toan

"Toan Cao" <toancv3010@gmail.com> wrote in message <l11tt6$c0q$1@newscl01ah.mathworks.com>...

> "Rakesh Kumar" wrote in message <l0vnuf$kp5$1@newscl01ah.mathworks.com>...

> > "Toan Cao" <toancv3010@gmail.com> wrote in message <l0vn9q$8pp$1@newscl01ah.mathworks.com>...

> > > "Steven_Lord" <slord@mathworks.com> wrote in message <l0v8n7$so5$1@newscl01ah.mathworks.com>...

> > > >

> > > >

> > > > "Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message

> > > > news:l0v783$po$1@newscl01ah.mathworks.com...

> > > > > "Toan Cao" <toancv3010@gmail.com> wrote in message

> > > > > <l0u1au$ei4$1@newscl01ah.mathworks.com>...

> > > > >> Hi,

> > > > >> I write an objective function in both m-file (fileA1.m) and C mex file

> > > > >> (fileA2.c). Given an input to this objective function, computation time

> > > > >> in C mex file is 15 times faster than that of m- file. However, when i

> > > > >> use the objective function in function lsqnonlin of Matlab, computation

> > > > >> time which lsqnonlin gets a optimal solution is almost the same for both

> > > > >> files (~101 minutes for C mex file and ~104 minutes for m-file).

> > > > >> I hope when i use C mex file, i will save more time for optimization

> > > > >> process, but it is not.

> > > > >> Can you help me explain why lsqnonlin spends almost the same computation

> > > > >> time?

> > > > >> I would like to understand more how lsqnonlin of Matlab works to reduce

> > > > >> its time?

> > > > >> Thanks in advance!

> > > > >>

> > > > >> Toan

> > > > >

> > > > > Hi Toan,

> > > > > This is unexpected. First, I would suggest to compare the #iterations and

> > > > > #function evaluations in two scenarios (just to make sure we have same

> > > > > implementation in MATLAB and C). If these numbers are similar, then I

> > > > > would profile and see where the time is spent in two scenarios. Can you

> > > > > report back here with your findings? Thanks!

> > > >

> > > > In addition, make sure you're comparing apples and oranges. Do fileA1.m and

> > > > fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given

> > > > the exact same inputs? Even a small difference in the return values could

> > > > affect the path in the solution space the optimization routines follow, if I

> > > > remember correctly.

> > > >

> > > > --

> > > > Steve Lord

> > > > slord@mathworks.com

> > > > To contact Technical Support use the Contact Us link on

> > > > http://www.mathworks.com

> > >

> > > Hi Rakesh,

> > >

> > > When i set just for 5 iteration , following outputs of optimization process make me more confusion:

> > > -----------------------------------------------------------------------------------------

> > > m-file:

> > > First-Order Norm of

> > > Iteration Func-count Residual optimality Lambda step

> > > 0 1 9.02706 1.1 0.02

> > > 1 4 8.15864 0.775 2 0.715913

> > > 2 6 7.12331 0.592 20 0.19392

> > > 3 7 6.6927 0.546 2 0.147436

> > > 4 9 6.4763 0.513 20 0.124842

> > > 5 10 6.37183 0.486 2 0.112678

> > > 6 12 6.28601 0.465 20 0.106625

> > > Elapsed time is 3240.325396 seconds.

> > >

> > > C mex file:

> > >

> > > First-Order Norm of

> > > Iteration Func-count Residual optimality Lambda step

> > > 0 1 9.02706 1.1 0.02

> > > 1 4 8.15864 0.775 2 0.715913

> > > 2 6 7.12331 0.592 20 0.19392

> > > 3 7 6.6927 0.546 2 0.147436

> > > 4 9 6.4763 0.513 20 0.124842

> > > 5 10 6.37183 0.486 2 0.112678

> > > 6 12 6.28601 0.465 20 0.106625

> > > Elapsed time is 8275.102153 seconds.

> > > -----------------------------------------------------------------------------------------

> > > Uhmmmm, as you see every parameters are similar but it is terrible when elapsed time for C mex file is even longer than that of m-file for above case.

> > >

> > > Hi Steve Lord,

> > > There are some things relating to what you say. In Jacobian matrix returned by the objective function, when i use Matlab to compare element by element between these two matrices of 2 files, Matlab indicates some different values among elements.

> > > However, if i run comparison two times of the same output Jacobian matrices , locations of difference at each running time are not the same. Then, i check some locations , some values of 2 elements are different at last digit as Matlab indicates. (for example: 4.559150742630094 vs 4.559150742630095), some values are the same (for example: 1.641766402933731 vs 1.641766402933731).

> > > Therefore, i think this is due to machine error between values returned by c mex file and m-file. But i can not still explain where is the problem relating computation time of lsqnonlin.

> > >

> > > Looking forward to your reply.

> > > Toan

> >

> > What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?

>

> Hi Rakesh,

> Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.

> Profiler of optimzation using m-file:

> http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg

> Profiler of optimzation using C mexfile:

> http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg

> As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.

> Is there any solution to reduce computation time for LevenbergMarquardt ?

>

> Toan

Hi Toan,

First, in your previous post you mentioned these overall time for two runs.

8275 seconds (C-mex file)

3240 seconds (MATLAB file)

The profiler result you posted shows these total time for testCM:

3249 seconds (C-mex file)

3234 seconds (MATLAB file).

They don't match. Are we looking at the same data?

In addition, the C-mex fie by itself takes very little time (1.188 seconds, 16 calls) compared to MATLAB file (18.90 in 16 calls). This is the expected speed up. However, if you look at the profiler results closely, the bottleneck appears to be the LevenbergMarquardt function and not your objective function.

In this problem, most of the time is spent in LevenbergMarquardt function (MATLAB or MEX) and the speed up you get from MEX is lost by 10% slowdown of LevenbergMarquardt (painful!). In your profiler results, I see that LevenbergMarquardt takes more time in C-mex case (~ 10% more), which I can investigate further if you can please provide an example to me.

BTW, What was your findings with 'trust-region-reflective' algorithm (which is the default algorithm choice)? Was that slower or did not work?

Thanks for posting the profiler results.

Rakesh

"Rakesh Kumar" wrote in message <l17052$78$1@newscl01ah.mathworks.com>...

> "Toan Cao" <toancv3010@gmail.com> wrote in message <l11tt6$c0q$1@newscl01ah.mathworks.com>...

> > "Rakesh Kumar" wrote in message <l0vnuf$kp5$1@newscl01ah.mathworks.com>...

> > > "Toan Cao" <toancv3010@gmail.com> wrote in message <l0vn9q$8pp$1@newscl01ah.mathworks.com>...

> > > > "Steven_Lord" <slord@mathworks.com> wrote in message <l0v8n7$so5$1@newscl01ah.mathworks.com>...

> > > > >

> > > > >

> > > > > "Rakesh Kumar" <r.kumar_spamhole@mathworks.com> wrote in message

> > > > > news:l0v783$po$1@newscl01ah.mathworks.com...

> > > > > > "Toan Cao" <toancv3010@gmail.com> wrote in message

> > > > > > <l0u1au$ei4$1@newscl01ah.mathworks.com>...

> > > > > >> Hi,

> > > > > >> I write an objective function in both m-file (fileA1.m) and C mex file

> > > > > >> (fileA2.c). Given an input to this objective function, computation time

> > > > > >> in C mex file is 15 times faster than that of m- file. However, when i

> > > > > >> use the objective function in function lsqnonlin of Matlab, computation

> > > > > >> time which lsqnonlin gets a optimal solution is almost the same for both

> > > > > >> files (~101 minutes for C mex file and ~104 minutes for m-file).

> > > > > >> I hope when i use C mex file, i will save more time for optimization

> > > > > >> process, but it is not.

> > > > > >> Can you help me explain why lsqnonlin spends almost the same computation

> > > > > >> time?

> > > > > >> I would like to understand more how lsqnonlin of Matlab works to reduce

> > > > > >> its time?

> > > > > >> Thanks in advance!

> > > > > >>

> > > > > >> Toan

> > > > > >

> > > > > > Hi Toan,

> > > > > > This is unexpected. First, I would suggest to compare the #iterations and

> > > > > > #function evaluations in two scenarios (just to make sure we have same

> > > > > > implementation in MATLAB and C). If these numbers are similar, then I

> > > > > > would profile and see where the time is spent in two scenarios. Can you

> > > > > > report back here with your findings? Thanks!

> > > > >

> > > > > In addition, make sure you're comparing apples and oranges. Do fileA1.m and

> > > > > fileA2.$MEXEXT return the EXACT SAME, down to the last bit, results given

> > > > > the exact same inputs? Even a small difference in the return values could

> > > > > affect the path in the solution space the optimization routines follow, if I

> > > > > remember correctly.

> > > > >

> > > > > --

> > > > > Steve Lord

> > > > > slord@mathworks.com

> > > > > To contact Technical Support use the Contact Us link on

> > > > > http://www.mathworks.com

> > > >

> > > > Hi Rakesh,

> > > >

> > > > When i set just for 5 iteration , following outputs of optimization process make me more confusion:

> > > > -----------------------------------------------------------------------------------------

> > > > m-file:

> > > > First-Order Norm of

> > > > Iteration Func-count Residual optimality Lambda step

> > > > 0 1 9.02706 1.1 0.02

> > > > 1 4 8.15864 0.775 2 0.715913

> > > > 2 6 7.12331 0.592 20 0.19392

> > > > 3 7 6.6927 0.546 2 0.147436

> > > > 4 9 6.4763 0.513 20 0.124842

> > > > 5 10 6.37183 0.486 2 0.112678

> > > > 6 12 6.28601 0.465 20 0.106625

> > > > Elapsed time is 3240.325396 seconds.

> > > >

> > > > C mex file:

> > > >

> > > > First-Order Norm of

> > > > Iteration Func-count Residual optimality Lambda step

> > > > 0 1 9.02706 1.1 0.02

> > > > 1 4 8.15864 0.775 2 0.715913

> > > > 2 6 7.12331 0.592 20 0.19392

> > > > 3 7 6.6927 0.546 2 0.147436

> > > > 4 9 6.4763 0.513 20 0.124842

> > > > 5 10 6.37183 0.486 2 0.112678

> > > > 6 12 6.28601 0.465 20 0.106625

> > > > Elapsed time is 8275.102153 seconds.

> > > > -----------------------------------------------------------------------------------------

> > > > Uhmmmm, as you see every parameters are similar but it is terrible when elapsed time for C mex file is even longer than that of m-file for above case.

> > > >

> > > > Hi Steve Lord,

> > > > There are some things relating to what you say. In Jacobian matrix returned by the objective function, when i use Matlab to compare element by element between these two matrices of 2 files, Matlab indicates some different values among elements.

> > > > However, if i run comparison two times of the same output Jacobian matrices , locations of difference at each running time are not the same. Then, i check some locations , some values of 2 elements are different at last digit as Matlab indicates. (for example: 4.559150742630094 vs 4.559150742630095), some values are the same (for example: 1.641766402933731 vs 1.641766402933731).

> > > > Therefore, i think this is due to machine error between values returned by c mex file and m-file. But i can not still explain where is the problem relating computation time of lsqnonlin.

> > > >

> > > > Looking forward to your reply.

> > > > Toan

> > >

> > > What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?

> >

> > Hi Rakesh,

> > Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.

> > Profiler of optimzation using m-file:

> > http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg

> > Profiler of optimzation using C mexfile:

> > http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg

> > As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.

> > Is there any solution to reduce computation time for LevenbergMarquardt ?

> >

> > Toan

>

> Hi Toan,

> First, in your previous post you mentioned these overall time for two runs.

> 8275 seconds (C-mex file)

> 3240 seconds (MATLAB file)

>

> The profiler result you posted shows these total time for testCM:

> 3249 seconds (C-mex file)

> 3234 seconds (MATLAB file).

>

> They don't match. Are we looking at the same data?

>

> In addition, the C-mex fie by itself takes very little time (1.188 seconds, 16 calls) compared to MATLAB file (18.90 in 16 calls). This is the expected speed up. However, if you look at the profiler results closely, the bottleneck appears to be the LevenbergMarquardt function and not your objective function.

>

> In this problem, most of the time is spent in LevenbergMarquardt function (MATLAB or MEX) and the speed up you get from MEX is lost by 10% slowdown of LevenbergMarquardt (painful!). In your profiler results, I see that LevenbergMarquardt takes more time in C-mex case (~ 10% more), which I can investigate further if you can please provide an example to me.

>

> BTW, What was your findings with 'trust-region-reflective' algorithm (which is the default algorithm choice)? Was that slower or did not work?

>

> Thanks for posting the profiler results.

> Rakesh

Hi Rakesh,

The computation time of my two posts are different because i ran at two different time but the same number of iteration.

However, for the case where 8275 seconds for C-mex file and 3240 seconds for MATLAB file, i can not explain why computation time with C-mex file is bigger than although number of iteration is the same for both C mex file and m-file.

Relating to computing burden of Levenberg-Martquardt, i think Matlab is slow because of my big sparse Jacobian matrix (10000x12000) to find movement direction at each iteration.

Toan

"Toan Cao" <toancv3010@gmail.com> wrote in message <l11tt6$c0q$1@newscl01ah.mathworks.com>...

> >

> > What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?

>

> Hi Rakesh,

> Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.

> Profiler of optimzation using m-file:

> http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg

> Profiler of optimzation using C mexfile:

> http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg

> As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.

> Is there any solution to reduce computation time for LevenbergMarquardt ?

>

> Toan

Can you post some more images from clicking on the Profiler report for the LevenbergMarquardt.m function, and its slowest lines/function calls/etc? Maybe there is a way to speed it up.

"Eric Sampson" wrote in message <l17906$ac5$1@newscl01ah.mathworks.com>...

> "Toan Cao" <toancv3010@gmail.com> wrote in message <l11tt6$c0q$1@newscl01ah.mathworks.com>...

> > >

> > > What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?

> >

> > Hi Rakesh,

> > Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.

> > Profiler of optimzation using m-file:

> > http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg

> > Profiler of optimzation using C mexfile:

> > http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg

> > As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.

> > Is there any solution to reduce computation time for LevenbergMarquardt ?

> >

> > Toan

>

> Can you post some more images from clicking on the Profiler report for the LevenbergMarquardt.m function, and its slowest lines/function calls/etc? Maybe there is a way to speed it up.

Hi Eric Sampson,

here is spending time of levenberg-marquardt function:

http://i906.photobucket.com/albums/ac264/toancv3010/levenbergTime_zps2f500238.jpg

I hope to receive some comments from you. Thank you.

Toan

"Toan Cao" <toancv3010@gmail.com> wrote in message <l17cvm$558$1@newscl01ah.mathworks.com>...

> "Eric Sampson" wrote in message <l17906$ac5$1@newscl01ah.mathworks.com>...

> > "Toan Cao" <toancv3010@gmail.com> wrote in message <l11tt6$c0q$1@newscl01ah.mathworks.com>...

> > > >

> > > > What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?

> > >

> > > Hi Rakesh,

> > > Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.

> > > Profiler of optimzation using m-file:

> > > http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg

> > > Profiler of optimzation using C mexfile:

> > > http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg

> > > As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.

> > > Is there any solution to reduce computation time for LevenbergMarquardt ?

> > >

> > > Toan

> >

> > Can you post some more images from clicking on the Profiler report for the LevenbergMarquardt.m function, and its slowest lines/function calls/etc? Maybe there is a way to speed it up.

>

> Hi Eric Sampson,

> here is spending time of levenberg-marquardt function:

> http://i906.photobucket.com/albums/ac264/toancv3010/levenbergTime_zps2f500238.jpg

> I hope to receive some comments from you. Thank you.

>

> Toan

The backslash operator (where ~99% of the time is spent) has some heuristics to analyze and decide what method (and pre-processing) to use for solving the equation. It is likely that in these two cases, the backslash operator thinks the matrix structure (probably symmetry) is different and ends up with a different and expensive step. This is just a guess without looking at the actual matrices.

Rakesh

"Rakesh Kumar" wrote in message <l1aa18$7n5$1@newscl01ah.mathworks.com>...

> "Toan Cao" <toancv3010@gmail.com> wrote in message <l17cvm$558$1@newscl01ah.mathworks.com>...

> > "Eric Sampson" wrote in message <l17906$ac5$1@newscl01ah.mathworks.com>...

> > > "Toan Cao" <toancv3010@gmail.com> wrote in message <l11tt6$c0q$1@newscl01ah.mathworks.com>...

> > > > >

> > > > > What about the profiler report? Can you post the profiler report for top 10 self-time for functions and how many times they are called?

> > > >

> > > > Hi Rakesh,

> > > > Here my profiler when i run two optimization processes, where testCM is m-file storing program of optimization, the object function in m-file is MydeformfuncSub7Ext, and in C mex file is MydeformfuncSub7ExtC.

> > > > Profiler of optimzation using m-file:

> > > > http://i906.photobucket.com/albums/ac264/toancv3010/Mtime_zps92ae907a.jpg

> > > > Profiler of optimzation using C mexfile:

> > > > http://i906.photobucket.com/albums/ac264/toancv3010/Ctime_zpsbbbc5fb3.jpg

> > > > As seen, subfunction LevenbergMarquardt consumes almost time of optimization process.

> > > > Is there any solution to reduce computation time for LevenbergMarquardt ?

> > > >

> > > > Toan

> > >

> > > Can you post some more images from clicking on the Profiler report for the LevenbergMarquardt.m function, and its slowest lines/function calls/etc? Maybe there is a way to speed it up.

> >

> > Hi Eric Sampson,

> > here is spending time of levenberg-marquardt function:

> > http://i906.photobucket.com/albums/ac264/toancv3010/levenbergTime_zps2f500238.jpg

> > I hope to receive some comments from you. Thank you.

> >

> > Toan

>

> The backslash operator (where ~99% of the time is spent) has some heuristics to analyze and decide what method (and pre-processing) to use for solving the equation. It is likely that in these two cases, the backslash operator thinks the matrix structure (probably symmetry) is different and ends up with a different and expensive step. This is just a guess without looking at the actual matrices.

>

> Rakesh

I agree with Rakesh. If you want to pursue this matter further, then what I would suggest is to put a debugging breakpoint on line 178 of the LM function, run the code with the original M file objective function and then with the MEX objective function, and in each case when it stops at the breakpoint save the AugJac and AugRes variables to a .mat file. Then create a support request at The Mathworks, include the two M files, and tell them to run 'tic, AugJac \ AugRes;, toc' with the contents of each .mat file and ask for their help in explaining the difference. Good luck!

You can think of your watch list as threads that you have bookmarked.

You can add tags, authors, threads, and even search results to your watch list. This way you can easily keep track of topics that you're interested in. To view your watch list, click on the "My Newsreader" link.

To add items to your watch list, click the "add to watch list" link at the bottom of any page.

To add search criteria to your watch list, search for the desired term in the search box. Click on the "Add this search to my watch list" link on the search results page.

You can also add a tag to your watch list by searching for the tag with the directive "tag:tag_name" where tag_name is the name of the tag you would like to watch.

To add an author to your watch list, go to the author's profile page and click on the "Add this author to my watch list" link at the top of the page. You can also add an author to your watch list by going to a thread that the author has posted to and clicking on the "Add this author to my watch list" link. You will be notified whenever the author makes a post.

To add a thread to your watch list, go to the thread page and click the "Add this thread to my watch list" link at the top of the page.

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

The newsgroups are a worldwide forum that is open to everyone. Newsgroups are used to discuss a huge range of topics, make announcements, and trade files.

Discussions are threaded, or grouped in a way that allows you to read a posted message and all of its replies in chronological order. This makes it easy to follow the thread of the conversation, and to see what’s already been said before you post your own reply or make a new posting.

Newsgroup content is distributed by servers hosted by various organizations on the Internet. Messages are exchanged and managed using open-standard protocols. No single entity “owns” the newsgroups.

There are thousands of newsgroups, each addressing a single topic or area of interest. The MATLAB Central Newsreader posts and displays messages in the comp.soft-sys.matlab newsgroup.

**MATLAB Central**

You can use the integrated newsreader at the MATLAB Central website to read and post messages in this newsgroup. MATLAB Central is hosted by MathWorks.

Messages posted through the MATLAB Central Newsreader are seen by everyone using the newsgroups, regardless of how they access the newsgroups. There are several advantages to using MATLAB Central.

**One Account**

Your MATLAB Central account is tied to your MathWorks Account for easy access.

**Use the Email Address of Your Choice**

The MATLAB Central Newsreader allows you to define an alternative email address as your posting address, avoiding clutter in your primary mailbox and reducing spam.

**Spam Control**

Most newsgroup spam is filtered out by the MATLAB Central Newsreader.

**Tagging**

Messages can be tagged with a relevant label by any signed-in user. Tags can be used as keywords to find particular files of interest, or as a way to categorize your bookmarked postings. You may choose to allow others to view your tags, and you can view or search others’ tags as well as those of the community at large. Tagging provides a way to see both the big trends and the smaller, more obscure ideas and applications.

**Watch lists**

Setting up watch lists allows you to be notified of updates made to postings selected by author, thread, or any search variable. Your watch list notifications can be sent by email (daily digest or immediate), displayed in My Newsreader, or sent via RSS feed.

- Use a newsreader through your school, employer, or internet service provider
- Pay for newsgroup access from a commercial provider
- Use Google Groups
- Mathforum.org provides a newsreader with access to the comp.soft sys.matlab newsgroup
- Run your own server. For typical instructions, see: http://www.slyck.com/ng.php?page=2