Thread Subject: how can I save optimal parameters during the iterations?

Subject: how can I save optimal parameters during the iterations?

From: Didi Cvet

Date: 24 Nov, 2009 15:53:19

Message: 1 of 6

I have noticed that after some iteration (sometimes second, sometimes fifth) fmincon finds parameters that give minimal objective function. But when the iterations are finished i.e. fmincon is finished with exit flag
>>Maximum number of iterations exceeded;
 increase OPTIONS.MaxIter.<<
or
>>Maximum number of function evaluations exceeded;
 increase OPTIONS.MaxFunEvals.<<
I get initial values as optimal because during the iteration fmincon enters in some loop where it can't update Hessian matrix.
Is there any way to get parameters from iteration where objective function is minimal not the ones that fmincon returns?

Subject: how can I save optimal parameters during the iterations?

From: Matt

Date: 24 Nov, 2009 16:35:24

Message: 2 of 6

"Didi Cvet" <didi_cvet@yahoo.com> wrote in message <hegvhf$jss$1@fred.mathworks.com>...
> I have noticed that after some iteration (sometimes second, sometimes fifth) fmincon finds parameters that give minimal objective function. But when the iterations are finished i.e. fmincon is finished with exit flag
> >>Maximum number of iterations exceeded;
> increase OPTIONS.MaxIter.<<
> or
> >>Maximum number of function evaluations exceeded;
> increase OPTIONS.MaxFunEvals.<<
> I get initial values as optimal because during the iteration fmincon enters in some loop where it can't update Hessian matrix.
> Is there any way to get parameters from iteration where objective function is minimal not the ones that fmincon returns?
===============

If fmincon is terminating with the initial parameter values, it means that all iterations (even the very first one) have failed to make any progress. Therefore, there is no iteration where the output will be any better than your initial guess. You should probably investigate why the Hessian cannot be updated.

Subject: how can I save optimal parameters during the iterations?

From: Alan Weiss

Date: 24 Nov, 2009 18:27:45

Message: 3 of 6

Didi Cvet wrote:
> I have noticed that after some iteration (sometimes second, sometimes fifth) fmincon finds parameters that give minimal objective function. But when the iterations are finished i.e. fmincon is finished with exit flag
>>> Maximum number of iterations exceeded;
> increase OPTIONS.MaxIter.<<
> or
>>> Maximum number of function evaluations exceeded;
> increase OPTIONS.MaxFunEvals.<<
> I get initial values as optimal because during the iteration fmincon enters in some loop where it can't update Hessian matrix.
> Is there any way to get parameters from iteration where objective function is minimal not the ones that fmincon returns?

I don't know what is going on in your optimization, but you can write an
"output function" to record anything you like about the iterations. See
http://www.mathworks.com/access/helpdesk/help/toolbox/optim/ug/brhkghv-56.html
and
http://www.mathworks.com/access/helpdesk/help/toolbox/optim/ug/f19175.html#f11022
for more information.

Alan Weiss
MATLAB mathematical toolbox documentation

Subject: how can I save optimal parameters during the iterations?

From: Didi Cvet

Date: 25 Nov, 2009 08:04:20

Message: 4 of 6

"Matt " <xys@whatever.com> wrote in message <heh20c$79r$1@fred.mathworks.com>...
> "Didi Cvet" <didi_cvet@yahoo.com> wrote in message <hegvhf$jss$1@fred.mathworks.com>...
> > I have noticed that after some iteration (sometimes second, sometimes fifth) fmincon finds parameters that give minimal objective function. But when the iterations are finished i.e. fmincon is finished with exit flag
> > >>Maximum number of iterations exceeded;
> > increase OPTIONS.MaxIter.<<
> > or
> > >>Maximum number of function evaluations exceeded;
> > increase OPTIONS.MaxFunEvals.<<
> > I get initial values as optimal because during the iteration fmincon enters in some loop where it can't update Hessian matrix.
> > Is there any way to get parameters from iteration where objective function is minimal not the ones that fmincon returns?
> ===============
>
> If fmincon is terminating with the initial parameter values, it means that all iterations (even the very first one) have failed to make any progress. Therefore, there is no iteration where the output will be any better than your initial guess. You should probably investigate why the Hessian cannot be updated.

Thanks for response
Here is how I noticed that I have better parameters during the iterations. I've put a break point in my objective function (before it's end) and I was checking output after every call, and I'm sure that there was better output than initial one. I've tried to investigate why my Hessian is not updated but I can't find it out
Anyway thanks very much, I will try with output function but if anyone have idea why is this happening please let him write.
Respectfully
DidiCvet

Subject: how can I save optimal parameters during the iterations?

From: Marcelo Marazzi

Date: 25 Nov, 2009 20:07:20

Message: 5 of 6

Didi,

Could you provide a snippet of code that shows the call to fmincon, including the options
you're passing to the solver. (There are several algorithms in fmincon and it's not clear
to me which one you are using.)

I'm not sure what you mean by "my Hessian is not updated". How do you notice this is the
case? Are you supplying your own Hessian or relying on the solver's internal approximation
to it?

You may also want to set the option Display to 'iter' and post (part of) the output you
are getting, as this may be helpful.

-Marcelo

Didi Cvet wrote:
> "Matt " <xys@whatever.com> wrote in message <heh20c$79r$1@fred.mathworks.com>...
>> "Didi Cvet" <didi_cvet@yahoo.com> wrote in message <hegvhf$jss$1@fred.mathworks.com>...
>>> I have noticed that after some iteration (sometimes second, sometimes fifth) fmincon finds parameters that give minimal objective function. But when the iterations are finished i.e. fmincon is finished with exit flag
>>>>> Maximum number of iterations exceeded;
>>> increase OPTIONS.MaxIter.<<
>>> or
>>>>> Maximum number of function evaluations exceeded;
>>> increase OPTIONS.MaxFunEvals.<<
>>> I get initial values as optimal because during the iteration fmincon enters in some loop where it can't update Hessian matrix.
>>> Is there any way to get parameters from iteration where objective function is minimal not the ones that fmincon returns?
>> ===============
>>
>> If fmincon is terminating with the initial parameter values, it means that all iterations (even the very first one) have failed to make any progress. Therefore, there is no iteration where the output will be any better than your initial guess. You should probably investigate why the Hessian cannot be updated.
>
> Thanks for response
> Here is how I noticed that I have better parameters during the iterations. I've put a break point in my objective function (before it's end) and I was checking output after every call, and I'm sure that there was better output than initial one. I've tried to investigate why my Hessian is not updated but I can't find it out
> Anyway thanks very much, I will try with output function but if anyone have idea why is this happening please let him write.
> Respectfully
> DidiCvet

Subject: how can I save optimal parameters during the iterations?

From: Didi Cvet

Date: 26 Nov, 2009 09:11:09

Message: 6 of 6

Marcelo Marazzi <mREMOVEmaALLraCAPITALSzzi@mathworks.com> wrote in message <hek2po$cpc$1@fred.mathworks.com>...
> Didi,
>
> Could you provide a snippet of code that shows the call to fmincon, including the options
> you're passing to the solver. (There are several algorithms in fmincon and it's not clear
> to me which one you are using.)
>
> I'm not sure what you mean by "my Hessian is not updated". How do you notice this is the
> case? Are you supplying your own Hessian or relying on the solver's internal approximation
> to it?
>
> You may also want to set the option Display to 'iter' and post (part of) the output you
> are getting, as this may be helpful.
>
> -Marcelo
>
> Didi Cvet wrote:
> > "Matt " <xys@whatever.com> wrote in message <heh20c$79r$1@fred.mathworks.com>...
> >> "Didi Cvet" <didi_cvet@yahoo.com> wrote in message <hegvhf$jss$1@fred.mathworks.com>...
> >>> I have noticed that after some iteration (sometimes second, sometimes fifth) fmincon finds parameters that give minimal objective function. But when the iterations are finished i.e. fmincon is finished with exit flag
> >>>>> Maximum number of iterations exceeded;
> >>> increase OPTIONS.MaxIter.<<
> >>> or
> >>>>> Maximum number of function evaluations exceeded;
> >>> increase OPTIONS.MaxFunEvals.<<
> >>> I get initial values as optimal because during the iteration fmincon enters in some loop where it can't update Hessian matrix.
> >>> Is there any way to get parameters from iteration where objective function is minimal not the ones that fmincon returns?
> >> ===============
> >>
> >> If fmincon is terminating with the initial parameter values, it means that all iterations (even the very first one) have failed to make any progress. Therefore, there is no iteration where the output will be any better than your initial guess. You should probably investigate why the Hessian cannot be updated.
> >
> > Thanks for response
> > Here is how I noticed that I have better parameters during the iterations. I've put a break point in my objective function (before it's end) and I was checking output after every call, and I'm sure that there was better output than initial one. I've tried to investigate why my Hessian is not updated but I can't find it out
> > Anyway thanks very much, I will try with output function but if anyone have idea why is this happening please let him write.
> > Respectfully
> > DidiCvet

Sorry, I wasn't clear
First I am relying on the solvers Hessian not my one. Here are options for the fmincon that I call:
>>
options = optimset('LargeScale','off','Algorithm','active-set','TolCon', 1e-10, 'TolConSQP',1e-10,...
    'DerivativeCheck','on', ...
    'DiffMinChange',1e-8, ...
    'FunValCheck','on', ...
    'Display','iter','MaxIter',15,'TolX',1e-6, 'FunValCheck', 'on', 'DiffMinChange', 1e-6);
<<
In my objective function I construct a spline and I want minimizer to find minimal length of this spline changing first parameters then knot vector. This means that I call fmincon twice in my code.
>>
dtop=fmincon(@(dpm) objobj(dU0, dpm, p, P, m), dt0,[],[],[],[],a,b,@(dpm) concon1(dpm, dU0, T, e2, p, P), options);
>>
this is first time, dt0 is initial guess and it present the intervals between parameters vector (if t0 are parameters, dt0=diff(t0))
>>
dUop=fmincon(@(dknt) objobj(dknt, dt, p, P, m), dU0,[],[],[],[],a,b,@(dknt) concon(dknt, dt, T, e1, p, P), options);
<<
dU0 is initial guess for knots intervals.
Here is my objective function:
>>
function J=objobj(diffknt, diffpar, p, P, h)
a=0;
n=length(P);
knot=knot_from_diff(diffknt, a, h, p);
param=par_from_diff(diffpar, a, n);
N=spcol(knot,p+1,param);
controls=ls_svd(P, N);
C=spmak(knot, controls');
len=quadgk(@(param) l(param, C), min(param), max(param));
J=len;
end

function y=l(s,C)
Cprim=fnder(C);
ff1=fnval(Cprim, s);
y=ff1(1,:).^2+ff1(2,:).^2;
end
<<
knot_from_diff and par_from_diff are functions that compute knots and parameters from their interval differences. This is like that because if fmincon changes the parameters or knots directly I can get a decreasing sequence of points which I can not pass to spcol routine. With ls_svd I compute spline's control points:
>>
function c_points=ls_svd(P, N)
[U S V]=svd(N);
Nplus=V*pinv(S)*U';
c_points=Nplus*P';
end
>>
(here P are input points).
Here is some of 'Display','iter' output:
>>
                                Max Line search Directional First-order
 Iter F-count f(x) constraint steplength derivative optimality Procedure
0 71 3.65098e+008 0
1 148 3.60829e+008 8.908e-013 0.0156 1.63e+008 1.13e+009
2 219 8.12286e+027 1.038e-014 1 5.55e+032 4.69e+033
3 289 8.12286e+027 1.038e-014 2 -5.55e+032 4.69e+033 MaxSQPIter
4 359 8.12286e+027 1.038e-014 2 -5.55e+032 4.69e+033 Hessian not updated;
5 429 8.12286e+027 1.038e-014 2 -5.55e+032 4.69e+033 Hessian not updated;
499 8.12286e+027 1.038e-014 2 -5.55e+032 4.69e+033 Hessian not updated;
>>
I am not sure how clear is this, but this is how much I can exsplain here. Hope you'll understand me and THANKS anyway!
Respectfully
DidiCvet

Tags for this Thread

Add a New Tag:

Separated by commas
Ex.: root locus, bode

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

rssFeed for this Thread

Contact us at files@mathworks.com