Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
fminunc + transformation

Subject: fminunc + transformation

From: Saad

Date: 30 Mar, 2013 21:26:05

Message: 1 of 6

Dear all,

I am replicating a paper who recommends using fminunc to do a "constrained" optimization through a transformation. I have tried to use directly fmincon (with different algorithms) but the function doesnt optimize, thats the reason I would like to follow the paper advice and use a transformation as follows:

C_bar=lamda*(exp(C)/1+ exp(C)) where lamda is a constant, C is the unconstrained variable and C_bar is the constrained variable. I would really appreciate if you could show me how I could use the transformation in matlab. Do I have to create a seperate function? How can I link it to the optimizer please?

Here is my code

C=[1; 1; 1 ; 1; 1; 1; 1; 1; 1];
options=optimset('Diagnostics','on','Display','iter','TolX',0.001,'TolFun',0.001,'LargeScale','off','HessUpdate','bfgs');
[beta,fval,exitflag,output,grad,hessian] =fminunc(@mll,C,options)

Thanks a lot for your help

Best Regards

S

Subject: fminunc + transformation

From: Alan_Weiss

Date: 3 Apr, 2013 17:58:43

Message: 2 of 6

On 3/30/2013 5:26 PM, Saad wrote:
> Dear all,
>
> I am replicating a paper who recommends using fminunc to do a
> "constrained" optimization through a transformation. I have tried to
> use directly fmincon (with different algorithms) but the function
> doesnt optimize, thats the reason I would like to follow the paper
> advice and use a transformation as follows:
>
> C_bar=lamda*(exp(C)/1+ exp(C)) where lamda is a constant, C is the
> unconstrained variable and C_bar is the constrained variable. I would
> really appreciate if you could show me how I could use the
> transformation in matlab. Do I have to create a seperate function? How
> can I link it to the optimizer please?
>
> Here is my code
>
> C=[1; 1; 1 ; 1; 1; 1; 1; 1; 1];
> options=optimset('Diagnostics','on','Display','iter','TolX',0.001,'TolFun',0.001,'LargeScale','off','HessUpdate','bfgs');
>
> [beta,fval,exitflag,output,grad,hessian] =fminunc(@mll,C,options)
>
> Thanks a lot for your help
>
> Best Regards
>
> S

It looks to me as if C_bar is a multidimensional variable constrained to
be between 0 and 1. Is that right?

You can certainly include this transformation in a MATLAB statement:

C_bar = lamda.*(exp(C)./(1+exp(C));

The ./ and .* statements mean componentwise division and multiplication.

However, I think you could avoid this complicated and (it seems to me)
fragile transformation by simply setting bounds on fmincon and using the
interior-point or sqp algorithms, which respect bounds.
http://www.mathworks.com/help/optim/ug/writing-constraints.html#br9p_ry

Good luck,

Alan Weiss
MATLAB mathematical toolbox documentation

Subject: fminunc + transformation

From: Saad

Date: 4 Apr, 2013 07:47:11

Message: 3 of 6



Alan_Weiss <aweiss@mathworks.com> wrote in message <kjhqkj$ioi$1@newscl01ah.mathworks.com>...
> On 3/30/2013 5:26 PM, Saad wrote:
> > Dear all,
> >
> > I am replicating a paper who recommends using fminunc to do a
> > "constrained" optimization through a transformation. I have tried to
> > use directly fmincon (with different algorithms) but the function
> > doesnt optimize, thats the reason I would like to follow the paper
> > advice and use a transformation as follows:
> >
> > C_bar=lamda*(exp(C)/1+ exp(C)) where lamda is a constant, C is the
> > unconstrained variable and C_bar is the constrained variable. I would
> > really appreciate if you could show me how I could use the
> > transformation in matlab. Do I have to create a seperate function? How
> > can I link it to the optimizer please?
> >
> > Here is my code
> >
> > C=[1; 1; 1 ; 1; 1; 1; 1; 1; 1];
> > options=optimset('Diagnostics','on','Display','iter','TolX',0.001,'TolFun',0.001,'LargeScale','off','HessUpdate','bfgs');
> >
> > [beta,fval,exitflag,output,grad,hessian] =fminunc(@mll,C,options)
> >
> > Thanks a lot for your help
> >
> > Best Regards
> >
> > S
>
> It looks to me as if C_bar is a multidimensional variable constrained to
> be between 0 and 1. Is that right?
>
> You can certainly include this transformation in a MATLAB statement:
>
> C_bar = lamda.*(exp(C)./(1+exp(C));
>
> The ./ and .* statements mean componentwise division and multiplication.
>
> However, I think you could avoid this complicated and (it seems to me)
> fragile transformation by simply setting bounds on fmincon and using the
> interior-point or sqp algorithms, which respect bounds.
> http://www.mathworks.com/help/optim/ug/writing-constraints.html#br9p_ry
>
> Good luck,
>
> Alan Weiss
> MATLAB mathematical toolbox documentation

Hi Alan

Thank you for your reply. I did try fmincon with different algorithms (interior-point and sqp algorithms) but could not locate the minimum. I also tightened the tolerance but still could not minimize (for info i am working on a kalman filtering problem). I think the issue is that I am not able to find a good initial guess. Any thoughts on how to handle that? Thanks a lot

Best

S

Subject: fminunc + transformation

From: Alan_Weiss

Date: 4 Apr, 2013 12:12:32

Message: 4 of 6

On 4/4/2013 3:47 AM, Saad wrote:
>
>
> Alan_Weiss <aweiss@mathworks.com> wrote in message
> <kjhqkj$ioi$1@newscl01ah.mathworks.com>...
>> On 3/30/2013 5:26 PM, Saad wrote:
>> > Dear all,
>> >
>> > I am replicating a paper who recommends using fminunc to do a >
>> "constrained" optimization through a transformation. I have tried to
>> > use directly fmincon (with different algorithms) but the function >
>> doesnt optimize, thats the reason I would like to follow the paper >
>> advice and use a transformation as follows:
>> >
>> > C_bar=lamda*(exp(C)/1+ exp(C)) where lamda is a constant, C is the
>> > unconstrained variable and C_bar is the constrained variable. I
>> would > really appreciate if you could show me how I could use the >
>> transformation in matlab. Do I have to create a seperate function?
>> How > can I link it to the optimizer please?
>> >
>> > Here is my code
>> >
>> > C=[1; 1; 1 ; 1; 1; 1; 1; 1; 1];
>> >
>> options=optimset('Diagnostics','on','Display','iter','TolX',0.001,'TolFun',0.001,'LargeScale','off','HessUpdate','bfgs');
>> >
>> > [beta,fval,exitflag,output,grad,hessian] =fminunc(@mll,C,options)
>> >
>> > Thanks a lot for your help
>> >
>> > Best Regards
>> >
>> > S
>>
>> It looks to me as if C_bar is a multidimensional variable constrained
>> to be between 0 and 1. Is that right?
>>
>> You can certainly include this transformation in a MATLAB statement:
>>
>> C_bar = lamda.*(exp(C)./(1+exp(C));
>>
>> The ./ and .* statements mean componentwise division and multiplication.
>>
>> However, I think you could avoid this complicated and (it seems to
>> me) fragile transformation by simply setting bounds on fmincon and
>> using the interior-point or sqp algorithms, which respect bounds.
>> http://www.mathworks.com/help/optim/ug/writing-constraints.html#br9p_ry
>>
>> Good luck,
>>
>> Alan Weiss
>> MATLAB mathematical toolbox documentation
>
> Hi Alan
>
> Thank you for your reply. I did try fmincon with different
> algorithms (interior-point and sqp algorithms) but could not locate
> the minimum. I also tightened the tolerance but still could not
> minimize (for info i am working on a kalman filtering problem). I
> think the issue is that I am not able to find a good initial guess.
> Any thoughts on how to handle that? Thanks a lot
>
> Best
>
> S

Well, assuming that you have upper and lower bounds on all components,
why not try a lot of random initial points?

x0 = lb + rand(size(lb)).*(ub - lb);

There are more suggestions here (written for Global Optimization
Toolbox, but you can use the suggestions anyway):
http://www.mathworks.com/help/gads/refining-the-start-points.html#bsfjxr9
http://www.mathworks.com/help/gads/isolated-global-minimum.html

Alan Weiss
MATLAB mathematical toolbox documentation

Subject: fminunc + transformation

From: Saad

Date: 11 Apr, 2013 18:27:07

Message: 5 of 6

Alan_Weiss <aweiss@mathworks.com> wrote in message <kjjqng$i08$1@newscl01ah.mathworks.com>...
> On 4/4/2013 3:47 AM, Saad wrote:
> >
> >
> > Alan_Weiss <aweiss@mathworks.com> wrote in message
> > <kjhqkj$ioi$1@newscl01ah.mathworks.com>...
> >> On 3/30/2013 5:26 PM, Saad wrote:
> >> > Dear all,
> >> >
> >> > I am replicating a paper who recommends using fminunc to do a >
> >> "constrained" optimization through a transformation. I have tried to
> >> > use directly fmincon (with different algorithms) but the function >
> >> doesnt optimize, thats the reason I would like to follow the paper >
> >> advice and use a transformation as follows:
> >> >
> >> > C_bar=lamda*(exp(C)/1+ exp(C)) where lamda is a constant, C is the
> >> > unconstrained variable and C_bar is the constrained variable. I
> >> would > really appreciate if you could show me how I could use the >
> >> transformation in matlab. Do I have to create a seperate function?
> >> How > can I link it to the optimizer please?
> >> >
> >> > Here is my code
> >> >
> >> > C=[1; 1; 1 ; 1; 1; 1; 1; 1; 1];
> >> >
> >> options=optimset('Diagnostics','on','Display','iter','TolX',0.001,'TolFun',0.001,'LargeScale','off','HessUpdate','bfgs');
> >> >
> >> > [beta,fval,exitflag,output,grad,hessian] =fminunc(@mll,C,options)
> >> >
> >> > Thanks a lot for your help
> >> >
> >> > Best Regards
> >> >
> >> > S
> >>
> >> It looks to me as if C_bar is a multidimensional variable constrained
> >> to be between 0 and 1. Is that right?
> >>
> >> You can certainly include this transformation in a MATLAB statement:
> >>
> >> C_bar = lamda.*(exp(C)./(1+exp(C));
> >>
> >> The ./ and .* statements mean componentwise division and multiplication.
> >>
> >> However, I think you could avoid this complicated and (it seems to
> >> me) fragile transformation by simply setting bounds on fmincon and
> >> using the interior-point or sqp algorithms, which respect bounds.
> >> http://www.mathworks.com/help/optim/ug/writing-constraints.html#br9p_ry
> >>
> >> Good luck,
> >>
> >> Alan Weiss
> >> MATLAB mathematical toolbox documentation
> >
> > Hi Alan
> >
> > Thank you for your reply. I did try fmincon with different
> > algorithms (interior-point and sqp algorithms) but could not locate
> > the minimum. I also tightened the tolerance but still could not
> > minimize (for info i am working on a kalman filtering problem). I
> > think the issue is that I am not able to find a good initial guess.
> > Any thoughts on how to handle that? Thanks a lot
> >
> > Best
> >
> > S
>
> Well, assuming that you have upper and lower bounds on all components,
> why not try a lot of random initial points?
>
> x0 = lb + rand(size(lb)).*(ub - lb);
>
> There are more suggestions here (written for Global Optimization
> Toolbox, but you can use the suggestions anyway):
> http://www.mathworks.com/help/gads/refining-the-start-points.html#bsfjxr9
> http://www.mathworks.com/help/gads/isolated-global-minimum.html
>
> Alan Weiss
> MATLAB mathematical toolbox documentation

Dear Alan

Hope you are well. I tried your suggestions and used Global Optimization Toolbox and my new problem is defined as follows:
%%%%%%%
 opts = optimset('Algorithm','interior-point','Hessian','lbfgs');
problem = createOptimProblem('fmincon','x0',[2; 2; 2 ; 2; 3; 3; 3; 3; 3],'objective',@(beta)mll(beta,y,R,xx),...
    'lb',[0;0; 0; 0; 0; 0; 0; 0; 0],'ub',[10; 10; 10; 10; 10; 10; 10; 10; 10], 'options',opts);
gs = GlobalSearch;
rng(14,'twister') % for reproducibility
gs.NumTrialPoints = 1000;%1e5;
gs.MaxTime = 300;
[xg fvalg] = run(gs,problem)

%%%%%

I was just wondering how can I retrieve the hessian matrix once the optimization is finished? Thanks a lot
Regards

S

Subject: fminunc + transformation

From: Alan_Weiss

Date: 12 Apr, 2013 17:37:41

Message: 6 of 6

On 4/11/2013 2:27 PM, Saad wrote:
> Alan_Weiss <aweiss@mathworks.com> wrote in message
> <kjjqng$i08$1@newscl01ah.mathworks.com>...
>> On 4/4/2013 3:47 AM, Saad wrote:
>> >
>> >
>> > Alan_Weiss <aweiss@mathworks.com> wrote in message >
>> <kjhqkj$ioi$1@newscl01ah.mathworks.com>...
>> >> On 3/30/2013 5:26 PM, Saad wrote:
>> >> > Dear all,
>> >> >
>> >> > I am replicating a paper who recommends using fminunc to do a >
>> >> "constrained" optimization through a transformation. I have tried
>> to >> > use directly fmincon (with different algorithms) but the
>> function > >> doesnt optimize, thats the reason I would like to
>> follow the paper > >> advice and use a transformation as follows:
>> >> >
>> >> > C_bar=lamda*(exp(C)/1+ exp(C)) where lamda is a constant, C is
>> the >> > unconstrained variable and C_bar is the constrained
>> variable. I >> would > really appreciate if you could show me how I
>> could use the > >> transformation in matlab. Do I have to create a
>> seperate function? >> How > can I link it to the optimizer please?
>> >> >
>> >> > Here is my code
>> >> >
>> >> > C=[1; 1; 1 ; 1; 1; 1; 1; 1; 1];
>> >> > >>
>> options=optimset('Diagnostics','on','Display','iter','TolX',0.001,'TolFun',0.001,'LargeScale','off','HessUpdate','bfgs');
>> >> >
>> >> > [beta,fval,exitflag,output,grad,hessian] =fminunc(@mll,C,options)
>> >> >
>> >> > Thanks a lot for your help
>> >> >
>> >> > Best Regards
>> >> >
>> >> > S
>> >>
>> >> It looks to me as if C_bar is a multidimensional variable
>> constrained >> to be between 0 and 1. Is that right?
>> >>
>> >> You can certainly include this transformation in a MATLAB statement:
>> >>
>> >> C_bar = lamda.*(exp(C)./(1+exp(C));
>> >>
>> >> The ./ and .* statements mean componentwise division and
>> multiplication.
>> >>
>> >> However, I think you could avoid this complicated and (it seems to
>> >> me) fragile transformation by simply setting bounds on fmincon and
>> >> using the interior-point or sqp algorithms, which respect bounds.
>> >>
>> http://www.mathworks.com/help/optim/ug/writing-constraints.html#br9p_ry
>> >>
>> >> Good luck,
>> >>
>> >> Alan Weiss
>> >> MATLAB mathematical toolbox documentation
>> >
>> > Hi Alan
>> >
>> > Thank you for your reply. I did try fmincon with different >
>> algorithms (interior-point and sqp algorithms) but could not locate
>> > the minimum. I also tightened the tolerance but still could not >
>> minimize (for info i am working on a kalman filtering problem). I >
>> think the issue is that I am not able to find a good initial guess. >
>> Any thoughts on how to handle that? Thanks a lot
>> >
>> > Best
>> >
>> > S
>>
>> Well, assuming that you have upper and lower bounds on all
>> components, why not try a lot of random initial points?
>>
>> x0 = lb + rand(size(lb)).*(ub - lb);
>>
>> There are more suggestions here (written for Global Optimization
>> Toolbox, but you can use the suggestions anyway):
>> http://www.mathworks.com/help/gads/refining-the-start-points.html#bsfjxr9
>>
>> http://www.mathworks.com/help/gads/isolated-global-minimum.html
>>
>> Alan Weiss
>> MATLAB mathematical toolbox documentation
>
> Dear Alan
>
> Hope you are well. I tried your suggestions and used Global
> Optimization Toolbox and my new problem is defined as follows:
> %%%%%%%
> opts = optimset('Algorithm','interior-point','Hessian','lbfgs');
> problem = createOptimProblem('fmincon','x0',[2; 2; 2 ; 2; 3; 3; 3; 3;
> 3],'objective',@(beta)mll(beta,y,R,xx),...
> 'lb',[0;0; 0; 0; 0; 0; 0; 0; 0],'ub',[10; 10; 10; 10; 10; 10; 10;
> 10; 10], 'options',opts);
> gs = GlobalSearch;
> rng(14,'twister') % for reproducibility
> gs.NumTrialPoints = 1000;%1e5;
> gs.MaxTime = 300;
> [xg fvalg] = run(gs,problem)
>
> %%%%%
>
> I was just wondering how can I retrieve the hessian matrix once the
> optimization is finished? Thanks a lot
> Regards
>
> S

You cannot retrieve the Hessian automatically. It wouldn't do you much
good, even if you did retrieve it; see
http://www.mathworks.com/help/optim/ug/hessian.html#bsapedt

If you are at a local minimum that is not at a constraint boundary, you
can run the fminunc solver from the solution point and get the Hessian
that way. But if a constraint is active, I am not sure what you can do,
because even if you re-ran fmincon starting from that point, the
returned Hessian is unreliable.

Alan Weiss
MATLAB mathematical toolbox documentation

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us