Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
fminsearch and fmincon

Subject: fminsearch and fmincon

From: Justin_Time

Date: 22 Jul, 2005 12:56:38

Message: 1 of 7

Hello,
I have a question about fminsearch vs fmincon.

Basically my problem is trying to fit some data with the equation:
(A*yval_1+B*yval_2)*exp(-C+D*x)

I was calling fminsearch with the parameters:
fminsearch('fitting_function',[A B C D], [], xvals, yval_1, yval_2)

where fitting_function.m had the equation above and basically found
the error between the function and my data for values of A, B, C and
D (which fminsearch worked to find)...

Now I'm working with a dataset in which A and B need to be
constrained:
A+B=1 and A,B={0..1}
but C and D must not be constrained.

I know there's a fmincon function, but I'm not sure of how to use it
in my situation. How can I have A and B be constrained, and let C
and D float?

I read the matlab help, but I'm still confused. Is there a tutorial
somewhere that covers this?

Thanks.

Subject: fminsearch and fmincon

From: John D'Errico

Date: 22 Jul, 2005 17:25:04

Message: 2 of 7

In article <ef0e9c7.-1@webx.raydaftYaTP>,
 Justin_Time <justin_time@mailinator.com> wrote:

> Hello,
> I have a question about fminsearch vs fmincon.
>
> Basically my problem is trying to fit some data with the equation:
> (A*yval_1+B*yval_2)*exp(-C+D*x)
>
> I was calling fminsearch with the parameters:
> fminsearch('fitting_function',[A B C D], [], xvals, yval_1, yval_2)
>
> where fitting_function.m had the equation above and basically found
> the error between the function and my data for values of A, B, C and
> D (which fminsearch worked to find)...
>
> Now I'm working with a dataset in which A and B need to be
> constrained:
> A+B=1 and A,B={0..1}
> but C and D must not be constrained.
>
> I know there's a fmincon function, but I'm not sure of how to use it
> in my situation. How can I have A and B be constrained, and let C
> and D float?
>
> I read the matlab help, but I'm still confused. Is there a tutorial
> somewhere that covers this?
>
> Thanks.

There is no unique solution to the coefficients you are
trying to estimate. This will cause problems for any
optimizer. At least this is so if you do not pose that
sum constraint on A and B. Consider that any value of C
can be absorbed into A and B. The problems you will see
are lack of convergence to a consistent solution from
different sets of starting values.

With the constraint there is no longer a problem. But
this constraint is trivial to impose. Since A+B=1,
simply eliminate B in your model.

(A*yval_1+(1-A)*yval_2)*exp(-C+D*x)

Note that C can no longer be taken out of the exponential
to absorb into A, because of the 1-A term.

If you really don't want to bother with fmincon, then
why not do one more transformtion (hack if you prefer)
to your model? If A + B = 1, then transform the problem
into a related one where

  A = [sin(theta)]^2

  B = [cos(theta)]^2

Note that A + B == 1 identically, and A and B are
always contained in the interval [0,1]. In effect
we have turned a 4 variable problem into a 3 variable
one.

  ([sin(theta)]^2*yval_1+[cos(theta)]^2*yval_2)*exp(-C+D*x)

Theta has no constraints upon it at all, but the
coefficients of yval_1 and yval_2 are implicitly
constrained. And, you can go back to fminsearch
with no mental energy consumed.

HTH,
John D'Errico


--
The best material model of a cat is another, or
preferably the same, cat.
A. Rosenblueth, Philosophy of Science, 1945

Subject: fminsearch and fmincon

From: Justin_Time

Date: 24 Jul, 2005 00:20:05

Message: 3 of 7

>
> There is no unique solution to the coefficients you are
> trying to estimate. This will cause problems for any
> optimizer. At least this is so if you do not pose that
> sum constraint on A and B. Consider that any value of C
> can be absorbed into A and B. The problems you will see
> are lack of convergence to a consistent solution from
> different sets of starting values.
>
> With the constraint there is no longer a problem. But
> this constraint is trivial to impose. Since A+B=1,
> simply eliminate B in your model.
>
> (A*yval_1+(1-A)*yval_2)*exp(-C+D*x)
>
> Note that C can no longer be taken out of the exponential
> to absorb into A, because of the 1-A term.
>
> If you really don't want to bother with fmincon, then
> why not do one more transformtion (hack if you prefer)
> to your model? If A + B = 1, then transform the problem
> into a related one where
>
> A = [sin(theta)]^2
>
> B = [cos(theta)]^2
>
> Note that A + B == 1 identically, and A and B are
> always contained in the interval [0,1]. In effect
> we have turned a 4 variable problem into a 3 variable
> one.
>
> ([sin(theta)]^2*yval_1+[cos(theta)]^2*yval_2)*exp(-C+D*x)
>
> Theta has no constraints upon it at all, but the
> coefficients of yval_1 and yval_2 are implicitly
> constrained. And, you can go back to fminsearch
> with no mental energy consumed.
>
> HTH,
> John D'Errico
>

Thanks for responding John!

I guess I was asking the question in the wrong way. More generally,
what is the right way to solve an equation with say 3 (or 4) unknowns
when I need to constrain 1 (or 2) of the unknowns within some limits
(and let the other unknowns float)... I've never really used the
fmincon function and am a bit unsure of how to proceed. I'm not
looking for an answer to the specific problem I posed (in the
original post), but rather a tutorial on the uses and differences
between fminsearch and fmincon...(so I better know how to apply
it)...

thanks again!

Subject: fminsearch and fmincon

From: John D'Errico

Date: 24 Jul, 2005 11:30:03

Message: 4 of 7

In article <ef0e9c7.1@webx.raydaftYaTP>,
 Justin_Time <justin_time@mailinator.com> wrote:

> I guess I was asking the question in the wrong way. More generally,
> what is the right way to solve an equation with say 3 (or 4) unknowns
> when I need to constrain 1 (or 2) of the unknowns within some limits
> (and let the other unknowns float)... I've never really used the
> fmincon function and am a bit unsure of how to proceed. I'm not
> looking for an answer to the specific problem I posed (in the
> original post), but rather a tutorial on the uses and differences
> between fminsearch and fmincon...(so I better know how to apply
> it)...

This is what the lb and ub inputs to fmincon do.
They provide bound constraints.

If a variable is unconstrained, then use +/- inf.
As an example, if x1 must lie in [0,3], and x3>=2,
but x2 and x4 are unconstrained, then

  lb = [0 -inf 2 -inf]
  ub = [3 inf inf inf]

As for the differences between the two optimizers,
that would take a book. Fminsearch is a Nelder/Mead
polytope method. (Some call it a simplex method,
but then someone always gets confused that it does
linear programming. Polytope has no such ambiguity.)

These simplex/polytope methods work by reflecting
a simplex (think triangle in 2-dimensions) around
the space. The simplex walks around, testing the
objective at each point. They walk AWAY from high
values, rather than towards low ones. There is a
difference.

Fmincon is a more sophisticated method, which
requires local knowledge of the gradient of the
surface. It computes the gradient using a finite
difference approximation, or you can supply it
yourself.

Fminsearch is limited in the number of unknowns you
can solve for, because it gets very slow for high
dimensional problems. I'd rarely (if ever) use it
for more than about 5 or 6 unknowns. Its fine for
2d or 3d cases that I will not need to solve often.

Think of fminsearch as the 1960's era VW beetle.
Its a fun little car that will get you where you
want to go, as long as you don't want to get there
too fast, or carry too many bags in the trunk. Its
easy enough to repair when there are problems - no
fancy diagnostic tools required and a simple shop
manual. Don't expect much in the way of heat in the
winter or power windows and seats.

On the other hand, fmincon is a new car. Its got
gadgets like power windows, heated seats that adjust
at the touch of a button, dvd players in the back
seat for the kids, gps mapping for the navigator.
It will get you up to 75 on the expressway in a
matter of seconds (not that I would ever exceed
the posted speed limit.) Its a little touchy under
the hood though.

HTH,
John

--
The best material model of a cat is another, or
preferably the same, cat.
A. Rosenblueth, Philosophy of Science, 1945

Subject: fminsearch and fmincon

From: Justin_Time

Date: 24 Jul, 2005 18:54:03

Message: 5 of 7

Thanks for responding again, John!

>
> This is what the lb and ub inputs to fmincon do.
> They provide bound constraints.
>
> If a variable is unconstrained, then use +/- inf.
> As an example, if x1 must lie in [0,3], and x3>=2,
> but x2 and x4 are unconstrained, then
>
> lb = [0 -inf 2 -inf]
> ub = [3 inf inf inf]
>

I guess I just need this kind of hand-holding... I've never used
fmincon before, so this is all new to me. However, this is exactly
what I was looking for!

> As for the differences between the two optimizers,
> that would take a book. Fminsearch is a Nelder/Mead
> polytope method. (Some call it a simplex method,
> but then someone always gets confused that it does
> linear programming. Polytope has no such ambiguity.)
>
> These simplex/polytope methods work by reflecting
> a simplex (think triangle in 2-dimensions) around
> the space. The simplex walks around, testing the
> objective at each point. They walk AWAY from high
> values, rather than towards low ones. There is a
> difference.
>
> Fmincon is a more sophisticated method, which
> requires local knowledge of the gradient of the
> surface. It computes the gradient using a finite
> difference approximation, or you can supply it
> yourself.
>
> Fminsearch is limited in the number of unknowns you
> can solve for, because it gets very slow for high
> dimensional problems. I'd rarely (if ever) use it
> for more than about 5 or 6 unknowns. Its fine for
> 2d or 3d cases that I will not need to solve often.
>
> Think of fminsearch as the 1960's era VW beetle.
> Its a fun little car that will get you where you
> want to go, as long as you don't want to get there
> too fast, or carry too many bags in the trunk. Its
> easy enough to repair when there are problems - no
> fancy diagnostic tools required and a simple shop
> manual. Don't expect much in the way of heat in the
> winter or power windows and seats.
>
> On the other hand, fmincon is a new car. Its got
> gadgets like power windows, heated seats that adjust
> at the touch of a button, dvd players in the back
> seat for the kids, gps mapping for the navigator.
> It will get you up to 75 on the expressway in a
> matter of seconds (not that I would ever exceed
> the posted speed limit.) Its a little touchy under
> the hood though.
>
> HTH,
> John
>

This really helped! thanks again! (heh, I really enjoyed the
analogy!) I've just always stuck with fminsearch (since that's what
I started using way back when) and I've never had a reason to
question it's abilities until now. It't probably time to learn some
new tricks!

A few more questions though. I'm getting a feel for constraining a
single variable bewteen bounds, but I'm still unsure of how to
express a linear combination of variables with constraint:
for instance - four variables, a(1) a(2) a(3) a(4)
0 <= a(1) <= 1 and 0 <= a(2) <= 1 while a(3) and a(4)
float,
I'll use the ub and lb as you stated above...
but also I may need:
a(1) + a(2) = 1
(or some other linear combination of variables equal to a constant)

Also, how do I pass values that are needed for fitting to fmincon.
When using fminsearch, variables that were in the first group were
minimized to fit, while everything else was passed as static
variables or arguments:
fminsearch('fitting_function',[A B C D], [], xvals, other_args,
etc...)
so [A B C D] are minimized from their starting values, while
xvals, other_args, etc... are static yet needed to fit A, B, C, and D
(or had other effects on the .m file)

I think what I need is a tutorial which explains how to put together
the correct syntax for fmincon vs fminsearch for specific problems
(the matlab help didn't help me much - maybe I'm too dense for
it!)...

Thanks again!

Subject: fminsearch and fmincon

From: Justin_Time

Date: 25 Jul, 2005 00:03:53

Message: 6 of 7

John,

I think I've gotten the hang of fmincon! Something just clicked,
and it all seems to work now... fmincon IS extremely more
useful/powerful as compared to fminsearch! Thanks for all your
help. I just need to find some more constraints for my fitting now.
  Thanks again!

Subject: fminsearch and fmincon

From: John D'Errico

Date: 25 Jul, 2005 09:22:40

Message: 7 of 7

In article <ef0e9c7.4@webx.raydaftYaTP>,
 Justin_Time <justin_time@mailinator.com> wrote:

> John,
>
> I think I've gotten the hang of fmincon! Something just clicked,
> and it all seems to work now... fmincon IS extremely more
> useful/powerful as compared to fminsearch! Thanks for all your
> help. I just need to find some more constraints for my fitting now.
> Thanks again!

Yes. An "AHA!" moment. They are nice.

Ask again if you have any questions about using
constraints.

John


--
The best material model of a cat is another, or
preferably the same, cat.
A. Rosenblueth, Philosophy of Science, 1945

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us