Thread Subject:

use fminsearch and fmincon to find MLE

Hi,
I'm trying to estimate coefficients and sigma in a MLE function. I guess sigma is around 9. But when I use fminsearch, the result is very different from when I use fmincon, constrainted sigma to be larger than 0.0000000001, even though I set the same initial value to sigma (set as 9). Can anyone please explain to me why it is like this?

"Lizzy " <lizzy1230@gmail.com> wrote in message <kadqg5$ehk$1@newscl01ah.mathworks.com>...
> Hi,
> I'm trying to estimate coefficients and sigma in a MLE function. I guess sigma is around 9. But when I use fminsearch, the result is very different from when I use fmincon, constrainted sigma to be larger than 0.0000000001, even though I set the same initial value to sigma (set as 9). Can anyone please explain to me why it is like this?
==============

A few ideas,

(1) You're running a constrained optimization with FMINCON but an unconstrained optimization with FMINSEARCH. So, why shouldn't their results be different?

(2) FMINSEARCH uses a heuristic algorithm, not gaurnateed to converge except for problems with a single unknown. Conversely, FMINCON uses algorithms which are more robustly convergent in multi-variable problems. How many unknowns do you have?

(3) What reason do we have to believe the solution is unique?

"Matt J" wrote in message <kadrlf$i7v$1@newscl01ah.mathworks.com>...
> "Lizzy " <lizzy1230@gmail.com> wrote in message <kadqg5$ehk$1@newscl01ah.mathworks.com>...
> > Hi,
> > I'm trying to estimate coefficients and sigma in a MLE function. I guess sigma is around 9. But when I use fminsearch, the result is very different from when I use fmincon, constrainted sigma to be larger than 0.0000000001, even though I set the same initial value to sigma (set as 9). Can anyone please explain to me why it is like this?
> ==============
>
> A few ideas,
>
> (1) You're running a constrained optimization with FMINCON but an unconstrained optimization with FMINSEARCH. So, why shouldn't their results be different?
>
> (2) FMINSEARCH uses a heuristic algorithm, not gaurnateed to converge except for problems with a single unknown. Conversely, FMINCON uses algorithms which are more robustly convergent in multi-variable problems. How many unknowns do you have?
>
> (3) What reason do we have to believe the solution is unique?

Well, when I use fminsearch, the results are better. And it says at the end
Optimization terminated:
 the current x satisfies the termination criteria using OPTIONS.TolX of 1.000000e-020
 and F(X) satisfies the convergence criteria using OPTIONS.TolFun of 1.000000e-010
So does it mean that it converges?
And I have many coefficients: 17. But all the coefficients are not constrained except for sigma (standard deviation of normal).
I'm now trying to use fminsearch, but change wherever there is a sigma to abs(sigma)+eps. And the final result of sigma I want to estimated is calculated by abs(result of sigma)+eps. (and the result is the same if I don't use this abs(.)+eps. Is it a correct way?

"Lizzy " <lizzy1230@gmail.com> wrote in message <kadsoq$lg2$1@newscl01ah.mathworks.com>...
>
> Well, when I use fminsearch, the results are better.
============

It's not clear what you mean by "the results are better". How are you measuring "better"?


> Optimization terminated:
> the current x satisfies the termination criteria using OPTIONS.TolX of 1.000000e-020
> and F(X) satisfies the convergence criteria using OPTIONS.TolFun of 1.000000e-010
> So does it mean that it converges?


It means that it converged to something, but not necessarily to an optimal point. You could plug the solution given by fminsearch into fmincon and see if it progresses further downhill.


> I'm now trying to use fminsearch, but change wherever there is a sigma to abs(sigma)+eps. And the final result of sigma I want to estimated is calculated by abs(result of sigma)+eps. (and the result is the same if I don't use this abs(.)+eps. Is it a correct way?
============

It sounds reasonable for FMINSEARCH. You couldn't use this objective in FMINCON because FMINCON only handles differentiable functions.

"Matt J" wrote in message <kadtqg$ou4$1@newscl01ah.mathworks.com>...
> "Lizzy " <lizzy1230@gmail.com> wrote in message <kadsoq$lg2$1@newscl01ah.mathworks.com>...
> >
> > Well, when I use fminsearch, the results are better.
> ============
>
> It's not clear what you mean by "the results are better". How are you measuring "better"?
>
>
> > Optimization terminated:
> > the current x satisfies the termination criteria using OPTIONS.TolX of 1.000000e-020
> > and F(X) satisfies the convergence criteria using OPTIONS.TolFun of 1.000000e-010
> > So does it mean that it converges?
>
>
> It means that it converged to something, but not necessarily to an optimal point. You could plug the solution given by fminsearch into fmincon and see if it progresses further downhill.
>
>
> > I'm now trying to use fminsearch, but change wherever there is a sigma to abs(sigma)+eps. And the final result of sigma I want to estimated is calculated by abs(result of sigma)+eps. (and the result is the same if I don't use this abs(.)+eps. Is it a correct way?
> ============
>
> It sounds reasonable for FMINSEARCH. You couldn't use this objective in FMINCON because FMINCON only handles differentiable functions.

Thanks Matt. But what if I have no idea if my function is differentiable or not? And if I use fminsearch, is it ok that I set sigma as abs(sigma)+eps to make it greater than 0? And my code is as follows:

coefInit = [1,1,1,1,1....1]];
options_fminsearch = optimset('Display','iter','TolFun',1e-10,'TolX',1e-20,'MaxIt',10000000,'MaxFunEvals',10000000000000);
coefEst = fminsearch(@(coef) llk(coef,x1....xn),coefInit,options_fminsearch);

And
aa = log(phi/(abs(sigma)+eps));
aa(isnan(aa))=-inf;
bb = log(1-phi2);
bb(isnan(bb))=-inf;
llk = d.*aa+(1-d).*bb;

res = -sum(llk);

Is it correct? Thanks a lot.

"Lizzy " <lizzy1230@gmail.com> wrote in message <kadus9$rpo$1@newscl01ah.mathworks.com>...
>
> Thanks Matt. But what if I have no idea if my function is differentiable or not?

You should know!

In this case, because abs(sigma) is a not a continuously differentiable function of sigma (at sigma=0), it's pretty clear that plugging it into a smooth functions like log() etc... will lead to something that is also not continuously differentiable. As I said before, that doesn't matter for FMINSEARCH, because FMINSEARCH doesn't use an algorithm that calculates derivatives. You would have to use a different objective function for FMINCON, however, and that makes comparison between with FMINSEARCH difficult.

On the other hand, instead of using abs(sigma), you could just make the change of variables

 sigma=z^2+eps

which is differentiable and could be used in FMINCON or any other Opt Toolbox solver. Although now, because you're not constraining the new variable z, you could use FMINUNC instead.

"Matt J" wrote in message <kae0s1$4os$1@newscl01ah.mathworks.com>...
> "Lizzy " <lizzy1230@gmail.com> wrote in message <kadus9$rpo$1@newscl01ah.mathworks.com>...
> >
> > Thanks Matt. But what if I have no idea if my function is differentiable or not?
>
> You should know!
>
> In this case, because abs(sigma) is a not a continuously differentiable function of sigma (at sigma=0), it's pretty clear that plugging it into a smooth functions like log() etc... will lead to something that is also not continuously differentiable. As I said before, that doesn't matter for FMINSEARCH, because FMINSEARCH doesn't use an algorithm that calculates derivatives. You would have to use a different objective function for FMINCON, however, and that makes comparison between with FMINSEARCH difficult.
>
> On the other hand, instead of using abs(sigma), you could just make the change of variables
>
> sigma=z^2+eps
>
> which is differentiable and could be used in FMINCON or any other Opt Toolbox solver. Although now, because you're not constraining the new variable z, you could use FMINUNC instead.


Thanks, Well, to make my life easier----is it necessary to impose a constraint of sigma>0 in mle estimation, if my estimated results for sigma are always >0?
If I only wanna use fminsearch (by using fmincon, the results always come to NAN, dunno why), how could I determine if it convergent to an optimal point? Just by plug my estimated results back in fminsearch again, and see if L still goes down?

"Lizzy " <lizzy1230@gmail.com> wrote in message <kae3m1$dlv$1@newscl01ah.mathworks.com>...
>
>
> Thanks, Well, to make my life easier----is it necessary to impose a constraint of sigma>0 in mle estimation, if my estimated results for sigma are always >0?
=============

How will you ensure the estimated sigma are always >0 without constraints?

> If I only wanna use fminsearch (by using fmincon, the results always come to NAN, dunno why), how could I determine if it convergent to an optimal point? Just by plug my estimated results back in fminsearch again, and see if L still goes down?
===================

You should try FMINCON's sqp algorithm. If certain regions of your function are NaN or non-finite (as is your case), the sqp algorithm can avoid those regions.

To determine whether a point is an unconstrained minimizer, it is sufficient to check whether the gradient is zero or approximately so.

"Matt J" wrote in message <kae5jt$jh0$1@newscl01ah.mathworks.com>...
> "Lizzy " <lizzy1230@gmail.com> wrote in message <kae3m1$dlv$1@newscl01ah.mathworks.com>...
> >
> >
> > Thanks, Well, to make my life easier----is it necessary to impose a constraint of sigma>0 in mle estimation, if my estimated results for sigma are always >0?
> =============
>
> How will you ensure the estimated sigma are always >0 without constraints?
>
> > If I only wanna use fminsearch (by using fmincon, the results always come to NAN, dunno why), how could I determine if it convergent to an optimal point? Just by plug my estimated results back in fminsearch again, and see if L still goes down?
> ===================
>
> You should try FMINCON's sqp algorithm. If certain regions of your function are NaN or non-finite (as is your case), the sqp algorithm can avoid those regions.
>
> To determine whether a point is an unconstrained minimizer, it is sufficient to check whether the gradient is zero or approximately so.

sigh, I have to admit the following:
1)I don't know how to use FMINCON'S sqp algorithm. Could you please provide some details?
2)How to check whether the gradient is zero? Is there any code to do it?

"Lizzy " <lizzy1230@gmail.com> wrote in message <kae7us$pl4$1@newscl01ah.mathworks.com>...
> "Matt J" wrote in message <kae5jt$jh0$1@newscl01ah.mathworks.com>...
> > "Lizzy " <lizzy1230@gmail.com> wrote in message <kae3m1$dlv$1@newscl01ah.mathworks.com>...
> > >
> > >
> > > Thanks, Well, to make my life easier----is it necessary to impose a constraint of sigma>0 in mle estimation, if my estimated results for sigma are always >0?
> > =============
> >
> > How will you ensure the estimated sigma are always >0 without constraints?
> >
> > > If I only wanna use fminsearch (by using fmincon, the results always come to NAN, dunno why), how could I determine if it convergent to an optimal point? Just by plug my estimated results back in fminsearch again, and see if L still goes down?
> > ===================
> >
> > You should try FMINCON's sqp algorithm. If certain regions of your function are NaN or non-finite (as is your case), the sqp algorithm can avoid those regions.
> >
> > To determine whether a point is an unconstrained minimizer, it is sufficient to check whether the gradient is zero or approximately so.
>
> sigh, I have to admit the following:
> 1)I don't know how to use FMINCON'S sqp algorithm. Could you please provide some details?

Show how you've been running FMINCON so far.

> 2)How to check whether the gradient is zero? Is there any code to do it?

You could calculate the formula for your gradient and check that way. Alternatively,
FMINCON and FMINUNC have many output arguments. As you will see in the documentation, one of the outputs is the gradient at the final solution. If you run for zero iterations, it should return the gradient at the initial guess, which can be any vector you want.

>the current x satisfies the termination criteria using OPTIONS.TolX of 1.000000e-020

By the way, 1e-20 looks like a super-ambitious tolerance. No doubt that's why you have to run such a huge number of iterations to achieve it. I can't imagine why you would need so much precision. In any case, to compare fairly with FMINCON, you would have to run it with the same tolerance and with however many iterations is required to achieve that.

"Lizzy " <lizzy1230@gmail.com> wrote in message
news:kadsoq$lg2$1@newscl01ah.mathworks.com...
> "Matt J" wrote in message <kadrlf$i7v$1@newscl01ah.mathworks.com>...
>> "Lizzy " <lizzy1230@gmail.com> wrote in message
>> <kadqg5$ehk$1@newscl01ah.mathworks.com>...
>> > Hi,
>> > I'm trying to estimate coefficients and sigma in a MLE function. I
>> > guess sigma is around 9. But when I use fminsearch, the result is very
>> > different from when I use fmincon, constrainted sigma to be larger than
>> > 0.0000000001, even though I set the same initial value to sigma (set as
>> > 9). Can anyone please explain to me why it is like this?
>> ==============
>>
>> A few ideas,
>>
>> (1) You're running a constrained optimization with FMINCON but an
>> unconstrained optimization with FMINSEARCH. So, why shouldn't their
>> results be different?
>>
>> (2) FMINSEARCH uses a heuristic algorithm, not gaurnateed to converge
>> except for problems with a single unknown. Conversely, FMINCON uses
>> algorithms which are more robustly convergent in multi-variable problems.
>> How many unknowns do you have?
>>
>> (3) What reason do we have to believe the solution is unique?
>
> Well, when I use fminsearch, the results are better.

In one sense that's not too surprising. FMINSEARCH is an unconstrained
solver. If I wanted to travel from Natick to Los Angeles, California and I
had no constraints I'd charter a flight for myself (plenty of legroom, no
screaming babies) and fly in style. Since money is a significant, binding
constraint that eliminates that as a solution, I'd buy a ticket on a
commercial flight and have a reasonable trip. Less leg room, possibly more
screaming babies, but much less expensive.

> And it says at the end
> Optimization terminated:
> the current x satisfies the termination criteria using OPTIONS.TolX of
> 1.000000e-020

That's QUITE tight. I recommend relaxing this by quite a bit. If you do
ABSOLUTELY need that tight a tolerance for some reason, solve the system
once with a loose tolerance to find a close approximation to your solution,
then solve the system again with the first system's result as your initial
guess and the tighter tolerance.

*snip*

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

"Matt J" wrote in message <kae8p4$rks$1@newscl01ah.mathworks.com>...
> "Lizzy " <lizzy1230@gmail.com> wrote in message <kae7us$pl4$1@newscl01ah.mathworks.com>...
> > "Matt J" wrote in message <kae5jt$jh0$1@newscl01ah.mathworks.com>...
> > > "Lizzy " <lizzy1230@gmail.com> wrote in message <kae3m1$dlv$1@newscl01ah.mathworks.com>...
> > > >
> > > >
> > > > Thanks, Well, to make my life easier----is it necessary to impose a constraint of sigma>0 in mle estimation, if my estimated results for sigma are always >0?
> > > =============
> > >
> > > How will you ensure the estimated sigma are always >0 without constraints?
> > >
> > > > If I only wanna use fminsearch (by using fmincon, the results always come to NAN, dunno why), how could I determine if it convergent to an optimal point? Just by plug my estimated results back in fminsearch again, and see if L still goes down?
> > > ===================
> > >
> > > You should try FMINCON's sqp algorithm. If certain regions of your function are NaN or non-finite (as is your case), the sqp algorithm can avoid those regions.
> > >
> > > To determine whether a point is an unconstrained minimizer, it is sufficient to check whether the gradient is zero or approximately so.
> >
> > sigh, I have to admit the following:
> > 1)I don't know how to use FMINCON'S sqp algorithm. Could you please provide some details?
>
> Show how you've been running FMINCON so far.
>
> > 2)How to check whether the gradient is zero? Is there any code to do it?
>
> You could calculate the formula for your gradient and check that way. Alternatively,
> FMINCON and FMINUNC have many output arguments. As you will see in the documentation, one of the outputs is the gradient at the final solution. If you run for zero iterations, it should return the gradient at the initial guess, which can be any vector you want.


When I'm using fmincon, the result shows
 Iter F-count f(x) constraint steplength derivative optimality Procedure
    0 18 41585.6 -9
    1 40 41584.3 -9.111 0.0625 -7.04 19.6
    2 63 41580.7 -8.925 0.0313 -6.53 16.8
    3 81 41571.4 -9.006 1 -2.61 6.38
    4 107 41520.5 -9.083 0.00391 -5.37 136 Hessian modified
    5 142 41454.3 -9.099 7.63e-006 -56.4 541 Hessian modified
    6 179 41339.6 -9.107 1.91e-006 -202 1.84e+003 Hessian modified
    7 242 41207.4 -9.11 2.84e-014 -660 4.08e+003 Hessian modified twice
    8 278 39010.3 6.692 -3.81e-006 -1.43e+003 441 Hessian modified twice
    9 296 NaN -2.804e-016 1 11.5 Inf Hessian modified twice
   10 314 NaN NaN 1 NaN Inf Hessian not updated
   11 332 NaN NaN 1 NaN Inf Hessian not updated
   12 350 NaN NaN 1 NaN Inf Hessian not updated

and so on.
What do you think could be the problem?
And is there no way that I can find the gradient using fminsearch?

Thanks.

"Steven_Lord" <slord@mathworks.com> wrote in message <kaffio$ja8$1@newscl01ah.mathworks.com>...
>
>
> "Lizzy " <lizzy1230@gmail.com> wrote in message
> news:kadsoq$lg2$1@newscl01ah.mathworks.com...
> > "Matt J" wrote in message <kadrlf$i7v$1@newscl01ah.mathworks.com>...
> >> "Lizzy " <lizzy1230@gmail.com> wrote in message
> >> <kadqg5$ehk$1@newscl01ah.mathworks.com>...
> >> > Hi,
> >> > I'm trying to estimate coefficients and sigma in a MLE function. I
> >> > guess sigma is around 9. But when I use fminsearch, the result is very
> >> > different from when I use fmincon, constrainted sigma to be larger than
> >> > 0.0000000001, even though I set the same initial value to sigma (set as
> >> > 9). Can anyone please explain to me why it is like this?
> >> ==============
> >>
> >> A few ideas,
> >>
> >> (1) You're running a constrained optimization with FMINCON but an
> >> unconstrained optimization with FMINSEARCH. So, why shouldn't their
> >> results be different?
> >>
> >> (2) FMINSEARCH uses a heuristic algorithm, not gaurnateed to converge
> >> except for problems with a single unknown. Conversely, FMINCON uses
> >> algorithms which are more robustly convergent in multi-variable problems.
> >> How many unknowns do you have?
> >>
> >> (3) What reason do we have to believe the solution is unique?
> >
> > Well, when I use fminsearch, the results are better.
>
> In one sense that's not too surprising. FMINSEARCH is an unconstrained
> solver. If I wanted to travel from Natick to Los Angeles, California and I
> had no constraints I'd charter a flight for myself (plenty of legroom, no
> screaming babies) and fly in style. Since money is a significant, binding
> constraint that eliminates that as a solution, I'd buy a ticket on a
> commercial flight and have a reasonable trip. Less leg room, possibly more
> screaming babies, but much less expensive.
>
> > And it says at the end
> > Optimization terminated:
> > the current x satisfies the termination criteria using OPTIONS.TolX of
> > 1.000000e-020
>
> That's QUITE tight. I recommend relaxing this by quite a bit. If you do
> ABSOLUTELY need that tight a tolerance for some reason, solve the system
> once with a loose tolerance to find a close approximation to your solution,
> then solve the system again with the first system's result as your initial
> guess and the tighter tolerance.
>
> *snip*
>
> --
> Steve Lord
> slord@mathworks.com
> To contact Technical Support use the Contact Us link on
> http://www.mathworks.com

---------------------------------------------------------------------------------------------------------------------------------------------------------
And can anyone please tell me if this code is correct?

coefInit = [1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;9];
options_fminsearch = optimset('Display','iter','TolFun',1e-4,'TolX',1e-4,'MaxIt',10000000,'MaxFunEvals',10000000000000);
[coefEst,fval,exitflag,output] = fmincon(@(coef) llk(coef,q,p,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12),coefInit,A,b,[],[],[],[],[],options_fminsearch);

On 12/14/2012 10:47 AM, Lizzy wrote:
> And can anyone please tell me if this code is correct?
>
> coefInit = [1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;9];
> options_fminsearch =
> optimset('Display','iter','TolFun',1e-4,'TolX',1e-4,'MaxIt',10000000,'MaxFunEvals',10000000000000);
> [coefEst,fval,exitflag,output] = fmincon(@(coef)
> llk(coef,q,p,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12),coefInit,A,b,[],[],[],[],[],options_fminsearch);

I don't know why you reset the fmincon options TolFun and TolX, and I
don't know why you call it options_fminsearch, and I don't know why you
don't use the 'sqp' or 'interior-point' algorithms.

But the syntax looks correct. And with that many dimensions (17), you
probably wouldn't get reliable fminsearch results.

Alan Weiss
MATLAB mathematical toolbox documentation

Alan_Weiss <aweiss@mathworks.com> wrote in message <kafrbk$69$1@newscl01ah.mathworks.com>...
> On 12/14/2012 10:47 AM, Lizzy wrote:
> > And can anyone please tell me if this code is correct?
> >
> > coefInit = [1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;9];
> > options_fminsearch =
> > optimset('Display','iter','TolFun',1e-4,'TolX',1e-4,'MaxIt',10000000,'MaxFunEvals',10000000000000);
> > [coefEst,fval,exitflag,output] = fmincon(@(coef)
> > llk(coef,q,p,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12),coefInit,A,b,[],[],[],[],[],options_fminsearch);
>
> I don't know why you reset the fmincon options TolFun and TolX, and I
> don't know why you call it options_fminsearch, and I don't know why you
> don't use the 'sqp' or 'interior-point' algorithms.
>
> But the syntax looks correct. And with that many dimensions (17), you
> probably wouldn't get reliable fminsearch results.
>
> Alan Weiss
> MATLAB mathematical toolbox documentation

Thanks. And after I get the coefficients, I also want the standard errors so that I can calculate significance. Can anyone please let me know how can I get the standard errors?

"Lizzy " <lookforaroommate@126.com> wrote in message <kafhls$qip$1@newscl01ah.mathworks.com>...
>
> And can anyone please tell me if this code is correct?
>
> coefInit = [1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;9];
> options_fminsearch = optimset('Display','iter','TolFun',1e-4,'TolX',1e-4,'MaxIt',10000000,'MaxFunEvals',10000000000000);
> [coefEst,fval,exitflag,output] = fmincon(@(coef) llk(coef,q,p,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12),coefInit,A,b,[],[],[],[],[],options_fminsearch);
==============

Since you only have upper and lower bound constraints, like sigma>=eps, you should be using the lb and ub input arguments, rather than A and b.

Also, as Alan and I mentioned, you should use optimset to set the 'Algorithm' option to something like 'interior-point' or 'sqp' that know how to deal with the NaN regions of your objective function.

"Matt J" wrote in message <kag5bs$44v$1@newscl01ah.mathworks.com>...
> "Lizzy " <lookforaroommate@126.com> wrote in message <kafhls$qip$1@newscl01ah.mathworks.com>...
> >
> > And can anyone please tell me if this code is correct?
> >
> > coefInit = [1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;9];
> > options_fminsearch = optimset('Display','iter','TolFun',1e-4,'TolX',1e-4,'MaxIt',10000000,'MaxFunEvals',10000000000000);
> > [coefEst,fval,exitflag,output] = fmincon(@(coef) llk(coef,q,p,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12),coefInit,A,b,[],[],[],[],[],options_fminsearch);
> ==============
>
> Since you only have upper and lower bound constraints, like sigma>=eps, you should be using the lb and ub input arguments, rather than A and b.

Hi Matt, I don't really think I can use lower bound because I will only constraint one of the parameters that I want to estimate. If I use lower bound, won't it be like lb=[0;0;0;0;...0.00001], which means all the others are constrainted>=0?
>
> Also, as Alan and I mentioned, you should use optimset to set the 'Algorithm' option to something like 'interior-point' or 'sqp' that know how to deal with the NaN regions of your objective function.

"Lizzy " <lookforaroommate@126.com> wrote in message
news:kagk9u$ged$1@newscl01ah.mathworks.com...
> "Matt J" wrote in message <kag5bs$44v$1@newscl01ah.mathworks.com>...
>> "Lizzy " <lookforaroommate@126.com> wrote in message
>> <kafhls$qip$1@newscl01ah.mathworks.com>...
>> >
>> > And can anyone please tell me if this code is correct?
>> >
>> > coefInit = [1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;9];
>> > options_fminsearch =
>> > optimset('Display','iter','TolFun',1e-4,'TolX',1e-4,'MaxIt',10000000,'MaxFunEvals',10000000000000);
>> > [coefEst,fval,exitflag,output] = fmincon(@(coef)
>> > llk(coef,q,p,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12),coefInit,A,b,[],[],[],[],[],options_fminsearch);
>> ==============
>>
>> Since you only have upper and lower bound constraints, like sigma>=eps,
>> you should be using the lb and ub input arguments, rather than A and b.
>
> Hi Matt, I don't really think I can use lower bound because I will only
> constraint one of the parameters that I want to estimate. If I use lower
> bound, won't it be like lb=[0;0;0;0;...0.00001], which means all the
> others are constrainted>=0?

Yes, if you did that it would constrain all your other parameter values to
be nonnegative. On the other hand, if you did:

lb = [-Inf; -Inf; -Inf; ... ;1e-6];

it would constrain all but your last parameter to be greater than or equal
to -Inf and the last parameter to be greater than or equal to 1e-6. The same
idea holds for ub and +Inf.

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com