http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199
MATLAB Central Newsreader  use fminsearch and fmincon to find MLE
Feed for thread: use fminsearch and fmincon to find MLE
enus
©19942015 by MathWorks, Inc.
webmaster@mathworks.com
MATLAB Central Newsreader
http://blogs.law.harvard.edu/tech/rss
60
MathWorks
http://www.mathworks.com/images/membrane_icon.gif

Fri, 14 Dec 2012 00:05:25 +0000
use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893698
Lizzy
Hi,<br>
I'm trying to estimate coefficients and sigma in a MLE function. I guess sigma is around 9. But when I use fminsearch, the result is very different from when I use fmincon, constrainted sigma to be larger than 0.0000000001, even though I set the same initial value to sigma (set as 9). Can anyone please explain to me why it is like this?

Fri, 14 Dec 2012 00:25:19 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893699
Matt J
"Lizzy " <lizzy1230@gmail.com> wrote in message <kadqg5$ehk$1@newscl01ah.mathworks.com>...<br>
> Hi,<br>
> I'm trying to estimate coefficients and sigma in a MLE function. I guess sigma is around 9. But when I use fminsearch, the result is very different from when I use fmincon, constrainted sigma to be larger than 0.0000000001, even though I set the same initial value to sigma (set as 9). Can anyone please explain to me why it is like this?<br>
==============<br>
<br>
A few ideas,<br>
<br>
(1) You're running a constrained optimization with FMINCON but an unconstrained optimization with FMINSEARCH. So, why shouldn't their results be different?<br>
<br>
(2) FMINSEARCH uses a heuristic algorithm, not gaurnateed to converge except for problems with a single unknown. Conversely, FMINCON uses algorithms which are more robustly convergent in multivariable problems. How many unknowns do you have?<br>
<br>
(3) What reason do we have to believe the solution is unique?

Fri, 14 Dec 2012 00:44:10 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893701
Lizzy
"Matt J" wrote in message <kadrlf$i7v$1@newscl01ah.mathworks.com>...<br>
> "Lizzy " <lizzy1230@gmail.com> wrote in message <kadqg5$ehk$1@newscl01ah.mathworks.com>...<br>
> > Hi,<br>
> > I'm trying to estimate coefficients and sigma in a MLE function. I guess sigma is around 9. But when I use fminsearch, the result is very different from when I use fmincon, constrainted sigma to be larger than 0.0000000001, even though I set the same initial value to sigma (set as 9). Can anyone please explain to me why it is like this?<br>
> ==============<br>
> <br>
> A few ideas,<br>
> <br>
> (1) You're running a constrained optimization with FMINCON but an unconstrained optimization with FMINSEARCH. So, why shouldn't their results be different?<br>
> <br>
> (2) FMINSEARCH uses a heuristic algorithm, not gaurnateed to converge except for problems with a single unknown. Conversely, FMINCON uses algorithms which are more robustly convergent in multivariable problems. How many unknowns do you have?<br>
> <br>
> (3) What reason do we have to believe the solution is unique?<br>
<br>
Well, when I use fminsearch, the results are better. And it says at the end<br>
Optimization terminated:<br>
the current x satisfies the termination criteria using OPTIONS.TolX of 1.000000e020 <br>
and F(X) satisfies the convergence criteria using OPTIONS.TolFun of 1.000000e010<br>
So does it mean that it converges?<br>
And I have many coefficients: 17. But all the coefficients are not constrained except for sigma (standard deviation of normal).<br>
I'm now trying to use fminsearch, but change wherever there is a sigma to abs(sigma)+eps. And the final result of sigma I want to estimated is calculated by abs(result of sigma)+eps. (and the result is the same if I don't use this abs(.)+eps. Is it a correct way?

Fri, 14 Dec 2012 01:02:08 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893702
Matt J
"Lizzy " <lizzy1230@gmail.com> wrote in message <kadsoq$lg2$1@newscl01ah.mathworks.com>...<br>
><br>
> Well, when I use fminsearch, the results are better. <br>
============<br>
<br>
It's not clear what you mean by "the results are better". How are you measuring "better"?<br>
<br>
<br>
> Optimization terminated:<br>
> the current x satisfies the termination criteria using OPTIONS.TolX of 1.000000e020 <br>
> and F(X) satisfies the convergence criteria using OPTIONS.TolFun of 1.000000e010<br>
> So does it mean that it converges?<br>
<br>
<br>
It means that it converged to something, but not necessarily to an optimal point. You could plug the solution given by fminsearch into fmincon and see if it progresses further downhill. <br>
<br>
<br>
> I'm now trying to use fminsearch, but change wherever there is a sigma to abs(sigma)+eps. And the final result of sigma I want to estimated is calculated by abs(result of sigma)+eps. (and the result is the same if I don't use this abs(.)+eps. Is it a correct way?<br>
============<br>
<br>
It sounds reasonable for FMINSEARCH. You couldn't use this objective in FMINCON because FMINCON only handles differentiable functions.

Fri, 14 Dec 2012 01:20:09 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893703
Lizzy
"Matt J" wrote in message <kadtqg$ou4$1@newscl01ah.mathworks.com>...<br>
> "Lizzy " <lizzy1230@gmail.com> wrote in message <kadsoq$lg2$1@newscl01ah.mathworks.com>...<br>
> ><br>
> > Well, when I use fminsearch, the results are better. <br>
> ============<br>
> <br>
> It's not clear what you mean by "the results are better". How are you measuring "better"?<br>
> <br>
> <br>
> > Optimization terminated:<br>
> > the current x satisfies the termination criteria using OPTIONS.TolX of 1.000000e020 <br>
> > and F(X) satisfies the convergence criteria using OPTIONS.TolFun of 1.000000e010<br>
> > So does it mean that it converges?<br>
> <br>
> <br>
> It means that it converged to something, but not necessarily to an optimal point. You could plug the solution given by fminsearch into fmincon and see if it progresses further downhill. <br>
> <br>
> <br>
> > I'm now trying to use fminsearch, but change wherever there is a sigma to abs(sigma)+eps. And the final result of sigma I want to estimated is calculated by abs(result of sigma)+eps. (and the result is the same if I don't use this abs(.)+eps. Is it a correct way?<br>
> ============<br>
> <br>
> It sounds reasonable for FMINSEARCH. You couldn't use this objective in FMINCON because FMINCON only handles differentiable functions.<br>
<br>
Thanks Matt. But what if I have no idea if my function is differentiable or not? And if I use fminsearch, is it ok that I set sigma as abs(sigma)+eps to make it greater than 0? And my code is as follows:<br>
<br>
coefInit = [1,1,1,1,1....1]];<br>
options_fminsearch = optimset('Display','iter','TolFun',1e10,'TolX',1e20,'MaxIt',10000000,'MaxFunEvals',10000000000000);<br>
coefEst = fminsearch(@(coef) llk(coef,x1....xn),coefInit,options_fminsearch);<br>
<br>
And <br>
aa = log(phi/(abs(sigma)+eps));<br>
aa(isnan(aa))=inf;<br>
bb = log(1phi2);<br>
bb(isnan(bb))=inf;<br>
llk = d.*aa+(1d).*bb;<br>
<br>
res = sum(llk);<br>
<br>
Is it correct? Thanks a lot.

Fri, 14 Dec 2012 01:54:09 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893706
Matt J
"Lizzy " <lizzy1230@gmail.com> wrote in message <kadus9$rpo$1@newscl01ah.mathworks.com>...<br>
><br>
> Thanks Matt. But what if I have no idea if my function is differentiable or not?<br>
<br>
You should know!<br>
<br>
In this case, because abs(sigma) is a not a continuously differentiable function of sigma (at sigma=0), it's pretty clear that plugging it into a smooth functions like log() etc... will lead to something that is also not continuously differentiable. As I said before, that doesn't matter for FMINSEARCH, because FMINSEARCH doesn't use an algorithm that calculates derivatives. You would have to use a different objective function for FMINCON, however, and that makes comparison between with FMINSEARCH difficult.<br>
<br>
On the other hand, instead of using abs(sigma), you could just make the change of variables<br>
<br>
sigma=z^2+eps<br>
<br>
which is differentiable and could be used in FMINCON or any other Opt Toolbox solver. Although now, because you're not constraining the new variable z, you could use FMINUNC instead.

Fri, 14 Dec 2012 02:42:09 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893708
Lizzy
"Matt J" wrote in message <kae0s1$4os$1@newscl01ah.mathworks.com>...<br>
> "Lizzy " <lizzy1230@gmail.com> wrote in message <kadus9$rpo$1@newscl01ah.mathworks.com>...<br>
> ><br>
> > Thanks Matt. But what if I have no idea if my function is differentiable or not?<br>
> <br>
> You should know!<br>
> <br>
> In this case, because abs(sigma) is a not a continuously differentiable function of sigma (at sigma=0), it's pretty clear that plugging it into a smooth functions like log() etc... will lead to something that is also not continuously differentiable. As I said before, that doesn't matter for FMINSEARCH, because FMINSEARCH doesn't use an algorithm that calculates derivatives. You would have to use a different objective function for FMINCON, however, and that makes comparison between with FMINSEARCH difficult.<br>
> <br>
> On the other hand, instead of using abs(sigma), you could just make the change of variables<br>
> <br>
> sigma=z^2+eps<br>
> <br>
> which is differentiable and could be used in FMINCON or any other Opt Toolbox solver. Although now, because you're not constraining the new variable z, you could use FMINUNC instead.<br>
<br>
<br>
Thanks, Well, to make my life easieris it necessary to impose a constraint of sigma>0 in mle estimation, if my estimated results for sigma are always >0?<br>
If I only wanna use fminsearch (by using fmincon, the results always come to NAN, dunno why), how could I determine if it convergent to an optimal point? Just by plug my estimated results back in fminsearch again, and see if L still goes down?

Fri, 14 Dec 2012 03:15:09 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893710
Matt J
"Lizzy " <lizzy1230@gmail.com> wrote in message <kae3m1$dlv$1@newscl01ah.mathworks.com>...<br>
><br>
> <br>
> Thanks, Well, to make my life easieris it necessary to impose a constraint of sigma>0 in mle estimation, if my estimated results for sigma are always >0?<br>
=============<br>
<br>
How will you ensure the estimated sigma are always >0 without constraints?<br>
<br>
> If I only wanna use fminsearch (by using fmincon, the results always come to NAN, dunno why), how could I determine if it convergent to an optimal point? Just by plug my estimated results back in fminsearch again, and see if L still goes down?<br>
===================<br>
<br>
You should try FMINCON's sqp algorithm. If certain regions of your function are NaN or nonfinite (as is your case), the sqp algorithm can avoid those regions.<br>
<br>
To determine whether a point is an unconstrained minimizer, it is sufficient to check whether the gradient is zero or approximately so.

Fri, 14 Dec 2012 03:55:08 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893711
Lizzy
"Matt J" wrote in message <kae5jt$jh0$1@newscl01ah.mathworks.com>...<br>
> "Lizzy " <lizzy1230@gmail.com> wrote in message <kae3m1$dlv$1@newscl01ah.mathworks.com>...<br>
> ><br>
> > <br>
> > Thanks, Well, to make my life easieris it necessary to impose a constraint of sigma>0 in mle estimation, if my estimated results for sigma are always >0?<br>
> =============<br>
> <br>
> How will you ensure the estimated sigma are always >0 without constraints?<br>
> <br>
> > If I only wanna use fminsearch (by using fmincon, the results always come to NAN, dunno why), how could I determine if it convergent to an optimal point? Just by plug my estimated results back in fminsearch again, and see if L still goes down?<br>
> ===================<br>
> <br>
> You should try FMINCON's sqp algorithm. If certain regions of your function are NaN or nonfinite (as is your case), the sqp algorithm can avoid those regions.<br>
> <br>
> To determine whether a point is an unconstrained minimizer, it is sufficient to check whether the gradient is zero or approximately so.<br>
<br>
sigh, I have to admit the following:<br>
1)I don't know how to use FMINCON'S sqp algorithm. Could you please provide some details?<br>
2)How to check whether the gradient is zero? Is there any code to do it?

Fri, 14 Dec 2012 04:09:08 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893713
Matt J
"Lizzy " <lizzy1230@gmail.com> wrote in message <kae7us$pl4$1@newscl01ah.mathworks.com>...<br>
> "Matt J" wrote in message <kae5jt$jh0$1@newscl01ah.mathworks.com>...<br>
> > "Lizzy " <lizzy1230@gmail.com> wrote in message <kae3m1$dlv$1@newscl01ah.mathworks.com>...<br>
> > ><br>
> > > <br>
> > > Thanks, Well, to make my life easieris it necessary to impose a constraint of sigma>0 in mle estimation, if my estimated results for sigma are always >0?<br>
> > =============<br>
> > <br>
> > How will you ensure the estimated sigma are always >0 without constraints?<br>
> > <br>
> > > If I only wanna use fminsearch (by using fmincon, the results always come to NAN, dunno why), how could I determine if it convergent to an optimal point? Just by plug my estimated results back in fminsearch again, and see if L still goes down?<br>
> > ===================<br>
> > <br>
> > You should try FMINCON's sqp algorithm. If certain regions of your function are NaN or nonfinite (as is your case), the sqp algorithm can avoid those regions.<br>
> > <br>
> > To determine whether a point is an unconstrained minimizer, it is sufficient to check whether the gradient is zero or approximately so.<br>
> <br>
> sigh, I have to admit the following:<br>
> 1)I don't know how to use FMINCON'S sqp algorithm. Could you please provide some details?<br>
<br>
Show how you've been running FMINCON so far.<br>
<br>
> 2)How to check whether the gradient is zero? Is there any code to do it?<br>
<br>
You could calculate the formula for your gradient and check that way. Alternatively,<br>
FMINCON and FMINUNC have many output arguments. As you will see in the documentation, one of the outputs is the gradient at the final solution. If you run for zero iterations, it should return the gradient at the initial guess, which can be any vector you want.

Fri, 14 Dec 2012 04:12:08 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893714
Matt J
>the current x satisfies the termination criteria using OPTIONS.TolX of 1.000000e020 <br>
<br>
By the way, 1e20 looks like a superambitious tolerance. No doubt that's why you have to run such a huge number of iterations to achieve it. I can't imagine why you would need so much precision. In any case, to compare fairly with FMINCON, you would have to run it with the same tolerance and with however many iterations is required to achieve that.

Fri, 14 Dec 2012 15:11:20 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893742
Steven_Lord
<br>
<br>
"Lizzy " <lizzy1230@gmail.com> wrote in message <br>
news:kadsoq$lg2$1@newscl01ah.mathworks.com...<br>
> "Matt J" wrote in message <kadrlf$i7v$1@newscl01ah.mathworks.com>...<br>
>> "Lizzy " <lizzy1230@gmail.com> wrote in message <br>
>> <kadqg5$ehk$1@newscl01ah.mathworks.com>...<br>
>> > Hi,<br>
>> > I'm trying to estimate coefficients and sigma in a MLE function. I <br>
>> > guess sigma is around 9. But when I use fminsearch, the result is very <br>
>> > different from when I use fmincon, constrainted sigma to be larger than <br>
>> > 0.0000000001, even though I set the same initial value to sigma (set as <br>
>> > 9). Can anyone please explain to me why it is like this?<br>
>> ==============<br>
>><br>
>> A few ideas,<br>
>><br>
>> (1) You're running a constrained optimization with FMINCON but an <br>
>> unconstrained optimization with FMINSEARCH. So, why shouldn't their <br>
>> results be different?<br>
>><br>
>> (2) FMINSEARCH uses a heuristic algorithm, not gaurnateed to converge <br>
>> except for problems with a single unknown. Conversely, FMINCON uses <br>
>> algorithms which are more robustly convergent in multivariable problems. <br>
>> How many unknowns do you have?<br>
>><br>
>> (3) What reason do we have to believe the solution is unique?<br>
><br>
> Well, when I use fminsearch, the results are better.<br>
<br>
In one sense that's not too surprising. FMINSEARCH is an unconstrained <br>
solver. If I wanted to travel from Natick to Los Angeles, California and I <br>
had no constraints I'd charter a flight for myself (plenty of legroom, no <br>
screaming babies) and fly in style. Since money is a significant, binding <br>
constraint that eliminates that as a solution, I'd buy a ticket on a <br>
commercial flight and have a reasonable trip. Less leg room, possibly more <br>
screaming babies, but much less expensive.<br>
<br>
> And it says at the end<br>
> Optimization terminated:<br>
> the current x satisfies the termination criteria using OPTIONS.TolX of <br>
> 1.000000e020<br>
<br>
That's QUITE tight. I recommend relaxing this by quite a bit. If you do <br>
ABSOLUTELY need that tight a tolerance for some reason, solve the system <br>
once with a loose tolerance to find a close approximation to your solution, <br>
then solve the system again with the first system's result as your initial <br>
guess and the tighter tolerance.<br>
<br>
*snip*<br>
<br>
 <br>
Steve Lord<br>
slord@mathworks.com<br>
To contact Technical Support use the Contact Us link on <br>
<a href="http://www.mathworks.com">http://www.mathworks.com</a>

Fri, 14 Dec 2012 15:40:08 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893747
Lizzy
"Matt J" wrote in message <kae8p4$rks$1@newscl01ah.mathworks.com>...<br>
> "Lizzy " <lizzy1230@gmail.com> wrote in message <kae7us$pl4$1@newscl01ah.mathworks.com>...<br>
> > "Matt J" wrote in message <kae5jt$jh0$1@newscl01ah.mathworks.com>...<br>
> > > "Lizzy " <lizzy1230@gmail.com> wrote in message <kae3m1$dlv$1@newscl01ah.mathworks.com>...<br>
> > > ><br>
> > > > <br>
> > > > Thanks, Well, to make my life easieris it necessary to impose a constraint of sigma>0 in mle estimation, if my estimated results for sigma are always >0?<br>
> > > =============<br>
> > > <br>
> > > How will you ensure the estimated sigma are always >0 without constraints?<br>
> > > <br>
> > > > If I only wanna use fminsearch (by using fmincon, the results always come to NAN, dunno why), how could I determine if it convergent to an optimal point? Just by plug my estimated results back in fminsearch again, and see if L still goes down?<br>
> > > ===================<br>
> > > <br>
> > > You should try FMINCON's sqp algorithm. If certain regions of your function are NaN or nonfinite (as is your case), the sqp algorithm can avoid those regions.<br>
> > > <br>
> > > To determine whether a point is an unconstrained minimizer, it is sufficient to check whether the gradient is zero or approximately so.<br>
> > <br>
> > sigh, I have to admit the following:<br>
> > 1)I don't know how to use FMINCON'S sqp algorithm. Could you please provide some details?<br>
> <br>
> Show how you've been running FMINCON so far.<br>
> <br>
> > 2)How to check whether the gradient is zero? Is there any code to do it?<br>
> <br>
> You could calculate the formula for your gradient and check that way. Alternatively,<br>
> FMINCON and FMINUNC have many output arguments. As you will see in the documentation, one of the outputs is the gradient at the final solution. If you run for zero iterations, it should return the gradient at the initial guess, which can be any vector you want.<br>
<br>
<br>
When I'm using fmincon, the result shows<br>
Iter Fcount f(x) constraint steplength derivative optimality Procedure <br>
0 18 41585.6 9 <br>
1 40 41584.3 9.111 0.0625 7.04 19.6 <br>
2 63 41580.7 8.925 0.0313 6.53 16.8 <br>
3 81 41571.4 9.006 1 2.61 6.38 <br>
4 107 41520.5 9.083 0.00391 5.37 136 Hessian modified <br>
5 142 41454.3 9.099 7.63e006 56.4 541 Hessian modified <br>
6 179 41339.6 9.107 1.91e006 202 1.84e+003 Hessian modified <br>
7 242 41207.4 9.11 2.84e014 660 4.08e+003 Hessian modified twice <br>
8 278 39010.3 6.692 3.81e006 1.43e+003 441 Hessian modified twice <br>
9 296 NaN 2.804e016 1 11.5 Inf Hessian modified twice <br>
10 314 NaN NaN 1 NaN Inf Hessian not updated <br>
11 332 NaN NaN 1 NaN Inf Hessian not updated <br>
12 350 NaN NaN 1 NaN Inf Hessian not updated<br>
<br>
and so on.<br>
What do you think could be the problem?<br>
And is there no way that I can find the gradient using fminsearch?<br>
<br>
Thanks.

Fri, 14 Dec 2012 15:47:08 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893749
Lizzy
"Steven_Lord" <slord@mathworks.com> wrote in message <kaffio$ja8$1@newscl01ah.mathworks.com>...<br>
> <br>
> <br>
> "Lizzy " <lizzy1230@gmail.com> wrote in message <br>
> news:kadsoq$lg2$1@newscl01ah.mathworks.com...<br>
> > "Matt J" wrote in message <kadrlf$i7v$1@newscl01ah.mathworks.com>...<br>
> >> "Lizzy " <lizzy1230@gmail.com> wrote in message <br>
> >> <kadqg5$ehk$1@newscl01ah.mathworks.com>...<br>
> >> > Hi,<br>
> >> > I'm trying to estimate coefficients and sigma in a MLE function. I <br>
> >> > guess sigma is around 9. But when I use fminsearch, the result is very <br>
> >> > different from when I use fmincon, constrainted sigma to be larger than <br>
> >> > 0.0000000001, even though I set the same initial value to sigma (set as <br>
> >> > 9). Can anyone please explain to me why it is like this?<br>
> >> ==============<br>
> >><br>
> >> A few ideas,<br>
> >><br>
> >> (1) You're running a constrained optimization with FMINCON but an <br>
> >> unconstrained optimization with FMINSEARCH. So, why shouldn't their <br>
> >> results be different?<br>
> >><br>
> >> (2) FMINSEARCH uses a heuristic algorithm, not gaurnateed to converge <br>
> >> except for problems with a single unknown. Conversely, FMINCON uses <br>
> >> algorithms which are more robustly convergent in multivariable problems. <br>
> >> How many unknowns do you have?<br>
> >><br>
> >> (3) What reason do we have to believe the solution is unique?<br>
> ><br>
> > Well, when I use fminsearch, the results are better.<br>
> <br>
> In one sense that's not too surprising. FMINSEARCH is an unconstrained <br>
> solver. If I wanted to travel from Natick to Los Angeles, California and I <br>
> had no constraints I'd charter a flight for myself (plenty of legroom, no <br>
> screaming babies) and fly in style. Since money is a significant, binding <br>
> constraint that eliminates that as a solution, I'd buy a ticket on a <br>
> commercial flight and have a reasonable trip. Less leg room, possibly more <br>
> screaming babies, but much less expensive.<br>
> <br>
> > And it says at the end<br>
> > Optimization terminated:<br>
> > the current x satisfies the termination criteria using OPTIONS.TolX of <br>
> > 1.000000e020<br>
> <br>
> That's QUITE tight. I recommend relaxing this by quite a bit. If you do <br>
> ABSOLUTELY need that tight a tolerance for some reason, solve the system <br>
> once with a loose tolerance to find a close approximation to your solution, <br>
> then solve the system again with the first system's result as your initial <br>
> guess and the tighter tolerance.<br>
> <br>
> *snip*<br>
> <br>
>  <br>
> Steve Lord<br>
> slord@mathworks.com<br>
> To contact Technical Support use the Contact Us link on <br>
> <a href="http://www.mathworks.com">http://www.mathworks.com</a> <br>
<br>
<br>
And can anyone please tell me if this code is correct?<br>
<br>
coefInit = [1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;9];<br>
options_fminsearch = optimset('Display','iter','TolFun',1e4,'TolX',1e4,'MaxIt',10000000,'MaxFunEvals',10000000000000);<br>
[coefEst,fval,exitflag,output] = fmincon(@(coef) llk(coef,q,p,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12),coefInit,A,b,[],[],[],[],[],options_fminsearch);

Fri, 14 Dec 2012 18:32:20 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893757
Alan_Weiss
On 12/14/2012 10:47 AM, Lizzy wrote:<br>
> And can anyone please tell me if this code is correct?<br>
><br>
> coefInit = [1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;9];<br>
> options_fminsearch = <br>
> optimset('Display','iter','TolFun',1e4,'TolX',1e4,'MaxIt',10000000,'MaxFunEvals',10000000000000);<br>
> [coefEst,fval,exitflag,output] = fmincon(@(coef) <br>
> llk(coef,q,p,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12),coefInit,A,b,[],[],[],[],[],options_fminsearch);<br>
<br>
I don't know why you reset the fmincon options TolFun and TolX, and I <br>
don't know why you call it options_fminsearch, and I don't know why you <br>
don't use the 'sqp' or 'interiorpoint' algorithms.<br>
<br>
But the syntax looks correct. And with that many dimensions (17), you <br>
probably wouldn't get reliable fminsearch results.<br>
<br>
Alan Weiss<br>
MATLAB mathematical toolbox documentation

Fri, 14 Dec 2012 18:54:08 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893759
Lizzy
Alan_Weiss <aweiss@mathworks.com> wrote in message <kafrbk$69$1@newscl01ah.mathworks.com>...<br>
> On 12/14/2012 10:47 AM, Lizzy wrote:<br>
> > And can anyone please tell me if this code is correct?<br>
> ><br>
> > coefInit = [1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;9];<br>
> > options_fminsearch = <br>
> > optimset('Display','iter','TolFun',1e4,'TolX',1e4,'MaxIt',10000000,'MaxFunEvals',10000000000000);<br>
> > [coefEst,fval,exitflag,output] = fmincon(@(coef) <br>
> > llk(coef,q,p,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12),coefInit,A,b,[],[],[],[],[],options_fminsearch);<br>
> <br>
> I don't know why you reset the fmincon options TolFun and TolX, and I <br>
> don't know why you call it options_fminsearch, and I don't know why you <br>
> don't use the 'sqp' or 'interiorpoint' algorithms.<br>
> <br>
> But the syntax looks correct. And with that many dimensions (17), you <br>
> probably wouldn't get reliable fminsearch results.<br>
> <br>
> Alan Weiss<br>
> MATLAB mathematical toolbox documentation<br>
<br>
Thanks. And after I get the coefficients, I also want the standard errors so that I can calculate significance. Can anyone please let me know how can I get the standard errors?

Fri, 14 Dec 2012 21:23:08 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893770
Matt J
"Lizzy " <lookforaroommate@126.com> wrote in message <kafhls$qip$1@newscl01ah.mathworks.com>...<br>
><br>
> And can anyone please tell me if this code is correct?<br>
> <br>
> coefInit = [1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;9];<br>
> options_fminsearch = optimset('Display','iter','TolFun',1e4,'TolX',1e4,'MaxIt',10000000,'MaxFunEvals',10000000000000);<br>
> [coefEst,fval,exitflag,output] = fmincon(@(coef) llk(coef,q,p,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12),coefInit,A,b,[],[],[],[],[],options_fminsearch);<br>
==============<br>
<br>
Since you only have upper and lower bound constraints, like sigma>=eps, you should be using the lb and ub input arguments, rather than A and b.<br>
<br>
Also, as Alan and I mentioned, you should use optimset to set the 'Algorithm' option to something like 'interiorpoint' or 'sqp' that know how to deal with the NaN regions of your objective function.

Sat, 15 Dec 2012 01:38:06 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893776
Lizzy
"Matt J" wrote in message <kag5bs$44v$1@newscl01ah.mathworks.com>...<br>
> "Lizzy " <lookforaroommate@126.com> wrote in message <kafhls$qip$1@newscl01ah.mathworks.com>...<br>
> ><br>
> > And can anyone please tell me if this code is correct?<br>
> > <br>
> > coefInit = [1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;9];<br>
> > options_fminsearch = optimset('Display','iter','TolFun',1e4,'TolX',1e4,'MaxIt',10000000,'MaxFunEvals',10000000000000);<br>
> > [coefEst,fval,exitflag,output] = fmincon(@(coef) llk(coef,q,p,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12),coefInit,A,b,[],[],[],[],[],options_fminsearch);<br>
> ==============<br>
> <br>
> Since you only have upper and lower bound constraints, like sigma>=eps, you should be using the lb and ub input arguments, rather than A and b.<br>
<br>
Hi Matt, I don't really think I can use lower bound because I will only constraint one of the parameters that I want to estimate. If I use lower bound, won't it be like lb=[0;0;0;0;...0.00001], which means all the others are constrainted>=0?<br>
> <br>
> Also, as Alan and I mentioned, you should use optimset to set the 'Algorithm' option to something like 'interiorpoint' or 'sqp' that know how to deal with the NaN regions of your objective function.

Sat, 15 Dec 2012 02:28:42 +0000
Re: use fminsearch and fmincon to find MLE
http://www.mathworks.com/matlabcentral/newsreader/view_thread/325199#893779
Steven_Lord
<br>
<br>
"Lizzy " <lookforaroommate@126.com> wrote in message <br>
news:kagk9u$ged$1@newscl01ah.mathworks.com...<br>
> "Matt J" wrote in message <kag5bs$44v$1@newscl01ah.mathworks.com>...<br>
>> "Lizzy " <lookforaroommate@126.com> wrote in message <br>
>> <kafhls$qip$1@newscl01ah.mathworks.com>...<br>
>> ><br>
>> > And can anyone please tell me if this code is correct?<br>
>> ><br>
>> > coefInit = [1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;1;9];<br>
>> > options_fminsearch = <br>
>> > optimset('Display','iter','TolFun',1e4,'TolX',1e4,'MaxIt',10000000,'MaxFunEvals',10000000000000);<br>
>> > [coefEst,fval,exitflag,output] = fmincon(@(coef) <br>
>> > llk(coef,q,p,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12),coefInit,A,b,[],[],[],[],[],options_fminsearch);<br>
>> ==============<br>
>><br>
>> Since you only have upper and lower bound constraints, like sigma>=eps, <br>
>> you should be using the lb and ub input arguments, rather than A and b.<br>
><br>
> Hi Matt, I don't really think I can use lower bound because I will only <br>
> constraint one of the parameters that I want to estimate. If I use lower <br>
> bound, won't it be like lb=[0;0;0;0;...0.00001], which means all the <br>
> others are constrainted>=0?<br>
<br>
Yes, if you did that it would constrain all your other parameter values to <br>
be nonnegative. On the other hand, if you did:<br>
<br>
lb = [Inf; Inf; Inf; ... ;1e6];<br>
<br>
it would constrain all but your last parameter to be greater than or equal <br>
to Inf and the last parameter to be greater than or equal to 1e6. The same <br>
idea holds for ub and +Inf.<br>
<br>
 <br>
Steve Lord<br>
slord@mathworks.com<br>
To contact Technical Support use the Contact Us link on <br>
<a href="http://www.mathworks.com">http://www.mathworks.com</a>