Hi Roger,
Thanks for your analysis.
I tried a redefined x*log(x) function and it worked. This really helps. My only concern is that whether this will create any instability for the algorithm. But I think it is very unlikely coz numerically x*log(x) approaches 0 as x approaches 0. So it is consistent with the original function except at 0.
thanks again,
Ming
"Roger Stafford" <ellieandrogerxyzzy@mindspring.com.invalid> wrote in message <gimskg$egm$1@fred.mathworks.com>...
> "Ming " <mingsu@u.washington.edu> wrote in message <gimpsb$ogc$1@fred.mathworks.com>...
> > ......
> > I have the following minimization problem,
> >
> > min x(1)*log(x(1)/0.7) + x(2)*log(x(2)/0.3)
> > s.t. x(1) + x(2) = 1;
> > x(1) >= 0;
> > x(2) >= 0;
> > Here is the Matlab code:
> >
> > *****************************************************
> > x0 = [0.5 0.5];
> > lb = [0 0];
> > options = optimset('LargeScale','on', 'GradObj', 'on', 'Hessian', 'off', 'MaxIter', 20000, 'MaxFunEvals', 20000000, 'TolX', 1e6, 'TolFun',1e6,'Display','iter');
> >
> > [x, fval] = fmincon(@obj,x0,[],[],[],[],lb,[],@constr,options);
> > *****************************************************
> >
> > After first iteration, the line search algorithm picks x(1) = 1 and x(2) = 0 making the objective function, derivative and first order optimality measures all NaN. This is because log(0) = inf and 0*inf = NaN while mathematically x*log(x) = 0 when x takes on zero.
> >
> > Mathematically, the optimization problem has bounded solutions. Interesting thing is when I take off the lower bound, the algorithm can quickly find the global minimum with x(1) = 0.7, x(2) = 0.3. However I do need the lower bound constraints because x(1) and x(2) represents probabilities.
> >
> > Please let me know if you ran into similar situation before or you know how to work around this kind of numerical problem. Thank you very much!
> >
> > Ming
>
> You may be using 'fmincon' in order to become familiar with it. However, for the problem you show, there is a much easier approach. Call x(1) = x and x(2) = 1x. Then you want to find the minimum value of
>
> f(x) = x*log(x/.7) + (1x)*log((1x)/.3)
>
> subject to 0 <= x <= 1. The minimum must occur either at a point where the derivative is zero or at one of the two bounding values. The derivative is
>
> f'(x) = log(x/.7) + 1  log((1x)/.3)  1 = log(x/(1x)*.3/.7)
>
> which is zero when
>
> x/(1x)*.3/.7 = 1
>
> or
>
> .3*x = .7*(1x)
> x = .7
>
> Hence the minimum occurs at either x = 0, x = .7, or x = 1. It is easily checked (via l'Hopital's rule) that there is a minimum of zero at x =.7.
>
> If you have a problem that actually needs to deal with x*log(x) at x = 0, you would need to define a special function which substitutes zero at x = 0 in order for it to be used as a successful function handle.
>
> Roger Stafford
>
