Steepest Descent Method with Stopping Criteria using MATLAB

I have the following quadratic function.

f(x1 ,x2 ) = 5x1^2 + x2^2 + 4x1x2 −14x1 − 6x2 + 20

I want to minimize it starting points X(0) = [x1^0, x2^0] = [0,10] and the stopping criteria (tolerance) = 10^-6 using steepest descent method. I need help.

Answer 1

[Riccardo Scorretti]

Hi. I don't know it steepest descend is part of any of the Matlab toolbox (anyway, you can find a version here: https://fr.mathworks.com/matlabcentral/fileexchange/22532-steepest-descent-algorithm), but you can use the standard method fminsearch, which in this case works quite well, because the problem is very simple:

% Setup the problem
X0 = [0 10];
fun = @(X) 5*X(:,1).^2 + X(:,2).^2 + 4*X(:,1).*X(:,2) - 14*X(:,1) - 6*X(:,2) + 20;

% Perform the optimization (indeed not by using the steepest descent algorithm)
options = optimset('Display','iter', 'TolX', 1E-6);
[X,FVAL,EXITFLAG] = fminsearch(fun, X0)

X = 1×2

1.0000 1.0000

FVAL = 10.0000

EXITFLAG = 1

Let's check if the result is reasonable:

[x, y] = meshgrid(-2:0.1:3, -2:0.1:3);
f = reshape(fun([x(:) y(:)]), size(x));
figure
mesh(x, y, f) ; hold on ; plot3(X(1), X(2), FVAL, 'r*') ; view(30, 60);

To be 100% sure, let's do some analytical computing:

syms x1 x2
f = 5*x1^2 + x2^2 + 4*x1*x2 - 14*x1 - 6*x2 + 20;
jac = jacobian(f)

jac = 

det([10 4 ; 4 2])

ans = 4.0000

The jacobian matrix is constant, and it is not singular: hence it exists only one solution:

solve(jac)

ans = struct with fields:
    x1: 1
    x2: 1

[Sayed Naseem Sadaat]

Hello Ricardo,

Thank you very much for the useful information and sharing the relevant thread.