Minimize a function using gradient descent

How can we minimise the following function using gradient descent (using a for loop for iterations and a surface plot to display a graph that shows the minimisation)

% initial values: x = y = 2
z = 2*(x^2) + 3*(y^2);

Answer 1

[Torsten]

X = -2:0.1:2;
Y = -2:0.1:2;
[X,Y] = meshgrid(X,Y);
Z = 2*X.^2+3*Y.^2;
surf(X,Y,Z)
hold on
x(1) = 2; % initial value of x 
y(1) = 2; % initial value of y 
z(1) =  2.*x(1).^2 + 3.*y(1).^2;
stepsize = 0.1;
for i = 1:30
    zx = 4*x(i); 
    zy = 6*y(i); 
    x(i+1) = x(i) - stepsize*zx; %gradient descent 
    y(i+1) = y(i) - stepsize*zy;
    z(i+1) = 2.*x(i+1).^2 + 3.*y(i+1).^2
end
plot3(x,y,z,'Markersize',10,'Color','red')
hold off

Answer 2

[Sam Chak]

@Shikhar Singh

Let us visualize and formulate the minimization problem first. So you want to start descending from the point , circled in the image. The contour plot can give you an estimation where you are heading to from the starting point.

f = @(x,y) 2*(x.^2) + 3*(y.^2);

[x,y] = meshgrid(-2.5:0.25:2.5, -2.5:0.25:2.5);
z = f(x, y);

[fx, fy] = gradient(z, 0.25);

cs = contour(x, y, z);
axis square
clabel(cs);
hold on

plot(2, 2, 'ro', 'linewidth', 1.5)

quiver(x, y, -fx, -fy);
hold off

xlabel('x')
ylabel('y')

We try to first obtain the solution with the fminsearch() function. Then, we can write the gradient descent algorithm to compare with the result.

fun = @(x) 2*(x(1).^2) + 3*(x(2).^2);
[x, fval] = fminsearch(fun, [2, 2])

x =

1.0e-04 *

0.0707 -0.3490

fval =

3.7533e-09

Surface plot with the mesh() function:

[x, y] = meshgrid(-3:0.375:3);
z = 2*(x.^2) + 3*(y.^2);
[u, v] = gradient(z, 0.375);
w = 1;
magnitude = sqrt(u.*u + v.*v + w.*w);
u = u./magnitude;
v = v./magnitude;
w = w./magnitude;
mesh(x, y, z)
axis square
xlabel('x');
ylabel('y');
zlabel('z');
hold on
quiver3(x, y, z, -0.75*u, -0.75*v, w, 0)
hold off

[Tatiana Danilova]

Minimize a cost function using gradient descent