| # Positive Semi-Definite Matrices Symmetric Matrices > Definitions | Spectral theorem | PSD matrices | PCA | Applications Definitions Special cases and examples Square root and Cholesky decomposition Ellipsoids ## Definitions For a given symmetric matrix , the associated quadratic form is the function with values A symmetric matrix is said to be positive semi-definite (PSD, notation: ) if and only if the associated quadratic form is non-negative everywhere: It is said to be positive definite (PD, notation: ) if the quadratic form is non-negative, and definite, that is, if and only if . It turns out that a matrix is PSD if and only if the eigenvalues of are non-negative. Thus, we can check if a form is PSD by computing the eigenvalue decomposition of the underlying symmetric matrix. Theorem: eigenvalues of PSD matrices A quadratic form , with is non-negative (resp. positive-definite) if and only if every eigenvalue of the symmetric matrix is non-negative (resp. positive). Proof. By definition, the PSD and PD properties are properties of the eigenvalues of the matrix only, not of the eigenvectors. Also, if the matrix is PSD, then for every matrix with columns, the matrix also is. ## Special cases and examples ### Symmetric dyads Special cases of PSD matrices include symmetric dyads. Indeed, if for some vector , then for every : More generally if , then is PSD, since ### Diagonal matrices A diagonal matrix is PSD (resp. PD) if and only if all of its (diagonal) elements are non-negative (resp. positive). ### Examples of PSD matrices Covariance matrix. Laplacian matrix of a graph. Gram matrix of data points. ## Square root and Cholesky decomposition For PD matrices, we can generalize the notion of ordinary square root of a non-negative number. Indeed, if is PSD, there exist a unique PSD matrix, denoted , such that . We can express this matrix square root in terms of the SED of , as, where is obtained from by taking the square root of its diagonal elements. If is PD, then so is its square root. Any PSD matrix can be written as a product for an appropriate matrix . The decomposition is not unique, and is only a possible choice (the only PSD one). Another choice, in terms of the SED of , is . If is positive-definite, then we can choose to be lower triangular, and invertible. The decomposition is then known as the Cholesky decomposition of . ## Ellipsoids There is a strong correspondence between ellipsoids and PSD matrices. ### Definition We define an ellipsoid to be affine transformation of the unit ball for the Euclidean norm: where is an arbitrary non-singular matrix. We can express the ellipsoid as where is PD. ### Geometric interpretation via SED We can interpret the eigenvectors and associated eigenvalues of in terms of geometrical properties of the ellipsoid, as follows. Consider the SED of : , with and diagonal, with diagonal elements positive. The SED of its inverse is . Let . We can express the condition as Now set , . The above writes : in -space, the ellipsoid is simply an unit ball. In -space, the ellipsoid corresponds to scaling each -axis by the square roots of the eigenvalues. The ellipsoid has principal axes parallel to the coordinate axes in -space. We then apply a rotation and a translation, to get the ellipsoid in the original -space. The rotation is determined by the eigenvectors of , which are contained in the orthogonal matrix . Thus, the geometry of the ellipsoid can be read from the SED of the PD matrix : the eigenvectors give the principal directions, and the semi-axis lengths are the square root of the eigenvalues.
| The graph on the left shows the ellipsoid , with The matrix admits the SED We check that the columns of determine the principal directions, and , are the semi-axis lengths.
The above shows in particular that an equivalent representation of an ellipsoid is where is PD. It is possible to define degenerate ellipsoids, which correspond to cases when the matrix in the above, or its inverse , is degenerate. For example, cylinders or slabs (intersection of two parallel half-spaces) are degenerate ellipsoids.