| # Sample covariance matrix The average of given real numbers is the quantity . Their /variance} is the non-negative number , and is a measure of the variability of the numbers around their average; in particular, the variance is zero if and only if all the numbers are equal. Now consider two series of real numbers, , , . We can define the means and variances of these numbers as before. In addition, we can define the /covariance} between these two collections of numbers as the value The covariance is a measure of closeness between the two series of numbers: if the covariance is large and positive, then in average, when ’s are far above or below their mean, then so is ; the series tend to move in the same direction simultaneously. If the covariance is large and negative, then the two series tend to move in opposite directions. If the covariance is zero, then the excursions of the -series above or below its mean tend to compensate those of the -series. The symmetric matrix is called the covariance matrix associated with the two series . Consider now a collection of vectors , . We can define, as before, the average as the vector . We then define the covariance matrix as the matrix formed with the covariance between the series , , , for . More compactly: A diagonal element of , , captures the individual variance of the -th component of the vectors, which is the series , . The covariance matrix is symmetric by construction. In addition, it is positive semi-definite, since it is a non-negative sum of symmetric dyads.