| # Scalar Product, Norms and Angles Vectors > Basics | Scalar product, Norms | Projection on a line | Orthogonalization | Hyperplanes | Linear functions | Application Scalar product Norms Three popular norms Cauchy-schwartz inequality Angles between vectors ## Scalar product ### Definition The scalar product (or, inner product, or dot product) between two vectors is the scalar denoted , and defined as The motivation for our notation above will come later, when we define the matrix-matrix product. The scalar product is also sometimes denoted , a notation which originates in physics. In matlab, we use a notation consistent with a later definition of matrix-matrix product. Matlab syntax
    >> x = [1; 2; 3]; y = [4; 5; 6];
    >> scal_prod = x'*y;
Examples: Rate of return of a financial portfolio. Sample and weighted average. Beer-Lambert law in absorption spectroscopy. ### Orthogonality We say that two vectors are orthogonal if . Example: Two orthogonal vectors in . ## Norms ### Definition Measuring the size of a scalar value is unambiguous — we just take the magnitude (absolute value) of the number. However, when we deal with higher dimensions, and try to define the notion of size, or length, of a vector, we are faced with many possible choices. These choices are encapsulated in the notion of norm. Norms are real-valued functions that satisfy a basic set of rules that a sensible notion of size should involve. You can consult the formal definition of a norm here. The norm of a vector is usually denoted . ### Three popular norms In this course, we focus on the following three popular norms for a vector :
| The Euclidean norm: corresponds to the usual notion of distance in two or three dimensions. The set of points with equal -norm is a circle (in 2D), a sphere (in 3D), or a hyper-sphere in higher dimensions.
| The -norm: corresponds to the distance travelled on a rectangular grid to go from one point to another.
| The -norm: is useful in measuring peak values.
Matlab syntax
    >> x = [1; 2; -3];
    >> r2 = norm(x,2); % l2-norm
    >> r1 = norm(x,1); % l1 norm
    >> rinf = norm(x,inf); % l-infty norm
Examples: A given vector will in general have different ‘‘lengths" under different norms. For example, the vector yields , , and . Sample standard deviation. ## Cauchy-Schwartz inequality The Cauchy-Schwartz inequality allows to bound the scalar product of two vectors in terms of their Euclidean norm. Theorem: Cauchy-Schwartz inequality For any two vectors , we have The above inequality is an equality if and only if are collinear. In other words: with optimal given by if is non-zero. For a proof, see here. The Cauchy-Schwartz inequality can be generalized to other norms, using the concept of dual norm. ## Angles between vectors When none of the vectors is zero, we can define the corresponding angle as such that Applying the Cauchy-Schwartz inequality above to and we see that indeed the number above is in . The notion above generalizes the usual notion of angle between two directions in two dimensions, and is useful in measuring the similarity (or, closeness) between two vectors. When the two vectors are orthogonal, that is, , we do obtain that their angle is . Example: Similarity of two documents.