| # Linear Functions and Maps Vectors, Matrices > Vectors | Scalar products | Matrices | Linear functions | Applications Linear and affine functions Linear and affine maps First-order approximation of non-linear functions Other sources of linear models ## Linear and affine functions ### Definition Linear functions are functions which preserve scaling and addition of the input argument. Affine functions are ‘‘linear plus constant’’ functions. Formal definition, linear and affine functions. A function is linear if and only if preserves scaling and addition of its arguments: for every , and , ; and for every , . A function is affine if and only if the function with values is affine. An alternative characterization of linear functions is given here. Examples: Consider the functions with values , , . The function is linear; is affine; and is neither. ### Connection with vectors via the scalar product The following shows the connection between linear functions and scalar products. Theorem: Representation of affine function via the scalar product. A function is affine if and only if it can be expressed via a scalar product: for some unique pair , with and . The function is linear if and only if . The theorem shows that a vector can be seen as a (linear) function from the ‘‘input“ space to the ‘‘output” space . Both points of view (matrices as simple collections of numbers, or as linear functions) are useful. ### Gradient of an affine function The gradient of a function at a point , denoted , is the vector of first derivatives with respect to (see here for a formal definition and examples). An affine function , with values has a very simple gradient: the constant vector . That is, for an affine function , we have for every : ### Interpretations The interpretation of are as follows. The is the constant term. For this reason, it is sometimes referred to as the bias, or intercept (as it is the point where intercepts the vertical axis if we were to plot the graph of the function). The terms , give the coefficients of influence of on . For example, if , then the first component of has much greater influence on the value of than the third. Example: Beer-Lambert law in absorption spectrometry. ## Linear and affine maps ### Definition A map is affine if and only if every one of its components is. The formal definition we saw above for functions applies verbatim to maps (see here). To an matrix , we can associate a linear map , with values . Conversely, to any linear map, we can uniquely associate a matrix which satisfies for every . Indeed, if the components of , , , are linear, then they can be expressed as for some . The matrix is the matrix that has as its -th row: Hence, there is a one-to-one correspondence between matrices and linear maps. This is summarized as follows. Theorem: Representation of affine maps via the matrix-vector product. A function is affine if and only if it can be expressed via a matrix-vector product: for some unique pair , with and . The function is linear if and only if . The theorem shows that a matrix can be seen as a (linear) map from the ‘‘input“ space to the ‘‘output” space . Both points of view (matrices as simple collections of vectors, or as linear maps) are useful. ### Interpretations Consider an affine map . An element gives the coefficient of influence of over . In this sense, if we can say that has much more influence on than . Or, says that does not depend at all on . Often the constant term is referred to as the ‘‘bias’’ vector. ## First-order approximation of non-linear functions Many maps are non-linear. A common engineering practice is to approximate a given non-linear map with a linear (or affine) one, by taking derivatives. This is the main reason for linearity to be such an ubiquituous tool in Engineering. ### One-dimensional case Consider a function of one variable , and assume it is differentiable everywhere. Then we can approximate the values function at a point near a point as follows: where denotes the derivative of at . ### Multi-dimensional case With more than one variable, we have a similar result. Let us approximate a differentiable function by a linear function , so that and coincide up and including to the first derivatives. The corresponding approximation is called the first-order approximation to at . The approximate function must be of the form where and . Our condition that coincides with up and including to the first derivatives shows that we must have where the gradient, of at . Solving for we obtain the following result: Theorem: First-order expansion of a function. The first-order approximation of a differentiable function at a point is of the form where is the gradient of at . ### The case of maps The above can be extended to maps. If is differentiable, then we can approximate the values of near a given point by an affine function : where is the derivative of the -th component of with respect to . ( is referred to as the Jacobian of at.) Example: Navigation by range measurement. ## Other sources of linear models ### Change of variables Linearity can arise from a simple change of variables. This is best illustrated with a specific example. Example: Power laws. ### Linear dynamical systems Many linear systems arise from models of dynamical systems, which describe how the state of a system evolves over time. The state is in general a vector fully describing the system at a given time (comprising say the temperature at several locations of a furnace, or the positions and velocities of several locations on a bridge, or various economic indicators, etc). A linear dynamical model postulates that the state at the next instant is a linear function of the state at past instants, and possibly other ‘‘exogeneous’’ inputs. Such systems can result from the linearization of a non-linear dynamical model. Example: Population dynamics.