# Differential Equations and Linear Algebra

Massachusetts Institute of Technology professor, Gilbert Strang, explains differential equations and linear algebra which are two crucial subjects in science and engineering. This video series develops those subjects both separately and together and supplements Gil Strang's textbook on this subject.

### Introduction

Overview of Differential Equations
Linear equations include *dy/dt* = *y, dy/dt* = –*y, dy/dt* = *2ty*. The equation *dy/dt* = *y***y* is nonlinear.

The Calculus You Need
The sum rule, product rule, and chain rule produce new derivatives from the derivatives of *x ^{n}*, sin(

*x*) and

*e*. The Fundamental Theorem of Calculus says that the integral inverts the derivative.

^{x}### First Order Equations

1.4b: Response to Exponential Input, exp(s*t)
With exponential input, *e ^{st}*, from outside and exponential growth,

*e*, from inside, the solution, y(t), is a combination of two exponentials.

^{at}
1.4c: Response to Oscillating Input, cos(w*t)
An oscillating input cos(ω*t*) produces an oscillating output with the same frequency ω (and a phase shift).

1.4d: Solution for Any Input, q(t)
To solve a linear first order equation, multiply each input *q(s)* by its growth factor and integrate those outputs*.*

1.4e: Step Function and Delta Function A unit step function jumps from 0 to 1. Its slope is a delta function: zero everywhere except infinite at the jump.

1.5: Response to Complex Exponential, exp(i*w*t) = cos(w*t)+i*sin(w*t)
For linear equations, the solution for *f =* cos(ω*t*) is the real part of the solution for *f = e ^{iωt}*. That complex solution has magnitude

*G*(the gain).

1.6: Integrating Factor for a Constant Rate, a
The integrating factor *e ^{-at}* multiplies the differential equation, y’=ay+q, to give the derivative of

*e*y: ready for integration.

^{-at}1.6b: Integrating Factor for a Varying Rate, a(t) The integral of a varying interest rate provides the exponent in the growing solution (the bank balance).

1.7: The Logistic Equation
When *–by ^{2}* slows down growth and makes the equation nonlinear, the solution approaches a steady state

*y(*∞

*) = a/b.*

1.7c: The Stability and Instability of Steady States Steady state solutions can be stable or unstable – a simple test decides.

1.8: Separable Equations
Separable equations can be solved by two separate integrations, one in *t* and the other in *y*. The simplest is *dy/dt = y*, when *dy/y* equals *dt*. Then ln(*y*) = *t + C*.

### Second Order Equations

2.1: Second Order Equations For the oscillation equation with no damping and no forcing, all solutions share the same natural frequency.

2.1b: Forced Harmonic Motion
With forcing *f* = cos(ω*t*), the particular solution is *Y**cos(ω*t*). But if the forcing frequency equals the natural frequency there is resonance.

2.3: Unforced Damped Motion
With constant coefficients in a differential equation, the basic solutions are exponentials *e ^{st}*. The exponent

*s*solves a simple equation such as

*As*.

^{2}+ Bs + C = 0
2.3c: Impulse Response and Step Response
The impulse response *g* is the solution when the force is an impulse (a delta function). This also solves a null equation (no force) with a nonzero initial condition.

2.4: Exponential Response - Possible Resonance Resonance occurs when the natural frequency matches the forcing frequency — equal exponents from inside and outside.

2.4b: Second Order Equations With Damping
A damped forced equation has a particular solution *y = G* cos(ω*t –* α). The damping ratio provides insight into the null solutions.

2.5: Electrical Networks: Voltages and Currents
Current flowing around an RLC loop solves a linear equation with coefficients *L* (inductance), *R* (resistance), and *1/C* (*C* = capacitance).

2.6: Methods of Undetermined Coefficients
With constant coefficients and special forcing terms (powers of *t*, cosines/sines, exponentials), a particular solution has this same form.

2.6b: An Example of Method of Undetermined Coefficients
This method is also successful for forces and solutions such as *(at ^{2} + bt +c) e^{st}*: substitute into the equation to find

*a, b, c*.

2.6c: Variations of Parameters
Combine null solutions *y _{1}* and

*y*with coefficients

_{2}*c*and

_{1}(t)*c*to find a particular solution for any

_{2}(t)*f(t).*

2.7: Laplace Transform: First Order Equation
Transform each term in the linear differential equation to create an algebra problem. You can then transform the algebra solution back to the ODE solution, *y(t)*.

2.7b: Laplace Transform: Second Order Equation
The second derivative transforms to *s ^{2}Y* and the algebra problem involves the transfer function

*1/ (As*

^{2}+ Bs +C).
2.7c: Laplace Transforms and Convolution
When the force is an impulse δ *(t)*, the impulse response is *g(t)*. When the force is *f(t)*, the response is the “convolution” of *f* and *g.*

### Graphical and Numerical Methods

3.1: Pictures of the Solutions
The direction field for *dy/dt = f(t,y)* has an arrow with slope *f* at each point *t, y*. Arrows with the same slope lie along an isocline.

3.2: Phase Plane Pictures: Source, Sink Saddle Solutions to second order equations can approach infinity or zero. Saddle points contain a positive and also a negative exponent or eigenvalue.

3.2b: Phase Plane Pictures: Spirals and Centers
Imaginary exponents with pure oscillation provide a “center” in the phase plane. The point *(y, dy/dt)* travels forever around an ellipse.

3.2c: Two First Order Equations: Stability
A second order equation gives two first order equations for *y* and *dy/dt*. The matrix becomes a companion matrix.

3.3: Linearization at Critical Points
A critical point is a constant solution *Y* to the differential equation *y’ = f(y)*. Near that *Y*, the sign of *df/dy* decides stability or instability.

3.3b: Linearization of y'=f(y,z) and z'=g(y,z)
With two equations, a critical point has *f(Y,Z)* = 0 and *g(Y,Z)* = 0. Near those constant solutions, the two linearized equations use the 2 by 2 matrix of partial derivatives of *f* and *g*.

3.3c: Eigenvalues and Stability: 2 by 2 Matrix, A
Two equations *y’ = Ay* are stable (solutions approach zero) when the trace of *A* is negative and the determinant is positive.

3.3d: The Tumbling Box in 3-D A box in the air can rotate around its shortest and longest axes. Around the middle axis it tumbles wildly.

### Vector Spaces and Subspaces

5.1: The Column Space of a Matrix, A
An *m* by *n* matrix *A* has *n* columns each in **R**^{m}. Capturing all combinations Av of these columns gives the column space – a subspace of **R*** ^{m}*.

5.4: Independence, Basis, and Dimension Vectors v 1 to v d are a basis for a subspace if their combinations span the whole subspace and are independent: no basis vector is a combination of the others. Dimension d = number of basis vectors.

5.5: The Big Picture of Linear Algebra A matrix produces four subspaces – column space, row space (same dimension), the space of vectors perpendicular to all rows (the nullspace), and the space of vectors perpendicular to all columns.

5.6: Graphs
A graph has *n* nodes connected by *m* edges (other edges can be missing). This is a useful model for the Internet, the brain, pipeline systems, and much more.

5.6b: Incidence Matrices of Graphs
The incidence matrix *A* has a row for every edge, containing -1 and +1 to show the two nodes (two columns of *A*) that are connected by that edge.

### Eigenvalues and Eigenvectors

6.1: Eigenvalues and Eigenvectors
The eigenvectors **x** remain in the same direction when multiplied by the matrix (*A***x** *=* λ**x**). An *n* x *n* matrix has *n* eigenvalues.

6.2: Diagonalizing a Matrix
A matrix can be diagonalized if it has *n* independent eigenvectors. The diagonal matrix Λis the eigenvalue matrix.

6.2b: Powers, A^n, and Markov Matrices
Diagonalizing *A* = *V*Λ*V ^{–1}* also diagonalizes

*A*=

^{n}*V*Λ

*.*

^{n}V^{–1}
6.3: Solving Linear Systems
*d***y***/dt = A***y** contains solutions **y** *= e ^{λt}*

**x**where

*λ*and

**x**are an eigenvalue / eigenvector pair for

*A*.

6.4: The Matrix Exponential, exp(A*t)
The shortest form of the solution uses the matrix exponential **y** *= e ^{At}*

**y**

*(0)*. The matrix

*e*has eigenvalues

^{At}*e*and the eigenvectors of

^{λt}*A.*

6.4b: Similar Matrices, A and B=M^(-1)*A*M
*A* and *B* are “similar” if *B* = *M ^{-1}AM* for some matrix

*M*.

*B*then has the same eigenvalues as

*A*.

6.5: Symmetric Matrices, Real Eigenvalues, Orthogonal Eigenvectors
Symmetric matrices have *n* perpendicular eigenvectors and *n* real eigenvalues.

6.5b: Second Order Systems, y''+Sy=0
An oscillation equation *d ^{2}y/dt^{2} + Sy =* 0 has

*2n*solutions (sines and cosines). Solutions use the eigenvectors of

*S.*

### Applied Mathematics and ATA

7.2: Positive Definite Matrices, S=A'*A
A positive definite matrix S has positive eigenvalues, positive pivots, positive determinants, and positive energy v^{T}Sv for every vector v. S = A^{T}A is always positive definite if A has independent columns.

7.2b: Singular Value Decomposition, SVD
The SVD factors each matrix *A* into an orthogonal matrix *U* times a diagonal matrix Σ (the singular value) times another orthogonal matrix V^{T}: rotation times stretch times rotation.

7.3: Boundary Conditions Replace Initial Conditions
A second order equation can change its initial conditions on *y(0)* and *dy/dt(0)* to boundary conditions on *y(0)* and *y(1)*.

7.4: Laplace Equation
The partial differential equation ∂* ^{2}u/*∂

*x*∂

^{2}+*∂*

^{2}u/*y*describes temperature distribution inside a circle or a square or any plane region.

^{2}= 0### Fourier and Laplace Transforms

8.1: Fourier Series
A Fourier series separates a periodic function *F(x)* into a combination (infinite) of all basis functions cos(*nx)* and sin(*nx)*.

8.1b: Examples of Fourier Series
Even functions use only cosines (*F(–x) = F(x)*) and odd functions use only sines. The coefficients *a _{n}* and

*b*come from integrals of

_{n}*F(x)*cos(

*nx*) and

*F(x)*sin(

*nx*).

8.1c: Fourier Series Solution of Laplace's Equation
Inside a circle, the solution *u*(*r*, θ) combines *r ^{n}* cos(

*n*θ) and

*r*sin(

^{n}*n*θ). The boundary solution combines all entries in a Fourier series to match the boundary conditions.

8.3: Heat Equation
The heat equation ∂*u*/∂*t* = ∂* ^{2}u*/∂

*x*starts from a temperature distribution

^{2}*u*at

*t*= 0 and follows it for

*t*> 0 as it quickly becomes smooth.

8.4: Wave Equation
The wave equation ∂* ^{2}u*/∂

*t*= ∂

^{2}*/∂*

^{2}u*x*shows how waves move along the

^{2}*x*axis, starting from a wave shape

*u*(0) and its velocity ∂

*u*/∂

*t*(0).