Least squares solution of linear equations
This functionality does not run in MATLAB.
numeric::leastSquares(A
, B
, <mode
>, <method
>, options
)
numeric::leastSquares(A, B)
computes a matrix X that
solves the linear matrix equation A X = B in
the least squares sense: the columns X_{j} of X
minimize
where
the B_{j} are
the columns of B
.
For a given vector B, a vector X minimizes if and only if X is a solution of the "normal equations" A^{H} A X = A^{H} B, where A^{H} is the Hermitian transpose of the m×n matrix A. The solution is unique if rank(A) = n.
numeric::leastSquares
allows to solve several
least squares problems simultaneously by combining several ‘right
hand sides' B_{j} columnwise
to a matrix B
.
If no return type is specified via the option ReturnType
= d
, the domain type of the return data depends on the type
of the input matrix A
:
The special solution X as well as the kernel of an array A are returned as arrays.
The special solution and the kernel of an hfarray of domain type DOM_HFARRAY
are
returned as hfarrays.
For a dense matrix A of
type Dom::DenseMatrix()
, both the special solution X as
well as the kernel are returned as matrices of type Dom::DenseMatrix()
over
the ring of MuPAD^{®} expressions.
For all other matrices of category Cat::Matrix
, both the special
solution X as
well as the kernel are returned as matrices of
type Dom::Matrix()
over the ring of MuPAD expressions.
This includes input matrices A
of type Dom::Matrix(...)
, Dom::SquareMatrix(...)
, Dom::MatrixGroup(...)
etc.
Without Symbolic
, the input data are converted
to floatingpoint numbers. The matrix A must
not contain nonconvertible parameters, unless Symbolic
is
used. If such objects are found, numeric::leastSquares
automatically
switches to its symbolic mode, issuing a warning. This warning may
be suppressed via NoWarning
.
Symbolic parameters in B are
accepted without warning. However, HardwareFloats
cannot
be used if there are any symbolic parameters in A or B.
If A^{H} A has
a nontrivial kernel, the least squares solution X is
not unique. The return value X
is a special solution
of the equation A^{H} A X = A^{H} B.
With the SVD
method, X
is the
special solution with columns of minimal Euclidean length.
Note:
The result computed with 
The kernel is computed only in the symbolic mode (option Symbolic
).
All floatingpoint methods return the value NIL
for
the kernel.
With Symbolic
, the n×d matrix KernelBasis
is
the most general solution of A^{H} A X =
0. Its columns span the ddimensional
kernel of A^{H} A.
If the kernel is 0dimensional,
the return value of KernelBasis
is the integer 0.
If KernelBasis
is returned as an array, the dimension d of
the kernel is d
= op(KernelBasis, [0,
3, 2]])
. If KernelBasis
is returned as
a matrix of type Dom::Matrix()
or Dom::DenseMatrix()
,
the dimension d of
the kernel is d
= KernelBasis::dom::matdim(KernelBasis)[2]
.
Note:
Without the option With 
All entries of A
and B
must
be arithmetical
expressions.
Note:
Apart from matrices of type The same holds true for matrices 
The function is sensitive to the environment variable DIGITS
,
which determines the numerical working precision.
We consider a matrix A of rank 1:
A := array(1..3, 1..2, [[1, 2], [1, 2], [1, 2]]): B := [3, 4, 5]:
The normal equations have a 1parameter set of of solutions:
[X, KernelBasis, Res] := numeric::leastSquares(A, B, Symbolic)
The numerical method QRD
produces a special
solution:
[X, KernelBasis, Res] := numeric::leastSquares(A, B, QRD)
The numerical method SVD
produces a solution X of
minimal norm:
[X, KernelBasis, Res] := numeric::leastSquares(A, B, SVD)
delete A, B, X, KernelBasis, Res:
We consider an illconditioned least squares problem. By construction, the following overdetermined system has an exact solution X = [1, 2, …, n]:
m := 10: n := 8: A := array(1..m, 1..n, [[1/(i + j + 100) $ j=1..n] $ i=1..m]): B := array(1..m, [_plus(A[i,j]*j $ j=1..n) $ i=1..m]): numeric::leastSquares(A, B, Symbolic)
The coefficient matrix A is rather illconditioned:
singvals := numeric::singularvalues(A): conditionOfA := max(op(singvals))/min(op(singvals))
Consequently, roundoff has a drastic effect in a numerical approximation. The methods yield results of different quality:
numeric::leastSquares(A, B, QRD)
numeric::leastSquares(A, B, SVD)
delete m, n, A, B, singvals, conditionOfA:
This example involves a symbolic parameter c
in
the matrix A.
The option Symbolic
must be used:
A:= matrix([[c, 2], [1/3, 2/3], [1/7, 2/7]]): B:= [1, 2, 3]: numeric::leastSquares(A, B, Symbolic)
normal(%)
delete A, B:
Floating point entries may cause problems in conjunction with
the option Symbolic
, because the computation is
not stabilized numerically in the symbolic node. The following matrix A has
rank 2:
A := matrix([[1, 30], [10.0^(15), 31*10.0^(15)]]):
However, due to roundoff, the ‘normal matrix' A^{H} A has
rank 1. No solution is found
with Symbolic
:
A::dom::transpose(A) * A
numeric::leastSquares(A, [31, 32*10^5], Symbolic)
No such problem arises in the numerical schemes. Note, however, that the numerical methods yield different results in this extremely illconditioned problem:
numeric::leastSquares(A, [31, 32*10^5], QRD)
numeric::leastSquares(A, [31, 32*10^5], SVD)
delete A:

An m×n matrix
of domain type 

An m×p matrix
of domain type 

One of the flags 

One of the flags 

With With Compared to the If no If the result cannot be computed with hardware floats, software arithmetic by the MuPAD kernel is tried. If the current value of There may be several reasons for hardware arithmetic to fail:
If neither If Note that The trailing digits in floatingpoint results computed with  

Prevents the conversion of the input data to floatingpoint
numbers. Exact arithmetic is used. This option overrides This option must be used, if the matrix A contains symbolic parameters that cannot be converted to floatingpoint numbers. The normal equations A^{H} A X = A^{H} B are
passed to If the least squares problem does not have a unique solution, a special solution X is returned together with the kernel of A^{H} A. Cf. Example 1.
 

Use a QR decomposition. All entries of A must be convertible to floatingpoint values. This is the default method. The matrix A must
not contain symbolic parameters that cannot be converted to floating
point numbers. If such objects are found, then The warning may be suppressed by the option Symbolic parameters in B are accepted without warning. They are processed by the floatingpoint algorithm. Numerical expressions such as etc. are accepted and converted to floats. If the least squares problem does not have a unique solution,
only a special solution is returned. The kernel is not computed: it
is returned as The method The conditioning is given by the ratio of the largest singular value of A divided by the smallest singular value of A. If this value is large, the problem is illconditioned. Cf. Example 2.  

Use a singular value decomposition. All entries of A must be convertible to floatingpoint values. The matrix A must
not contain symbolic parameters that cannot be converted to floating
point numbers. If such objects are found, then The warning may be suppressed by the option Symbolic parameters in B are accepted without warning. They are processed by the floatingpoint algorithm. Numerical expressions such as etc. are accepted and converted to floats. If the least squares problem does not have a unique solution, the columns X_{j} of the solution X have a minimal Euclidean length . The kernel is not computed: it is returned as A singular value decomposition A = U D V^{H} is used to solve the normal equations in the form D^{2} V^{H} X = D U^{H} B. For small or zero singular values d_{j} in D = diag(d_{1}, d_{2}, …), the corresponding components of V^{H} x are set to zero. Usually, the numerical method The conditioning is given by the ratio of the largest singular value of A divided by the smallest singular value of A. If this value is large, the problem is illconditioned.  

Suppresses warnings If symbolic coefficients are found in A,  

Option, specified as Return the (special) solution and the kernel as matrices of domain type 
A list [X, KernelBasis, Residues]
is returned.
The (special) least squares solution X
is
an n×p matrix.
With Symbolic
, KernelBasis
is
an n×d matrix
(d is
the dimension of the kernel of A^{H} A).
Its columns span the kernel of A^{H} A.
If the kernel is trivial, KernelBasis
is the integer 0.
Without Symbolic
, the kernel is not computed.
The value NIL
is returned for the KernelBasis
.
The list of arithmetical
expressions Residues
consists of the minimized
least squares deviations
corresponding
to the columns of X
and B
.