Description of sllinreg

Home > sltoolbox > regression > sllinreg.m

sllinreg

PURPOSE

SLLINREG Performs Multivariate Linear Regression and Ridge Regression

SYNOPSIS

function A = sllinreg(X, Y, varargin)

DESCRIPTION

SLLINREG Performs Multivariate Linear Regression and Ridge Regression

 $ Syntax $
   - A = sllinreg(X, Y, ...)

 $ Arguments $
   - X:        The matrix of x samples
   - Y:        The matrix of y samples
   - A:        The solved transform matrix

 $ Description $
   - A = sllinreg(X, Y, varargin) solves the linear regression problem to
     get the linear transforms A: (y = Ax + e). The solution is given by 
     the following optimization problem:
       A = argmin_{A} sum_i ||y_i - A x_i||^2 + sum_k lambda_k ||a_k||^2
     Here, Y is a dy x n matrix with the i-th column giving y_i; 
           X is a dx x n matrix with the i-th column giving x_i
           A is a dy x dx transform matrix
           a_k is the k-row vector of A, which corresponds to the k-th 
                  component of y
           lambda_k is the regularization weight in ridge regression
     According to mathematical analysis, the solution is given as
       A = (Y * X^T) * (X * X^T + diag(lambda))^{-1}
     You can specify the following properties to control the regression:
     \*
     \t    Table  Linear Regression Properties
     \h        name       &             description
              'lambdas'   &  The regularization coefficients in ridge 
                             linear regression. If all components share
                             the same value, then lambdas can be a scalar.
                             If different values are for different 
                             dimensions, then lambdas can be a dy x 1 
                             column vector. (default = 0)
              'weights'   &  The sample weights, can be either [] or
                             an 1 x n row vector.
              'invparams' &  The parameters for invoking slinvcov to 
                             compute inverse matrix, in the form of
                             {method, ...}. default = [], means directly
                             using inv to do inverse.
     \*

 $ Remarks $
   - To solve the linear problem like y = Ax + b, you have two ways:
     (1) centralize x and y respectively, and invoke sllinreg, then set
         b = my - A * mx
     (2) invoke sllinrega, which is specially designed for such an
         augmented problem.

 $ History $
   - Created by Dahua Lin, on Sep 15, 2006

CROSS-REFERENCE INFORMATION

This function calls:

• slmulvec SLMULVEC multiplies a vector to columns or rows of a matrix
• slinvcov SLINVCOV Compute the inverse of an covariance matrix
• raise_lackinput RAISE_LACKINPUT Raises an error indicating lack of input argument
• slparseprops SLPARSEPROPS Parses input parameters

This function is called by:

• sllinrega SLLINREGA Performs Augmented Multivariate Linear Regression

SOURCE CODE

0001 function A = sllinreg(X, Y, varargin)
0002 %SLLINREG Performs Multivariate Linear Regression and Ridge Regression
0003 %
0004 % $ Syntax $
0005 %   - A = sllinreg(X, Y, ...)
0006 %
0007 % $ Arguments $
0008 %   - X:        The matrix of x samples
0009 %   - Y:        The matrix of y samples
0010 %   - A:        The solved transform matrix
0011 %
0012 % $ Description $
0013 %   - A = sllinreg(X, Y, varargin) solves the linear regression problem to
0014 %     get the linear transforms A: (y = Ax + e). The solution is given by
0015 %     the following optimization problem:
0016 %       A = argmin_{A} sum_i ||y_i - A x_i||^2 + sum_k lambda_k ||a_k||^2
0017 %     Here, Y is a dy x n matrix with the i-th column giving y_i;
0018 %           X is a dx x n matrix with the i-th column giving x_i
0019 %           A is a dy x dx transform matrix
0020 %           a_k is the k-row vector of A, which corresponds to the k-th
0021 %                  component of y
0022 %           lambda_k is the regularization weight in ridge regression
0023 %     According to mathematical analysis, the solution is given as
0024 %       A = (Y * X^T) * (X * X^T + diag(lambda))^{-1}
0025 %     You can specify the following properties to control the regression:
0026 %     \*
0027 %     \t    Table  Linear Regression Properties
0028 %     \h        name       &             description
0029 %              'lambdas'   &  The regularization coefficients in ridge
0030 %                             linear regression. If all components share
0031 %                             the same value, then lambdas can be a scalar.
0032 %                             If different values are for different
0033 %                             dimensions, then lambdas can be a dy x 1
0034 %                             column vector. (default = 0)
0035 %              'weights'   &  The sample weights, can be either [] or
0036 %                             an 1 x n row vector.
0037 %              'invparams' &  The parameters for invoking slinvcov to
0038 %                             compute inverse matrix, in the form of
0039 %                             {method, ...}. default = [], means directly
0040 %                             using inv to do inverse.
0041 %     \*
0042 %
0043 % $ Remarks $
0044 %   - To solve the linear problem like y = Ax + b, you have two ways:
0045 %     (1) centralize x and y respectively, and invoke sllinreg, then set
0046 %         b = my - A * mx
0047 %     (2) invoke sllinrega, which is specially designed for such an
0048 %         augmented problem.
0049 %
0050 % $ History $
0051 %   - Created by Dahua Lin, on Sep 15, 2006
0052 %
0053 
0054 %% parse and verify input arguments
0055 
0056 if nargin < 2
0057     raise_lackinput('sllinreg', 2);
0058 end
0059 
0060 if ~isnumeric(X) || ~isnumeric(Y) || ndims(X) ~= 2 || ndims(Y) ~= 2
0061     error('sltoolbox:invalidarg', ...
0062         'The X and Y should be both 2D numeric matrices');
0063 end
0064 
0065 [dx, n] = size(X);
0066 [dy, ny] = size(Y);
0067 if n ~= ny
0068     error('sltoolbox:sizmismatch', ...
0069         'X and Y contain different numbers of samples');
0070 end
0071 
0072 opts.lambdas = 0;
0073 opts.rv = 1e-3;
0074 opts.weights = [];
0075 opts.invparams = [];
0076 opts = slparseprops(opts, varargin{:});
0077 
0078 lambdas = opts.lambdas;
0079 if ~isscalar(lambdas) && ~isequal(size(lambdas), [dy, 1])
0080     error('lambdas should be either a scalar or a dy x 1 column vector');
0081 end
0082 
0083 w = opts.weights;
0084 if ~isempty(w)
0085     if ~isequal(size(w), [1, n])
0086         error('The sample weights should be a 1 x n row vector');
0087     end
0088 end
0089 
0090 
0091 %% main
0092 
0093 if isempty(w)
0094     Xt = X';
0095 else
0096     Xt = slmulvec(X, w, 2)';
0097 end
0098 
0099 M2 = X * Xt;    
0100 if ~isequal(lambdas, 0)
0101     inds = (1:dx)' * (dx+1) - dx;
0102     M2(inds) = M2(inds) + lambdas;
0103 end
0104 
0105 if isempty(opts.invparams)
0106     invM2 = inv(M2);
0107 else
0108     invM2 = slinvcov(M2, opts.invparams{:});
0109 end
0110 clear M2;
0111 
0112 M1 = Y * Xt;
0113 
0114 A = M1 * invM2;
0115 
0116     
0117     
0118 
0119 
0120 
0121 
0122 
0123 
0124 
0125 
0126 
0127 
0128

---

Generated on Wed 20-Sep-2006 12:43:11 by m2html © 2003