# RegressionGP

Gaussian process regression model

## Description

`RegressionGP` is a Gaussian process regression (GPR) model. You can train a GPR model, using `fitrgp`. Using the trained model, you can

• Predict responses for training data using `resubPredict` or new predictor data using `predict`. You can also compute the prediction intervals.

• Compute the regression loss for training data using `resubLoss` or new data using `loss`.

## Creation

Create a `RegressionGP` object by using `fitrgp`.

## Properties

expand all

### Fitting

Method used to estimate the basis function coefficients, β; noise standard deviation, σ; and kernel parameters, θ, of the GPR model, stored as a character vector. It can be one of the following.

Fit MethodDescription
`'none'`No estimation. `fitrgp` uses the initial parameter values as the parameter values.
`'exact'`Exact Gaussian process regression.
`'sd'`Subset of data points approximation.
`'sr'`Subset of regressors approximation.
`'fic'`Fully independent conditional approximation.

Explicit basis function used in the GPR model, stored as a character vector or a function handle. It can be one of the following. If n is the number of observations, the basis function adds the term H*β to the model, where H is the basis matrix and β is a p-by-1 vector of basis coefficients.

Explicit BasisBasis Matrix
`'none'`Empty matrix.
`'constant'`

`$H=1$`

H is an n-by-1 vector of 1s, where n is the number of observations.

`'linear'`

`$H=\left[1,X\right]$`

X is the expanded predictor data after the software creates dummy variables for the categorical variables. For details about creating dummy variables, see `CategoricalPredictors`.

`'pureQuadratic'`

`$H=\left[1,X,{X}_{2}\right],$`

where

`${X}_{2}=\left[\begin{array}{cccc}{x}_{11}^{2}& {x}_{12}^{2}& \cdots & {x}_{1d}^{2}\\ {x}_{21}^{2}& {x}_{22}^{2}& \cdots & {x}_{2d}^{2}\\ ⋮& ⋮& ⋮& ⋮\\ {x}_{n1}^{2}& {x}_{n2}^{2}& \cdots & {x}_{nd}^{2}\end{array}\right].$`

For this basis option, `RegressionGP` does not support X with categorical predictors.

Function handle

Function handle, `hfcn`, that `fitrgp` calls as:

`$H=hfcn\left(X\right),$`

where X is an n-by-d matrix of predictors, d is the number of predictors after the software creates dummy variables for the categorical variables, and H is an n-by-p matrix of basis functions.

Data Types: `char` | `function_handle`

Estimated coefficients for the explicit basis functions, stored as a vector. You can define the explicit basis function by using the `BasisFunction` name-value pair argument in `fitrgp`.

Data Types: `double`

Estimated noise standard deviation of the GPR model, stored as a scalar value.

Data Types: `double`

Categorical predictor indices, specified as a vector of positive integers. `CategoricalPredictors` contains index values indicating that the corresponding predictors are categorical. The index values are between 1 and `p`, where `p` is the number of predictors used to train the model. If none of the predictors are categorical, then this property is empty (`[]`).

Data Types: `single` | `double`

This property is read-only.

Cross-validation optimization of hyperparameters, specified as a `BayesianOptimization` object or a table of hyperparameters and associated values. This property is nonempty if the `'OptimizeHyperparameters'` name-value pair argument is nonempty when you create the model. The value of `HyperparameterOptimizationResults` depends on the setting of the `Optimizer` field in the `HyperparameterOptimizationOptions` structure when you create the model.

Value of `Optimizer` FieldValue of `HyperparameterOptimizationResults`
`'bayesopt'` (default)Object of class `BayesianOptimization`
`'gridsearch'` or `'randomsearch'`Table of hyperparameters used, observed objective function values (cross-validation loss), and rank of observations from lowest (best) to highest (worst)

Maximized marginal log likelihood of the GPR model, stored as a scalar value if the `FitMethod` is different from `'none'`. If `FitMethod` is `'none'`, then `LogLikelihood` is empty.

If `FitMethod` is `'sd'`, `'sr'`, or `'fic'`, then `LogLikelihood` is the maximized approximation of the marginal log likelihood of the GPR model.

Data Types: `double`

Parameters used for training the GPR model, stored as a `GPParams` object.

### Kernel Function

Form of the covariance function used in the GPR model, stored as a character vector containing the name of the built-in covariance function or a function handle. It can be one of the following.

FunctionDescription
`'squaredexponential'`Squared exponential kernel.
`'matern32'`Matern kernel with parameter 3/2.
`'matern52'`Matern kernel with parameter 5/2.
`'ardsquaredexponential'`Squared exponential kernel with a separate length scale per predictor.
`'ardmatern32'`Matern kernel with parameter 3/2 and a separate length scale per predictor.
`'ardmatern52'`Matern kernel with parameter 5/2 and a separate length scale per predictor.
Function handleA function handle that `fitrgp` can call like this:
`Kmn = kfcn(Xm,Xn,theta)`
where `Xm` is an m-by-d matrix, `Xn` is an n-by-d matrix, and `Kmn` is an m-by-n matrix of kernel products such that `Kmn`(i,j) is the kernel product between `Xm`(i,:) and `Xn`(j,:). d is the number of predictor variables after the software creates dummy variables for the categorical variables. For details about creating dummy variables, see `CategoricalPredictors`.
`theta` is the r-by-1 unconstrained parameter vector for `kfcn`.

Data Types: `char` | `function_handle`

Information about the parameters of the kernel function used in the GPR model, stored as a structure with the following fields.

Field NameDescription
`Name`Name of the kernel function
`KernelParameters`Vector of the estimated kernel parameters
`KernelParameterNames`Names associated with the elements of `KernelParameters`.

Data Types: `struct`

### Prediction

Method that `predict` uses to make predictions from the GPR model, stored as a character vector. It can be one of the following.

`PredictMethod`Description
`'exact'`Exact Gaussian process regression
`'bcd'`Block Coordinate Descent
`'sd'`Subset of Data points approximation
`'sr'`Subset of Regressors approximation
`'fic'`Fully Independent Conditional approximation

Weights used to make predictions from the trained GPR model, stored as a numeric vector. `predict` computes the predictions for a new predictor matrix `Xnew` by using the product

`$K\left({X}_{new},A\right)*\alpha \text{\hspace{0.17em}}.$`

$K\left({X}_{new},A\right)$ is the matrix of kernel products between ${X}_{new}$ and active set vector A and α is a vector of weights.

Data Types: `double`

Information on block coordinate descent (BCD)-based computation of `Alpha` when `PredictMethod` is `'bcd'`, stored as a structure containing the following fields.

Field NameDescription
`Gradient`n-by-1 vector containing the gradient of the BCD objective function at convergence.
`Objective`Scalar containing the BCD objective function at convergence.
`SelectionCounts`n-by-1 integer vector indicating the number of times each point was selected into a block during BCD.

`Alpha` property contains the `Alpha` vector computed from BCD.

If `PredictMethod` is not `'bcd'`, then `BCDInformation` is empty.

Data Types: `struct`

Transformation applied to the predicted response, stored as a character vector describing how the response values predicted by the model are transformed. In `RegressionGP`, `ResponseTransform` is `'none'` by default, and `RegressionGP` does not use `ResponseTransform` when making predictions.

### Active Set Selection

Subset of training data used to make predictions from the GPR model, stored as a matrix.

`predict` computes the predictions for a new predictor matrix `Xnew` by using the product

`$K\left({X}_{new},A\right)*\alpha \text{\hspace{0.17em}}.$`

$K\left({X}_{new},A\right)$ is the matrix of kernel products between ${X}_{new}$ and active set vector A and α is a vector of weights.

`ActiveSetVectors` is equal to the training data `X` for exact GPR fitting and a subset of the training data `X` for sparse GPR methods. When there are categorical predictors in the model, `ActiveSetVectors` contains dummy variables for the corresponding predictors.

Data Types: `double`

History of interleaved active set selection and parameter estimation for `FitMethod` equal to `'sd'`, `'sr'`, or `'fic'`, stored as a structure with the following fields.

Field NameDescription
`ParameterVector`Cell array containing the parameter vectors: basis function coefficients, β, kernel function parameters θ, and noise standard deviation σ.
`ActiveSetIndices`Cell array containing the active set indices.
`Loglikelihood`Vector containing the maximized log likelihoods.
`CriterionProfile`Cell array containing the active set selection criterion values as the active set grows from size 0 to its final size.

Data Types: `struct`

Method used to select the active set for sparse methods (`'sd'`,`'sr'`, or `'fic'`), stored as a character vector. It can be one of the following.

`ActiveSetMethod`Description
`'sgma'`Sparse greedy matrix approximation
`'entropy'`Differential entropy-based selection
`'likelihood'`Subset of regressors log likelihood-based selection
`'random'`Random selection

The selected active set is used in parameter estimation or prediction, depending on the choice of `FitMethod` and `PredictMethod` in `fitrgp`.

Size of the active set for sparse methods (`'sd'`,`'sr'`, or `'fic'`), stored as an integer value.

Data Types: `double`

Indicators for selected active set for making predictions from the trained GPR model, stored as a logical vector. These indicators mark the subset of training data that `fitrgp` selects as the active set. For example, if `X` is the original training data, then ```ActiveSetVectors = X(IsActiveSetVector,:)```.

Data Types: `logical`

### Training Data

Number of observations in training data, stored as a scalar value.

Data Types: `double`

Training data, stored as an n-by-d table or matrix, where n is the number of observations and d is the number of predictor variables (columns) in the training data. If the GPR model is trained on a table, then `X` is a table. Otherwise, `X` is a matrix.

Data Types: `double` | `table`

Observed response values used to train the GPR model, stored as an n-by-1 vector, where n is the number of observations.

Data Types: `double`

Names of predictors used in the GPR model, stored as a cell array of character vectors. Each name (cell) corresponds to a column in `X`.

Data Types: `cell`

Names of expanded predictors for the GPR model, stored as a cell array of character vectors. Each name (cell) corresponds to a column in `ActiveSetVectors`.

If the model uses dummy variables for categorical variables, then `ExpandedPredictorNames` includes the names that describe the expanded variables. Otherwise, `ExpandedPredictorNames` is the same as `PredictorNames`.

Data Types: `cell`

Name of the response variable in the GPR model, stored as a character vector.

Data Types: `char`

Means of predictors used for training the GPR model if the training data is standardized, stored as a 1-by-d vector. If the training data is not standardized, `PredictorLocation` is empty.

If `PredictorLocation` is not empty, then the `predict` method centers the predictor values by subtracting the respective element of `PredictorLocation` from every column of `X`.

If there are categorical predictors, then `PredictorLocation` includes a 0 for each dummy variable corresponding to those predictors. The dummy variables are not centered or scaled.

Data Types: `double`

Standard deviations of predictors used for training the GPR model if the training data is standardized, stored as a 1-by-d vector. If the training data is not standardized, `PredictorScale` is empty.

If `PredictorScale` is not empty, the `predict` method scales the predictors by dividing every column of `X` by the respective element of `PredictorScale` (after centering using `PredictorLocation`).

If there are categorical predictors, then `PredictorLocation` includes a 1 for each dummy variable corresponding to those predictors. The dummy variables are not centered or scaled.

Data Types: `double`

Rows of the original training data stored in the model, specified as a logical vector. This property is empty if all rows are stored in `X` and `Y`.

Data Types: `logical`

## Object Functions

 `compact` Reduce size of machine learning model `crossval` Cross-validate machine learning model `lime` Local interpretable model-agnostic explanations (LIME) `loss` Regression error for Gaussian process regression model `partialDependence` Compute partial dependence `plotPartialDependence` Create partial dependence plot (PDP) and individual conditional expectation (ICE) plots `postFitStatistics` Compute post-fit statistics for the exact Gaussian process regression model `predict` Predict response of Gaussian process regression model `resubLoss` Resubstitution regression loss `resubPredict` Predict responses for training data using trained regression model `shapley` Shapley values

## Examples

collapse all

Generate sample data.

```rng(0,'twister'); % For reproducibility n = 1000; x = linspace(-10,10,n)'; y = 1 + x*5e-2 + sin(x)./x + 0.2*randn(n,1);```

Fit a GPR model using a linear basis function and the exact fitting method to estimate the parameters. Also use the exact prediction method.

```gprMdl = fitrgp(x,y,'Basis','linear',... 'FitMethod','exact','PredictMethod','exact');```

Predict the response corresponding to the rows of `x` (resubstitution predictions) using the trained model.

`ypred = resubPredict(gprMdl);`

Plot the true response with the predicted values.

```plot(x,y,'b.'); hold on; plot(x,ypred,'r','LineWidth',1.5); xlabel('x'); ylabel('y'); legend('Data','GPR predictions'); hold off```

expand all

## Tips

• You can access the properties of this class using dot notation. For example, `KernelInformation` is a structure holding the kernel parameters and their names. Hence, to access the kernel function parameters of the trained model `gprMdl`, use `gprMdl.KernelInformation.KernelParameters`.

## Version History

Introduced in R2015b

expand all