Learn more about System Identification Toolbox

Residual Analysis

On this page…
What Is Residual Analysis? Supported Model Types What Residual Plots Show for Different Data Domains Displaying the Confidence Interval How to Plot Residuals Using the GUI How to Plot Residuals at the Command Line Example – Examining Model Residuals

What Is Residual Analysis?

Residuals are differences between the one-step-predicted output from the model and the measured output from the validation data set. Thus, residuals represent the portion of the validation data not explained by the model.

Residual analysis consists of two tests: the whiteness test and the independence test.

According to the whiteness test criteria, a good model has the residual autocorrelation function inside the confidence interval of the corresponding estimates, indicating that the residuals are uncorrelated.

According to the independence test criteria, a good model has residuals uncorrelated with past inputs. Evidence of correlation indicates that the model does not describe how part of the output relates to the corresponding input. For example, a peak outside the confidence interval for lag k means that the output y(t) that originates from the input u(t-k) is not properly described by the model.

Your model should pass both the whiteness and the independence tests, except in the following cases:

• For output-error (OE) models and when using instrumental-variable (IV) methods, make sure that your model shows independence of e and u, and pay less attention to the results of the whiteness of e.

  In this case, the modeling focus is on the dynamics G and not the disturbance properties H.

• Correlation between residuals and input for negative lags, is not necessarily an indication of an inaccurate model.

  When current residuals at time t affect future input values, there might be feedback in your system. In the case of feedback, concentrate on the positive lags in the cross-correlation plot during model validation.

Back to Top

Supported Model Types

You can validate parametric linear and nonlinear models by checking the behavior of the model residuals. For a description of residual analysis, see What Residual Plots Show for Different Data Domains.

Note   For nonparametric models, including impulse-response, step-response, and frequency-response models, residual analysis plots are not available. For time-series models, you can only generate model-output plots for parametric models using time-domain time-series (no input) measured data.

Back to Top

What Residual Plots Show for Different Data Domains

Residual analysis plots show different information depending on whether you use time-domain or frequency-domain input-output validation data.

For time-domain validation data, the plot shows the following two axes:

• Autocorrelation function of the residuals for each output

• Cross-correlation between the input and the residuals for each input-output pair

  Note   For time-series models, the residual analysis plot does not provide any input-residual correlation plots.

For frequency-domain validation data, the plot shows the following two axes:

• Estimated power spectrum of the residuals for each output

• Transfer-function amplitude from the input to the residuals for each input-output pair

For linear models, you can estimate a model using time-domain data, and then validate the model using frequency domain data. For nonlinear models, the System Identification Toolbox product supports only time-domain data.

The following figure shows a sample Residual Analysis plot, created in the System Identification Tool GUI.

Back to Top

Displaying the Confidence Interval

You can display a confidence interval on the plot in the GUI to gain insight into the quality of the model. To learn how to show or hide confidence interval, see the description of the plot settings in How to Plot Residuals Using the GUI.

Note   If you are working in the System Identification Tool GUI, you can specify a custom confidence interval. If you are using the resid command, the confidence interface is fixed at 99%.

The confidence interval corresponds to the range of residual values with a specific probability of being statistically insignificant for the system. The toolbox uses the estimated uncertainty in the model parameters to calculate confidence intervals and assumes the estimates have a Gaussian distribution.

For example, for a 95% confidence interval, the region around zero represents the range of residual values that have a 95% probability of being statistically insignificant. You can specify the confidence interval as a probability (between 0 and 1) or as the number of standard deviations of a Gaussian distribution. For example, a probability of 0.99 (99%) corresponds to 2.58 standard deviations.

Back to Top

How to Plot Residuals Using the GUI

To create a residual analysis plot for parametric linear and nonlinear models in the System Identification Tool GUI, select the Model resids check box in the Model Views area. For general information about creating and working with plots, see Working with Plots in the System Identification Tool GUI.

To include or exclude a model on the plot, click the corresponding model icon in the System Identification Tool GUI. Active models display a thick line inside the Model Board icon.

The following table summarizes the Residual Analysis plot settings.

Residual Analysis Plot Settings

Action	Command
Display confidence intervals around zero. Note   Confidence internal are not available for nonlinear ARX and Hammerstein-Wiener models.	To display the dashed lines on either side of the nominal model curve, select Options > Show confidence intervals. Select this option again to hide the confidence intervals. To change the confidence value, select Options > Set % confidence level and choose a value from the list. To enter your own confidence level, select Options > Set confidence level > Other. Enter the value as a probability (between 0 and 1) or as the number of standard deviations of a Gaussian distribution.
Change the number of lags (data samples) for which to compute autocorellation and cross-correlation functions. Note   For frequency-domain validation data, increasing the number of lags increases the frequency resolution of the residual spectrum and the transfer function.	Select Options > Number of lags and choose the value from the list. To enter your own lag value, select Options > Set confidence level > Other. Enter the value as the number of data samples.
(Multiple-output system only) Select a different input-output pair.	Select the input-output by name in the Channel menu.

Back to Top

How to Plot Residuals at the Command Line

The following table summarizes commands that generate residual-analysis plots for linear and nonlinear models. For detailed information about this command, see the corresponding reference page.

Note   Apply pe and resid to one model at a time.

Command	Description	Example
pe	Computes and plots model prediction errors.	To plot the prediction errors for the model model using data data, type the following command: pe(model,data)
resid	Performs whiteness and independence tests on model residuals, or prediction errors. Uses validation data input as model input.	To plot residual correlations for the model model using data data, type the following command: resid(model,data)

Back to Top

Example – Examining Model Residuals

This example shows how you can use residual analysis to evaluate model quality.

Creating Residual Plots

1. To load the sample System Identification Tool session that contains estimated models, type the following command in the MATLAB Command Window:

   ident('dryer2_linear_models')

2. To generate a residual analysis plot, select the Model resids check box in the System Identification Tool GUI.

   This opens an empty plot.

3. In the System Identification Tool window, click each model icon to display it on the Residual Analysis plot.

   

Description of the Residual Plot Axes

The top axes show the autocorrelation of residuals for the output (whiteness test). The horizontal scale is the number of lags, which is the time difference (in samples) between the signals at which the correlation is estimated. The horizontal dashed lines on the plot represent the confidence interval of the corresponding estimates. Any fluctuations within the confidence interval are considered to be insignificant. Two of the models, n4s3 and arx223, produce residuals that enter outside the confidence interval. A good model should have a residual autocorrelation function within the confidence interval, indicating that the residuals are uncorrelated.

The bottom axes show the cross-correlation of the residuals with the input. A good model should have residuals uncorrelated with past inputs (independence test). Evidence of correlation indicates that the model does not describe how the output is formed from the corresponding input. For example, when there is a peak outside the confidence interval for lag k, this means that the contribution to the output y(t) that originates from the input u(t-k) is not properly described by the model. The models arxqs and amx2222 extend beyond the confidence interval and do not perform as well as the other models.

Validating Models Using Analyzing Residuals

To remove models with poor performance from the Residual Analysis plot, click the model icons arxqs, n4s3, arx223, and amx2222 in the System Identification Tool GUI.

The Residual Analysis plot now includes only the three models that pass the residual tests: arx692, n4s6, and amx3322.

The plots for these models fall within the confidence intervals. Thus, when choosing the best model among several estimated models, it is reasonable to pick amx3322 because it is a simpler, low-order model.

Back to Top

Simulating and Predicting Model Output

Impulse and Step Response Plots

© 1984-2009- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS