Validating Models After Estimation

On this page…
When to Validate Models Ways to Validate Models Data for Model Validation Supported Model Plots Definition: Confidence Interval

When to Validate Models

After estimating each model, you can validate whether the model reproduces system behavior within acceptable bounds. You iterate between estimation and validation until you find the simplest model that best captures the system dynamics.

For ideas on how to adjust your modeling strategy based on validation results, see Troubleshooting Models.

Tip   If you have installed the Control System Toolbox product, you can also view models using the LTI Viewer. For more information, see Viewing Model Response Using the LTI Viewer.

Back to Top

Ways to Validate Models

You can use the following approaches to validate models:

• Comparing simulated or predicted model output to measured output.

  See Simulating and Predicting Model Output.

  To simulate identified models in the Simulink environment, see Simulating Identified Model Output in Simulink.

• Analyzing autocorrelation and cross-correlation of the residuals with input.

  See Residual Analysis.

• Analyzing model response. For more information, see the following:

  • Impulse and Step Response Plots

  • Frequency Response Plots

  For information about the response of the noise model, see Noise Spectrum Plots.

• Plotting the poles and zeros of the linear parametric model.

  For more information, see Pole and Zero Plots.

• Comparing the response of nonparametric models, such as impulse-, step-, and frequency-response models, to parametric models, such as linear polynomial models, state-space model, and nonlinear parametric models.

  Note   Do not use this comparison when feedback is present in the system because feedback makes nonparametric models unreliable. To test if feedback is present in the system, use the advice command on the data.

• Compare models using Akaike Information Criterion or Akaike Final Prediction Error.

  For more information, see the aic and fpe reference page.

• Plotting linear and nonlinear blocks of Hammerstein-Wiener and nonlinear ARX models.

  For more information, see Nonlinear Black-Box Model Identification.

Displaying confidence intervals on supported plots helps you assess the uncertainty of model parameters. For more information, see Computing Model Uncertainty.

Back to Top

Data for Model Validation

For plots that compare model response to measured response, such as model output and residual analysis plots, you designate two types of data sets: one for estimating the models (estimation data), and the other for validating the models (validation data). Although you can designate the same data set to be used for estimating and validating the model, you risk overfitting your data. When you validate a model using an independent data set, this process is called cross-validation.

Note   Validation data should be the same in frequency content as the estimation data. If you detrended the estimation data, you must remove the same trend from the validation data. For more information about detrending, see Handling Offsets and Trends in Data.

Back to Top

Supported Model Plots

The following table summarizes the types of supported model plots.

Plot Type	Supported Models	Learn More
Model Output	All linear and nonlinear models	Simulating and Predicting Model Output
Residual Analysis	All linear and nonlinear models	Residual Analysis
Transient Response	All linear parametric models Correlation analysis (nonparametric) models For nonlinear models, only step response.	Impulse and Step Response Plots
Frequency Response	All linear parametric models Spectral analysis (nonparametric) models	Frequency Response Plots
Noise Spectrum	All linear parametric models Spectral analysis (nonparametric) models	Noise Spectrum Plots
Poles and Zeros	All linear parametric models	Pole and Zero Plots
Nonlinear ARX	Nonlinear ARX models only	Nonlinear ARX Plots
Hammerstein-Wiener	Hammerstein-Wiener models only	Hammerstein-Wiener Plots

Back to Top

Definition: Confidence Interval

You can display the confidence interval on the following plot types:

Plot Type	Confidence Interval Corresponds to the Range of ...	More Information on Displaying Confidence Interval
Simulated and Predicted Output	Output values with a specific probability of being the actual output of the system.	Model Output Plots
Residuals	Residual values with a specific probability of being statistically insignificant for the system.	Residuals Plots
Impulse and Step	Response values with a specific probability of being the actual response of the system.	Impulse and Step Plots
Frequency Response	Response values with a specific probability of being the actual response of the system.	Frequency Response Plots
Noise Spectrum	Power-spectrum values with a specific probability of being the actual noise spectrum of the system.	Noise Spectrum Plots
Poles and Zeros	Pole or zero values with a specific probability of being the actual pole or zero of the system.	Pole-Zero Plots

Back to Top

Model Analysis

Plotting Models in the GUI

© 1984-2012- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS