A nonlinear ARX model can be understood as an extension of a linear model. A linear SISO ARX model has this structure:
$$\begin{array}{l}y(t)+{a}_{1}y(t-1)+{a}_{2}y(t-2)+\mathrm{...}+{a}_{na}y(t-na)=\\ \text{}{b}_{1}u(t)+{b}_{2}u(t-1)+\mathrm{...}+{b}_{nb}u(t-nb+1)+e\left(t\right)\end{array}$$
where the input delay nk is zero to simplify the notation.
This structure implies that the current output y(t) is predicted as a weighted sum of past output values and current and past input values. Rewriting the equation as a product:
$$\begin{array}{l}{y}_{p}(t)=\left[-{a}_{1},-{a}_{2},\mathrm{...},-{a}_{na},{b}_{1},{b}_{2},\mathrm{..},{b}_{nb}\right]\ast \\ \text{}{\left[y(t-1),y(t-2),\mathrm{...},y(t-na),u(t),u(t-1),\mathrm{...},u(t-nb-1)\right]}^{T}\end{array}$$
where $$y(t-1),y(t-2),\mathrm{...},y(t-na),u(t),u(t-1),\mathrm{...},u(t-nb-1)$$ are delayed input and output variables, called regressors. The linear ARX model thus predicts the current output y_{p} as a weighted sum of its regressors.
This structure can be extended to create a nonlinear form as:
Instead of the weighted sum that represents a linear mapping, the nonlinear ARX model has a more flexible nonlinear mapping function:
$${y}_{p}(t)=f(y(t-1),y(t-2),y(t-3),\mathrm{...},u(t),u(t-1),u(t-2),\mathrm{..})$$
where f is a nonlinear function. Inputs to f are model regressors. When you specify the nonlinear ARX model structure, you can choose one of several available nonlinear mapping functions in this toolbox (see Nonlinearity Estimators for Nonlinear ARX Models).
Nonlinear ARX regressors can be both delayed input-output variables and more complex, nonlinear expressions of delayed input and output variables. Examples of such nonlinear regressors are y(t-1)^{2}, u(t-1)*y(t-2), tan(u(t-1)), and u(t-1)*y(t-3).
This block diagram represents the structure of a nonlinear ARX model in a simulation scenario:
The nonlinear ARX model computes the output y in two stages:
Computes regressors from the current and past input values and past output data.
In the simplest case, regressors are delayed inputs and outputs, such as u(t-1) and y(t-3)—called standard regressors. You can also specify custom regressors, which are nonlinear functions of delayed inputs and outputs. For example, tan(u(t-1)) or u(t-1)*y(t-3).
By default, all regressors are inputs to both the linear and the nonlinear function blocks of the nonlinearity estimator. You can choose a subset of regressors as inputs to the nonlinear function block.
The nonlinearity estimator block maps the regressors to the model output using a combination of nonlinear and linear functions. You can select from available nonlinearity estimators, such as tree-partition networks, wavelet networks, and multilayer neural networks. You can also exclude either the linear or the nonlinear function block from the nonlinearity estimator.
The nonlinearity estimator block can include linear and nonlinear blocks in parallel. For example:
$$F(x)={L}^{T}(x-r)+d+g\left(Q(x-r)\right)$$
x is a vector of the regressors. $${L}^{T}(x)+d$$ is the output of the linear function block and is affine when d≠0. d is a scalar offset. $$g\left(Q(x-r)\right)$$ represents the output of the nonlinear function block. r is the mean of the regressors x. Q is a projection matrix that makes the calculations well conditioned. The exact form of F(x) depends on your choice of the nonlinearity estimator.
Estimating a nonlinear ARX model computes the model parameter
values, such as L, r, d, Q,
and other parameters specifying g. Resulting models
are idnlarx
objects that store all model data,
including model regressors and parameters of the nonlinearity estimator.
See the idnlarx
reference page
for more information.
System Identification Toolbox™ software provides several nonlinearity estimators F(x) for nonlinear ARX models. For more information about F(x), see Structure of Nonlinear ARX Models.
Each nonlinearity estimator corresponds to an object class in this toolbox. When you estimate nonlinear ARX models in the app, System Identification Toolbox creates and configures objects based on these classes. You can also create and configure nonlinearity estimators at the command line.
Most nonlinearity estimators represent the nonlinear function as a summed series of nonlinear units, such as wavelet networks or sigmoid functions. You can configure the number of nonlinear units n for estimation. For a detailed description of each estimator, see the references page of the corresponding nonlinearity class.
Nonlinearity | Class | Structure | Comments |
---|---|---|---|
Wavelet network (default) | wavenet | $$g(x)={\displaystyle \sum _{k=1}^{n}{\alpha}_{k}}\kappa \left({\beta}_{k}\left(x-{\gamma}_{k}\right)\right)$$ where $$\kappa (s)$$ is the wavelet function. | By default, the estimation algorithm determines the number of units n automatically. |
One layer sigmoid network | sigmoidnet | $$g(x)={\displaystyle \sum _{k=1}^{n}{\alpha}_{k}}\kappa \left({\beta}_{k}\left(x-{\gamma}_{k}\right)\right)$$ where $$\kappa (s)={\left({e}^{s}+1\right)}^{-1}$$ is the sigmoid function. $${\beta}_{k}$$ is a row vector such that $${\beta}_{k}(x-{\gamma}_{k})$$ is a scalar. | Default number of units n is 10. |
Tree partition | treepartition | Piecewise linear function over partitions of the regressor space defined by a binary tree. | The estimation algorithm determines the number of units automatically. Try using tree partitions for modeling data collected over a range of operating conditions. |
F is linear in x | linear | This estimator produces a model that is similar to the linear ARX model, but offers the additional flexibility of specifying custom regressors. | Use to specify custom regressors as the nonlinearity estimator and exclude a nonlinearity mapping function. |
Multilayered neural network | neuralnet | Uses as a network object created using
the Neural Network Toolbox™ software. | |
Custom network (user-defined) | customnet | Similar to sigmoid network but you specify $$\kappa (s)$$. | (For advanced use) Uses the unit function that you specify. |
You can adjust various elements of the nonlinear ARX model structure and fit different models to your data.
Configure model regressors by:
Specifying model order and delay, which creates the set of standard regressors.
For a definition, see Nonlinear ARX Order and Delay.
Creating custom regressors.
Custom regressors are arbitrary functions of past inputs and outputs, such as products, powers, and other MATLAB^{®} expressions of input and output variables. You can specify custom regressors in addition to or instead of standard regressors for greater flexibility in modeling your data.
Including a subset of regressors in the nonlinear function of the nonlinear estimator block.
Selecting which regressors are inputs to the nonlinear function reduces model complexity and keeps the estimation well-conditioned.
Initializing using a linear ARX model.
You can perform this operation only at the command line. The initialization configures the nonlinear ARX model to use standard regressors, which the toolbox computes using the orders and delays of the linear model. See Using Linear Model for Nonlinear ARX Estimation.
Configure the nonlinearity estimator block by:
Specifying and configuring the nonlinear function, including the number of units.
Excluding the nonlinear function from the nonlinear estimator such that F(x)=$${L}^{T}(x)+d$$.
Excluding the linear function from the nonlinear estimator such that F(x)=$$g\left(Q(x-r)\right)$$.
Note: You cannot exclude the linear function from tree partitions and neural networks. |
See these topics for detailed steps to change the model structure:
Estimate a nonlinear ARX model with default configuration by one of the following:
Specifying only model order and input delay. Specifying the order automatically creates standard regressors.
Specifying a linear ARX model. The linear model sets the model orders and linear function of the nonlinear model. You can perform this operation only at the command line.
By default:
The nonlinearity estimator is a wavelet network (see
the wavenet
reference page).
This nonlinearity often provides satisfactory results and uses a fast estimation method.
All of the standard regressors are inputs to the linear and nonlinear functions of the wavelet network.
The order and delay of nonlinear ARX models are positive integers:
na — Number of past output terms used to predict the current output.
nb — Number of past input terms used to predict the current output.
nk — Delay from input to the output in terms of the number of samples.
The meaning of na, nb, and nk is similar to linear ARX model parameters. Orders are scalars for SISO data, and matrices for MIMO data. If you are not sure how to specify the order and delay, you can estimate them as described in Preliminary Step – Estimating Model Orders and Input Delays. Such an estimate is based on linear ARX models and only provides initial guidance—the best orders for a linear ARX model might not be the best orders for a nonlinear ARX model.
System Identification Toolbox software computes standard regressors using model orders.
For example, if you specify this order and delay for a SISO model with input u and output y:
na=2, nb=3, and nk=5
the toolbox computes standard regressors y(t-2), y(t-1), u(t-5), u(t-6), and u(t-7).
You can specify custom regressors in addition to standard regressors, as described in How to Estimate Nonlinear ARX Models in the System Identification App and How to Estimate Nonlinear ARX Models at the Command Line.
The estimation algorithm depends on your choice of nonlinearity
estimator and the estimation options specified using the nlarxOptions
option set. You can set algorithm
properties both in the app and at the command line.
Focus property of nlarxOptions
. By default, estimating nonlinear ARX models minimizes one-step
prediction errors, which corresponds to a Focus
value
of 'prediction'
.
If you want a model that is optimized for reproducing simulation
behavior, try setting the Focus
value to 'simulation'
.
In this case, you cannot use treepartition
and neuralnet
because
these nonlinearity estimators are not differentiable. Minimization
of simulation error requires differentiable nonlinear functions and
takes more time than one-step-ahead prediction error minimization.
Common algorithm properties in nlarxOptions
.
MaxIter
— Maximum number
of iterations.
SearchMethod
— Search method
for minimization of prediction or simulation errors, such as Gauss-Newton
and Levenberg-Marquardt line search, and Trust-region reflective Newton
approach. By default, the algorithm uses a combination of these methods.
Tolerance
— Condition for
terminating iterative search when the expected improvement of the
parameter values is less than a specified value.
Display
— Shows progress
of iterative minimization in the MATLAB Command Window.
In the System Identification app, select Estimate > Nonlinear models to open the Nonlinear Models dialog box.
In the Configure tab, verify
that Nonlinear ARX
is selected in the Model
type list.
(Optional) Edit the Model name by clicking . The name of the model should be unique to all nonlinear ARX models in the System Identification app.
(Optional) If you want to refine the parameters of a previously estimated model or configure the model structure to match that of an existing model:
Click Initialize. A Initial Model Specification dialog box opens.
In the Initial Model drop-down list, select a nonlinear ARX model.
The model must be in the Model Board of the System Identification app and the input/output dimensions of this initial model must match that of the estimation data, selected as Working Data in the app.
Click OK.
The model structure as well as the parameter values are updated to match that of the selected model.
Clicking Estimate causes the estimation to use the parameters of the initial model as the starting point.
Note: When you select an initial model, you can optionally update the estimation algorithm settings to match those used for the initial model by selecting the Inherit the model's algorithm properties option. |
Keep the default settings in the Nonlinear Models dialog box that specify the model structure and the algorithm, or modify these settings:
Note: For more information about available options, click Help in the Nonlinear Models dialog box to open the app help. |
What to Configure | Options in Nonlinear Models GUI | Comment |
---|---|---|
Model order | In the Regressors tab, edit the No. of Terms corresponding to each input and output channel. | Model order na is the output number of terms and nb is the input number of terms. |
Input delay | In the Regressors tab, edit the Delay corresponding to an input channel. | If you do not know the input delay value, click Infer Input Delay. This action opens the Infer Input Delay dialog box to suggest possible delay values. |
Regressors | In the Regressors tab, click Edit Regressors. | This action opens the Model Regressors dialog box. Use this dialog box to create custom regressors or to include specific regressors in the nonlinear block. |
Nonlinearity estimator | In the Model Properties tab. | To use all standard and custom regressors in the linear block
only, you can exclude the nonlinear block by setting Nonlinearity to None . |
Estimation algorithm | In the Estimate tab, click Algorithm Options. |
To obtain regularized estimates of model parameters, in the Estimate tab, click Estimation Options. Specify the regularization constants in the Regularization_Tradeoff_Constant and Regularization_Weighting fields. To learn more, see Regularized Estimates of Model Parameters.
Click Estimate to add this model to the System Identification app.
The Estimate tab displays the estimation progress and results.
Validate the model response by selecting the desired plot in the Model Views area of the System Identification app. For more information about validating models, see How to Plot Nonlinear ARX Plots Using the App.
If you get a poor fit, try changing the model structure or algorithm configuration in step 5.
You can export the model to the MATLAB workspace by dragging it to To Workspace in the System Identification app.
Linear and nonlinear regressors.
Configure the nonlinearity estimator.
Include only nonlinear function in nonlinearity estimator.
Include only linear function in nonlinearity estimator.
What if you cannot get a satisfactory model?
This example shows how to use nlarx
to estimate a nonlinear ARX model for measured input/output data.
Prepare the data for estimation.
load twotankdata
z = iddata(y, u, 0.2);
ze = z(1:1000); zv = z(1001:3000);
Estimate several models using different model orders, delays, and nonlinearity settings.
m1 = nlarx(ze,[2 2 1]); m2 = nlarx(ze,[2 2 3]); m3 = nlarx(ze,[2 2 3],wavenet('num',8)); m4 = nlarx(ze,[2 2 3],wavenet('num',8),... 'nlr', [1 2]);
An alternative way to perform the estimation is to configure the model structure first, and then to estimate this model.
m5 = idnlarx([2 2 3],sigmoidnet('num',14),'nlr',[1 2]); m5 = pem(ze,m5);
Compare the resulting models by plotting the model outputs with the measured output.
compare(zv, m1,m2,m3,m4,m5)
You can use an ARX structure polynomial model (idpoly
with
only A and B as active polynomials)
for nonlinear ARX estimation.
Tip To learn more about when to use linear models, see When to Fit Nonlinear Models. |
Typically, you create a linear ARX model using the arx
command. You can provide the linear
model only at the command line when constructing (see idnlarx
) or estimating (see nlarx
) a nonlinear ARX model.
The software uses the linear model for initializing the nonlinear ARX estimation:
Assigns the linear model orders as initial values
of nonlinear model orders (na
and nb
properties
of the idnlarx
object) and delays (nk
property)
to compute standard regressors in the nonlinear ARX model structure.
Uses the A and B polynomials
of the linear model to compute the linear function of the nonlinearity
estimators (LinearCoef
parameter of the nonlinearity
estimator object), except for neural network nonlinearity estimator.
During estimation, the estimation algorithm uses these values to further adjust the nonlinear model to the data. The initialization always provides a better fit to the estimation data than the linear ARX model.
Estimate a nonlinear ARX model initialized using a linear model by typing
m = nlarx(data,LinARXModel)
LinARXModel
is an idpoly
object
of ARX structure. m
is an idnlarx
object. data
is
a time-domain iddata
object.
By default, the nonlinearity estimator is the wavelet network
(wavenet
object). This network
takes all standard regressors computed using orders and delay of LinARXModel
as
inputs to its linear and nonlinear functions. The software computes
the LinearCoef
parameter of the wavenet
object
using the A
and B
polynomials
of the linear ARX model.
Tip When you use the same data set, a nonlinear ARX model initialized using a linear ARX model produces a better fit than the linear ARX model. |
Specify a different nonlinearity estimator, for example a sigmoid network:
m = nlarx(data,LinARXModel,'sigmoid')
Create an nlarxOptions
option set, setting
the Focus
property to simulation error minimization:
opt = nlarxOptions('Focus','simulation'); m = nlarx(data,LinARXModel,'sigmoid',opt)
After each estimation, validate the model by comparing the simulated response to the data. To improve the fit of the nonlinear ARX model, adjust various elements of the nonlinear ARX structure. For more information, see Ways to Configure Nonlinear ARX Estimation.
This example shows how to estimate nonlinear ARX models by using linear ARX models.
Load the estimation data.
load throttledata.mat
This command loads the data object ThrottleData
into
the workspace. The object contains input and output samples collected
from an engine throttle system, sampled at a rate of 100 Hz.
A DC motor controls the opening angle of the butterfly valve in the throttle system. A step signal (in volts) drives the DC motor. The output is the angular position (in degrees) of the valve.
Plot the data to view and analyze the data characteristics.
plot(ThrottleData)
In the normal operating range of 15-90 degrees, the input and output variables have a linear relationship. You use a linear model of low order to model this relationship.
In the throttle system, a hard stop limits the valve position
to 90
degrees, and a spring brings
the valve to 15
degrees when the
DC motor is turned off. These physical components introduce nonlinearities
that a linear model cannot capture.
Estimate an ARX model to model the linear behavior of this single-input single-output system in the normal operating range.
% Detrend the data because linear models cannot capture offsets. Tr = getTrend(ThrottleData); Tr.OutputOffset = 15; DetrendedData = detrend(ThrottleData,Tr); % Estimate a linear ARX model with na=2, nb=1, nk=1. opt = arxOptions('Focus','simulation'); LinearModel = arx(DetrendedData,[2 1 1],opt);
Compare the simulated model response with estimation data.
compare(DetrendedData, LinearModel)
The linear model captures the rising and settling behavior in the linear operating range but does not account for output saturation at 90 degrees.
Estimate a nonlinear ARX model to model the output saturation.
opt = nlarxOptions('Focus','simulation'); NonlinearModel = nlarx(ThrottleData,LinearModel,'sigmoidnet',opt);
The software uses the orders and delay of the linear model for
the orders of the nonlinear model. In addition, the software computes
the linear function of sigmoidnet
nonlinearity
estimator.
Compare the nonlinear model with data.
compare(ThrottleData, NonlinearModel)
The model captures the nonlinear effects (output saturation) and improves the overall fit to data.
The Nonlinear ARX plot displays the characteristics of model
nonlinearities as a function of one or two regressors. The model nonlinearity
(model.Nonlinearity
) is a nonlinearity estimator
function, such as wavenet
, sigmoidnet
, treepartition
,
and uses model regressors as its inputs. The value of the nonlinearity
is plotted by projecting its response in 2 or 3-dimensional space.
The plot uses one or two regressors as the plot axes for 2- or 3-D
plots, respectively and a center point (cross-section location) for
the other regressors.
Examining a nonlinear ARX plot can help you gain insight into which regressors have the strongest effect on the model output. Understanding the relative importance of the regressors on the output can help you decide which regressors should be included in the nonlinear function.
Furthermore, you can create several nonlinear models for the
same data set using different nonlinearity estimators, such a wavenet
network
and treepartition
, and then compare the nonlinear
surfaces of these models. Agreement between nonlinear surfaces increases
the confidence that these nonlinear models capture the true dynamics
of the system.
In the plot window, you can choose:
The regressors to use on the plot axes, and specify the center points for the other regressors in the configuration panel. For multi-output models, each output is plotted separately.
The output to view from the drop-down list located at the top of the plot.
You can plot linear and nonlinear blocks of nonlinear ARX models.
To create a nonlinear ARX plot in the System Identification app, select the Nonlinear ARX check box in the Model Views area. For general information about creating and working with plots, see Working with Plots.
Note: The Nonlinear ARX check box is unavailable if you do not have a nonlinear ARX model in the Model Board. |
The following figure shows a sample nonlinear ARX plot.
You can use the following approaches to validate nonlinear ARX models at the command line:
Compare Model Output to Measured Output
Simulate and Predict Model Response
Check Iterative Search Termination Conditions
To include or exclude a model on the plot, click the corresponding model icon in the System Identification app. Active models display a thick line inside the Model Board icon.
To configure the plot:
If your model contains multiple outputs, select the output channel in the Select nonlinearity at output drop-down list. Selecting the output channel displays the nonlinearity values that correspond to this output channel.
If the regressor selection options are not visible, click to expand the Nonlinear ARX Model Plot window.
Select Regressor 1 from the list of available regressors. In the Range field, enter the range of values to include on the plot for this regressor. The regressor values are plotted on the Reg1 axis.
Specify a second regressor for a 3-D plot by selecting one of the following types of options:
Select Regressor 2 to display three axes. In the Range field, enter the range of values to include on the plot for this regressor. The regressor values are plotted on the Reg2 axis.
Select <none>
in
the Regressor 2 list to display only two axes.
To fix the values of the regressor that are not displayed, click Fix Values. In the Fix Regressor Values dialog box, double-click the Value cell to edit the constant value of the corresponding regressor. The default values are determined during model estimation. Click OK.
Click Apply to update the plot.
To change the grid of the regressor space along each axis, Options > Set number of samples, and enter the number of samples to use for each regressor. Click Apply and then Close.
For example, if the number of samples is 20, each regressor variable contains 20 points in its specified range. For a 3-D plots, this results in evaluating the nonlinearity at 20 x 20 = 400 points.
The following table summarizes the commands to modify the appearance of the Nonlinear ARX plot.
Changing Appearance of the Nonlinear ARX Plot
Action | Command | |
---|---|---|
Change axis limits. | Select Options > Set axis limits to open the Axis Limits dialog box, and edit the limits. Click Apply. | |
Hide or show the legend. | Select Style > Legend. Select this option again to show the legend. | |
(Three axes only) Rotate in three dimensions.
| Select Style > Rotate 3D and drag the axes on the plot to a new orientation. To disable three-dimensional rotation, select Style > Rotate 3D again. |
In most applications, sim(idnlarx)
and predict
are sufficient for computing the
simulated and predicted model response, respectively. This advanced
topic describes how the software evaluates the output of nonlinearity
estimators and uses this output to compute the model response.
Evaluating the predicted output of a nonlinearity for a specific regressor value x requires that you first extract the nonlinearity F and regressors from the model:
F = m.Nonlinearity; x = getreg(m,'all',data) % computes regressors
Evaluate F(x):
y = evaluate(F,x)
where x
is a row vector of regressor values.
You can also evaluate predicted output values at multiple time instants by evaluating F for several regressor vectors simultaneously:
y = evaluate(F,[x1;x2;x3])
This example shows how the software computes the simulated and predicted output of the model as a result of evaluating the output of its nonlinearity estimator for given regressor values.