Documentation

Identifying Nonlinear ARX Models

Nonlinear ARX Model Extends the Linear ARX Structure

A nonlinear ARX model can be understood as an extension of a linear model. A linear SISO ARX model has this structure:

where the input delay nk is zero to simplify the notation.

This structure implies that the current output y(t) is predicted as a weighted sum of past output values and current and past input values. Rewriting the equation as a product:

where $y\left(t-1\right),y\left(t-2\right),...,y\left(t-na\right),u\left(t\right),u\left(t-1\right),...,u\left(t-nb-1\right)$ are delayed input and output variables, called regressors. The linear ARX model thus predicts the current output yp as a weighted sum of its regressors.

This structure can be extended to create a nonlinear form as:

• Instead of the weighted sum that represents a linear mapping, the nonlinear ARX model has a more flexible nonlinear mapping function:

${y}_{p}\left(t\right)=f\left(y\left(t-1\right),y\left(t-2\right),y\left(t-3\right),...,u\left(t\right),u\left(t-1\right),u\left(t-2\right),..\right)$

where f is a nonlinear function. Inputs to f are model regressors. When you specify the nonlinear ARX model structure, you can choose one of several available nonlinear mapping functions in this toolbox (see Nonlinearity Estimators for Nonlinear ARX Models).

• Nonlinear ARX regressors can be both delayed input-output variables and more complex, nonlinear expressions of delayed input and output variables. Examples of such nonlinear regressors are y(t-1)2, u(t-1)*y(t-2), tan(u(t-1)), and u(t-1)*y(t-3).

Structure of Nonlinear ARX Models

This block diagram represents the structure of a nonlinear ARX model in a simulation scenario:

The nonlinear ARX model computes the output y in two stages:

1. Computes regressors from the current and past input values and past output data.

In the simplest case, regressors are delayed inputs and outputs, such as u(t-1) and y(t-3)—called standard regressors. You can also specify custom regressors, which are nonlinear functions of delayed inputs and outputs. For example, tan(u(t-1)) or u(t-1)*y(t-3).

By default, all regressors are inputs to both the linear and the nonlinear function blocks of the nonlinearity estimator. You can choose a subset of regressors as inputs to the nonlinear function block.

2. The nonlinearity estimator block maps the regressors to the model output using a combination of nonlinear and linear functions. You can select from available nonlinearity estimators, such as tree-partition networks, wavelet networks, and multilayer neural networks. You can also exclude either the linear or the nonlinear function block from the nonlinearity estimator.

The nonlinearity estimator block can include linear and nonlinear blocks in parallel. For example:

$F\left(x\right)={L}^{T}\left(x-r\right)+d+g\left(Q\left(x-r\right)\right)$

x is a vector of the regressors. ${L}^{T}\left(x\right)+d$ is the output of the linear function block and is affine when d≠0. d is a scalar offset. $g\left(Q\left(x-r\right)\right)$ represents the output of the nonlinear function block. r is the mean of the regressors x. Q is a projection matrix that makes the calculations well conditioned. The exact form of F(x) depends on your choice of the nonlinearity estimator.

Estimating a nonlinear ARX model computes the model parameter values, such as L, r, d, Q, and other parameters specifying g. Resulting models are idnlarx objects that store all model data, including model regressors and parameters of the nonlinearity estimator. See the idnlarx reference page for more information.

Nonlinearity Estimators for Nonlinear ARX Models

System Identification Toolbox™ software provides several nonlinearity estimators F(x) for nonlinear ARX models. For more information about F(x), see Structure of Nonlinear ARX Models.

Each nonlinearity estimator corresponds to an object class in this toolbox. When you estimate nonlinear ARX models in the app, System Identification Toolbox creates and configures objects based on these classes. You can also create and configure nonlinearity estimators at the command line.

Most nonlinearity estimators represent the nonlinear function as a summed series of nonlinear units, such as wavelet networks or sigmoid functions. You can configure the number of nonlinear units n for estimation. For a detailed description of each estimator, see the references page of the corresponding nonlinearity class.

Wavelet network
(default)
wavenet
$g\left(x\right)=\sum _{k=1}^{n}{\alpha }_{k}\kappa \left({\beta }_{k}\left(x-{\gamma }_{k}\right)\right)$

where $\kappa \left(s\right)$ is the wavelet function.

By default, the estimation algorithm determines the number of units n automatically.
One layer sigmoid networksigmoidnet
$g\left(x\right)=\sum _{k=1}^{n}{\alpha }_{k}\kappa \left({\beta }_{k}\left(x-{\gamma }_{k}\right)\right)$

where $\kappa \left(s\right)={\left({e}^{s}+1\right)}^{-1}$ is the sigmoid function. ${\beta }_{k}$ is a row vector such that ${\beta }_{k}\left(x-{\gamma }_{k}\right)$ is a scalar.

Default number of units n is 10.
Tree partitiontreepartitionPiecewise linear function over partitions of the regressor space defined by a binary tree.The estimation algorithm determines the number of units automatically.
Try using tree partitions for modeling data collected over a range of operating conditions.
F is linear in xlinearThis estimator produces a model that is similar to the linear ARX model, but offers the additional flexibility of specifying custom regressors.Use to specify custom regressors as the nonlinearity estimator and exclude a nonlinearity mapping function.
Multilayered neural networkneuralnetUses as a network object created using the Neural Network Toolbox™ software.
Custom network
(user-defined)
customnetSimilar to sigmoid network but you specify $\kappa \left(s\right)$.(For advanced use)
Uses the unit function that you specify.

Ways to Configure Nonlinear ARX Estimation

Configurable Elements of Nonlinear ARX Structure

You can adjust various elements of the nonlinear ARX model structure and fit different models to your data.

Configure model regressors by:

• Specifying model order and delay, which creates the set of standard regressors.

For a definition, see Nonlinear ARX Order and Delay.

• Creating custom regressors.

Custom regressors are arbitrary functions of past inputs and outputs, such as products, powers, and other MATLAB® expressions of input and output variables. You can specify custom regressors in addition to or instead of standard regressors for greater flexibility in modeling your data.

• Including a subset of regressors in the nonlinear function of the nonlinear estimator block.

Selecting which regressors are inputs to the nonlinear function reduces model complexity and keeps the estimation well-conditioned.

• Initializing using a linear ARX model.

You can perform this operation only at the command line. The initialization configures the nonlinear ARX model to use standard regressors, which the toolbox computes using the orders and delays of the linear model. See Using Linear Model for Nonlinear ARX Estimation.

Configure the nonlinearity estimator block by:

• Specifying and configuring the nonlinear function, including the number of units.

• Excluding the nonlinear function from the nonlinear estimator such that F(x)=${L}^{T}\left(x\right)+d$.

• Excluding the linear function from the nonlinear estimator such that F(x)=$g\left(Q\left(x-r\right)\right)$.

 Note:   You cannot exclude the linear function from tree partitions and neural networks.

See these topics for detailed steps to change the model structure:

Default Nonlinear ARX Structure

Estimate a nonlinear ARX model with default configuration by one of the following:

• Specifying only model order and input delay. Specifying the order automatically creates standard regressors.

• Specifying a linear ARX model. The linear model sets the model orders and linear function of the nonlinear model. You can perform this operation only at the command line.

By default:

• The nonlinearity estimator is a wavelet network (see the wavenet reference page).

This nonlinearity often provides satisfactory results and uses a fast estimation method.

• All of the standard regressors are inputs to the linear and nonlinear functions of the wavelet network.

Nonlinear ARX Order and Delay

The order and delay of nonlinear ARX models are positive integers:

• na — Number of past output terms used to predict the current output.

• nb — Number of past input terms used to predict the current output.

• nk — Delay from input to the output in terms of the number of samples.

The meaning of na, nb, and nk is similar to linear ARX model parameters. Orders are scalars for SISO data, and matrices for MIMO data. If you are not sure how to specify the order and delay, you can estimate them as described in Preliminary Step – Estimating Model Orders and Input Delays. Such an estimate is based on linear ARX models and only provides initial guidance—the best orders for a linear ARX model might not be the best orders for a nonlinear ARX model.

System Identification Toolbox software computes standard regressors using model orders.

For example, if you specify this order and delay for a SISO model with input u and output y:

na=2, nb=3, and nk=5

the toolbox computes standard regressors y(t-2), y(t-1), u(t-5), u(t-6), and u(t-7).

You can specify custom regressors in addition to standard regressors, as described in How to Estimate Nonlinear ARX Models in the System Identification App and How to Estimate Nonlinear ARX Models at the Command Line.

Estimation Algorithm for Nonlinear ARX Models

The estimation algorithm depends on your choice of nonlinearity estimator and the estimation options specified using the nlarxOptions option set. You can set algorithm properties both in the app and at the command line.

Focus property of nlarxOptions.  By default, estimating nonlinear ARX models minimizes one-step prediction errors, which corresponds to a Focus value of 'prediction'.

If you want a model that is optimized for reproducing simulation behavior, try setting the Focus value to 'simulation'. In this case, you cannot use treepartition and neuralnet because these nonlinearity estimators are not differentiable. Minimization of simulation error requires differentiable nonlinear functions and takes more time than one-step-ahead prediction error minimization.

Common algorithm properties in nlarxOptions

• MaxIter — Maximum number of iterations.

• SearchMethod — Search method for minimization of prediction or simulation errors, such as Gauss-Newton and Levenberg-Marquardt line search, and Trust-region reflective Newton approach. By default, the algorithm uses a combination of these methods.

• Tolerance — Condition for terminating iterative search when the expected improvement of the parameter values is less than a specified value.

• Display — Shows progress of iterative minimization in the MATLAB Command Window.

How to Estimate Nonlinear ARX Models in the System Identification App

1. In the System Identification app, select Estimate > Nonlinear models to open the Nonlinear Models dialog box.

2. In the Configure tab, verify that Nonlinear ARX is selected in the Model type list.

3. (Optional) Edit the Model name by clicking . The name of the model should be unique to all nonlinear ARX models in the System Identification app.

4. (Optional) If you want to refine the parameters of a previously estimated model or configure the model structure to match that of an existing model:

1. Click Initialize. A Initial Model Specification dialog box opens.

2. In the Initial Model drop-down list, select a nonlinear ARX model.

The model must be in the Model Board of the System Identification app and the input/output dimensions of this initial model must match that of the estimation data, selected as Working Data in the app.

3. Click OK.

The model structure as well as the parameter values are updated to match that of the selected model.

Clicking Estimate causes the estimation to use the parameters of the initial model as the starting point.

 Note:   When you select an initial model, you can optionally update the estimation algorithm settings to match those used for the initial model by selecting the Inherit the model's algorithm properties option.
5. Keep the default settings in the Nonlinear Models dialog box that specify the model structure and the algorithm, or modify these settings:

 Note:   For more information about available options, click Help in the Nonlinear Models dialog box to open the app help.
What to ConfigureOptions in Nonlinear Models GUIComment
Model orderIn the Regressors tab, edit the No. of Terms corresponding to each input and output channel.Model order na is the output number of terms and nb is the input number of terms.
Input delayIn the Regressors tab, edit the Delay corresponding to an input channel.If you do not know the input delay value, click Infer Input Delay. This action opens the Infer Input Delay dialog box to suggest possible delay values.
RegressorsIn the Regressors tab, click Edit Regressors.This action opens the Model Regressors dialog box. Use this dialog box to create custom regressors or to include specific regressors in the nonlinear block.
Nonlinearity estimatorIn the Model Properties tab.To use all standard and custom regressors in the linear block only, you can exclude the nonlinear block by setting Nonlinearity to None.
Estimation algorithmIn the Estimate tab, click Algorithm Options.

6. To obtain regularized estimates of model parameters, in the Estimate tab, click Estimation Options. Specify the regularization constants in the Regularization_Tradeoff_Constant and Regularization_Weighting fields. To learn more, see Regularized Estimates of Model Parameters.

7. Click Estimate to add this model to the System Identification app.

The Estimate tab displays the estimation progress and results.

8. Validate the model response by selecting the desired plot in the Model Views area of the System Identification app. For more information about validating models, see How to Plot Nonlinear ARX Plots Using the App.

If you get a poor fit, try changing the model structure or algorithm configuration in step 5.

You can export the model to the MATLAB workspace by dragging it to To Workspace in the System Identification app.

How to Estimate Nonlinear ARX Models at the Command Line

Use nlarx to Estimate Nonlinear ARX Models

This example shows how to use nlarx to estimate a nonlinear ARX model for measured input/output data.

Prepare the data for estimation.

load twotankdata z = iddata(y, u, 0.2); ze = z(1:1000); zv = z(1001:3000); 

Estimate several models using different model orders, delays, and nonlinearity settings.

m1 = nlarx(ze,[2 2 1]); m2 = nlarx(ze,[2 2 3]); m3 = nlarx(ze,[2 2 3],wavenet('num',8)); m4 = nlarx(ze,[2 2 3],wavenet('num',8),... 'nlr', [1 2]); 

An alternative way to perform the estimation is to configure the model structure first, and then to estimate this model.

m5 = idnlarx([2 2 3],sigmoidnet('num',14),'nlr',[1 2]); m5 = nlarx(ze,m5); 

Compare the resulting models by plotting the model outputs with the measured output.

compare(zv, m1,m2,m3,m4,m5) 

Using Linear Model for Nonlinear ARX Estimation

You can use an ARX structure polynomial model (idpoly with only A and B as active polynomials) for nonlinear ARX estimation.

Typically, you create a linear ARX model using the arx command. You can provide the linear model only at the command line when constructing (see idnlarx) or estimating (see nlarx) a nonlinear ARX model.

The software uses the linear model for initializing the nonlinear ARX estimation:

• Assigns the linear model orders as initial values of nonlinear model orders (na and nb properties of the idnlarx object) and delays (nk property) to compute standard regressors in the nonlinear ARX model structure.

• Uses the A and B polynomials of the linear model to compute the linear function of the nonlinearity estimators (LinearCoef parameter of the nonlinearity estimator object), except for neural network nonlinearity estimator.

During estimation, the estimation algorithm uses these values to further adjust the nonlinear model to the data. The initialization always provides a better fit to the estimation data than the linear ARX model.

How to Initialize Nonlinear ARX Estimation Using Linear ARX Models

Estimate a nonlinear ARX model initialized using a linear model by typing

m = nlarx(data,LinARXModel)

LinARXModel is an idpoly object of ARX structure. m is an idnlarx object. data is a time-domain iddata object.

By default, the nonlinearity estimator is the wavelet network (wavenet object). This network takes all standard regressors computed using orders and delay of LinARXModel as inputs to its linear and nonlinear functions. The software computes the LinearCoef parameter of the wavenet object using the A and B polynomials of the linear ARX model.

 Tip   When you use the same data set, a nonlinear ARX model initialized using a linear ARX model produces a better fit than the linear ARX model.

Specify a different nonlinearity estimator, for example a sigmoid network:

m = nlarx(data,LinARXModel,'sigmoid')

Create an nlarxOptions option set, setting the Focus property to simulation error minimization:

opt = nlarxOptions('Focus','simulation'); m = nlarx(data,LinARXModel,'sigmoid',opt)

After each estimation, validate the model by comparing the simulated response to the data. To improve the fit of the nonlinear ARX model, adjust various elements of the nonlinear ARX structure. For more information, see Ways to Configure Nonlinear ARX Estimation.

Estimate Nonlinear ARX Models Using Linear ARX Models

This example shows how to estimate nonlinear ARX models by using linear ARX models.

load throttledata.mat 

This command loads the data object ThrottleData into the workspace. The object contains input and output samples collected from an engine throttle system, sampled at a rate of 100 Hz.

A DC motor controls the opening angle of the butterfly valve in the throttle system. A step signal (in volts) drives the DC motor. The output is the angular position (in degrees) of the valve.

Plot the data to view and analyze the data characteristics.

plot(ThrottleData) 

In the normal operating range of 15-90 degrees, the input and output variables have a linear relationship. You use a linear model of low order to model this relationship.

In the throttle system, a hard stop limits the valve position to 90 degrees, and a spring brings the valve to 15 degrees when the DC motor is turned off. These physical components introduce nonlinearities that a linear model cannot capture.

Estimate an ARX model to model the linear behavior of this single-input single-output system in the normal operating range.

Detrend the data because linear models cannot capture offsets.

Tr = getTrend(ThrottleData); Tr.OutputOffset = 15; DetrendedData = detrend(ThrottleData,Tr); 

Estimate a linear ARX model with na=2, nb=1, nk=1.

opt = arxOptions('Focus','simulation'); LinearModel = arx(DetrendedData,[2 1 1],opt); 

Compare the simulated model response with the estimation data.

compare(DetrendedData, LinearModel) 

The linear model captures the rising and settling behavior in the linear operating range but does not account for output saturation at 90 degrees.

Estimate a nonlinear ARX model to model the output saturation.

optNL = nlarxOptions('Focus','simulation'); NonlinearModel = nlarx(ThrottleData,LinearModel,'sigmoidnet',optNL); 

The software uses the orders and delay of the linear model for the orders of the nonlinear model. In addition, the software computes the linear function of sigmoidnet nonlinearity estimator.

Compare the nonlinear model with the estimation data.

compare(ThrottleData, NonlinearModel) 

The model captures the nonlinear effects (output saturation) and improves the overall fit to data.

Validating Nonlinear ARX Models

The Nonlinear ARX plot displays the characteristics of model nonlinearities as a function of one or two regressors. The model nonlinearity (model.Nonlinearity) is a nonlinearity estimator function, such as wavenet, sigmoidnet, treepartition, and uses model regressors as its inputs. The value of the nonlinearity is plotted by projecting its response in 2 or 3-dimensional space. The plot uses one or two regressors as the plot axes for 2- or 3-D plots, respectively and a center point (cross-section location) for the other regressors.

Examining a nonlinear ARX plot can help you gain insight into which regressors have the strongest effect on the model output. Understanding the relative importance of the regressors on the output can help you decide which regressors should be included in the nonlinear function.

Furthermore, you can create several nonlinear models for the same data set using different nonlinearity estimators, such a wavenet network and treepartition, and then compare the nonlinear surfaces of these models. Agreement between nonlinear surfaces increases the confidence that these nonlinear models capture the true dynamics of the system.

In the plot window, you can choose:

• The regressors to use on the plot axes, and specify the center points for the other regressors in the configuration panel. For multi-output models, each output is plotted separately.

• The output to view from the drop-down list located at the top of the plot.

How to Plot Nonlinear ARX Plots Using the App

You can plot linear and nonlinear blocks of nonlinear ARX models.

To create a nonlinear ARX plot in the System Identification app, select the Nonlinear ARX check box in the Model Views area. For general information about creating and working with plots, see Working with Plots.

 Note:   The Nonlinear ARX check box is unavailable if you do not have a nonlinear ARX model in the Model Board.

The following figure shows a sample nonlinear ARX plot.

How to Validate Nonlinear ARX Models at the Command Line

You can use the following approaches to validate nonlinear ARX models at the command line:

Configuring the Nonlinear ARX Plot

To include or exclude a model on the plot, click the corresponding model icon in the System Identification app. Active models display a thick line inside the Model Board icon.

To configure the plot:

1. If your model contains multiple outputs, select the output channel in the Select nonlinearity at output drop-down list. Selecting the output channel displays the nonlinearity values that correspond to this output channel.

2. If the regressor selection options are not visible, click to expand the Nonlinear ARX Model Plot window.

3. Select Regressor 1 from the list of available regressors. In the Range field, enter the range of values to include on the plot for this regressor. The regressor values are plotted on the Reg1 axis.

4. Specify a second regressor for a 3-D plot by selecting one of the following types of options:

• Select Regressor 2 to display three axes. In the Range field, enter the range of values to include on the plot for this regressor. The regressor values are plotted on the Reg2 axis.

• Select <none> in the Regressor 2 list to display only two axes.

5. To fix the values of the regressor that are not displayed, click Fix Values. In the Fix Regressor Values dialog box, double-click the Value cell to edit the constant value of the corresponding regressor. The default values are determined during model estimation. Click OK.

6. Click Apply to update the plot.

1. To change the grid of the regressor space along each axis, Options > Set number of samples, and enter the number of samples to use for each regressor. Click Apply and then Close.

For example, if the number of samples is 20, each regressor variable contains 20 points in its specified range. For a 3-D plots, this results in evaluating the nonlinearity at 20 x 20 = 400 points.

Axis Limits, Legend, and 3-D Rotation

The following table summarizes the commands to modify the appearance of the Nonlinear ARX plot.

Changing Appearance of the Nonlinear ARX Plot

ActionCommand
Change axis limits.

Select Options > Set axis limits to open the Axis Limits dialog box, and edit the limits. Click Apply.

Hide or show the legend.Select Style > Legend. Select this option again to show the legend.
(Three axes only)
Rotate in three dimensions.
 Note:   Available only when you have selected two regressors as independent variables.
Select Style > Rotate 3D and drag the axes on the plot to a new orientation. To disable three-dimensional rotation, select Style > Rotate 3D again.

How the Software Computes Nonlinear ARX Model Output

In most applications, sim and predict are sufficient for computing the simulated and predicted model response, respectively. This advanced topic describes how the software evaluates the output of nonlinearity estimators and uses this output to compute the model response.

Evaluating Nonlinearities

Evaluating the predicted output of a nonlinearity for a specific regressor value x requires that you first extract the nonlinearity F and regressors from the model:

F = m.Nonlinearity; x = getreg(m,'all',data) % computes regressors

Evaluate F(x):

y = evaluate(F,x)

where x is a row vector of regressor values.

You can also evaluate predicted output values at multiple time instants by evaluating F for several regressor vectors simultaneously:

y = evaluate(F,[x1;x2;x3])

Low-Level Simulation and Prediction of Sigmoid Network

This example shows how the software computes the simulated and predicted output of the model as a result of evaluating the output of its nonlinearity estimator for given regressor values.