Handling Offsets and Trends in Data :: Data Import and Processing (System Identification Toolbox™)

Learn more about System Identification Toolbox

Handling Offsets and Trends in Data

On this page…
When to Detrend Data Alternatives for Detrending Data in GUI or at the Command-Line How to Detrend Data Using the GUI How to Detrend Data at the Command Line Next Steps After Detrending

On this page…

When to Detrend Data

Alternatives for Detrending Data in GUI or at the Command-Line

How to Detrend Data Using the GUI

How to Detrend Data at the Command Line

Next Steps After Detrending

When to Detrend Data

Detrending is removing means, offsets, or linear trends from regularly sampled time-domain input-output data signals. This data processing operation helps you estimate more accurate linear models because linear models cannot capture arbitrary differences between the input and output signal levels. The linear models you estimate from detrended data describe the relationship between the change in input signals and the change in output signals.

For steady-state data, you should remove mean values and linear trends from both input and output signals.

For transient data, you should remove physical-equilibrium offsets measured prior to the excitation input signal.

Remove one linear trend or several piecewise linear trends when the levels drift during the experiment. Signal drift is considered a low-frequency disturbance and can result in unstable models.

You should not detrend data before model estimation when you want:

Linear models that capture offsets essential for describing important system dynamics. For example, when a model contains integration behavior, you could estimate a low-order transfer function (process model) from nondetrended data. For more information, see Identifying Low-Order Transfer Functions (Process Models).
Nonlinear black-box models, such as nonlinear ARX or Hammerstein-Wiener models. For more information, see Nonlinear Black-Box Model Identification.
Nonlinear ODE parameters (nonlinear grey-box models). For more information, see Estimating Nonlinear Grey-Box Models.

To simulate or predict the linear model response at the system operating conditions, you can restore the removed trend to the simulated or predicted model output using the retrend command.

For more information about handling drifts in the data, see the chapter on preprocessing data in System Identification: Theory for the User, Second Edition, by Lennart Ljung, Prentice Hall PTR, 1999.

Alternatives for Detrending Data in GUI or at the Command-Line

You can detrend data using the System Identification Tool GUI and at the command line using the detrend command.

Both the GUI and the command line let you subtract the mean values and one linear trend from steady-state time-domain signals.

However, the detrend command provides the following additional functionality (not available in the GUI):

Subtracting piecewise linear trends at specified breakpoints. A breakpoint is a time value that defines the discontinuities between successive linear trends.
Subtracting arbitrary offsets and linear trends from transient data signals.
Saving trend information to a variable so that you can apply it to multiple data sets.

To learn how to detrend data, see:

How to Detrend Data Using the GUI

Before you can perform this task, you must have regularly-sampled, steady-state time-domain data imported into the System Identification Tool GUI. See Importing Time-Domain Data into the GUI). For transient data, see How to Detrend Data at the Command Line.

Tip You can use the shortcut Preprocess > Quick start to perform several operations: remove the mean value from each signal, split data into two halves, specify the first half as model estimation data (or Working Data), and specify the second half as model Validation Data.

In the System Identification Tool, drag the data set you want to detrend to the Working Data rectangle.
Detrend the data.
- To remove linear trends, select Preprocess > Remove trends.
- To remove mean values from each input and output data signal, select Preprocess > Remove means.

How to Detrend Data at the Command Line

Detrending Steady-State Data

Before you can perform this task, you must have time-domain data as an iddata object. See Representing Time- and Frequency-Domain Data Using iddata Objects.

Note If you plan to estimate models from this data, your data must be regularly sampled.

Use the detrend command to remove the signal means or linear trends:

[data_d,T]=detrend(data,Type)

where data is the data to be detrended. The second input argument Type=0 removes signal means or Type=1 removes linear trends. data_d is the detrended data. T is a TrendInfo object that stores the values of the subtracted offsets and slopes of the removed trends.

Detrending Transient Data

Before you can perform this task, you must have

Time-domain data as an iddata object. See Representing Time- and Frequency-Domain Data Using iddata Objects.
Values of the offsets you want to remove from the input and output data. If you do not know these values, visually inspect a time plot of your data. For more information, see How to Plot Data at the Command Line.

Create a default object for storing input-output offsets that you want to remove from the data.
```
T = getTrend(data)
```
where T is a TrendInfo object.
Assign offset values to T.
```
T.InputOffset=I_value;
T.OutputOffset=O_value;
```
where I_value is the input offset value, and O_value is the input offset value.
Remove the specified offsets from data.
```
data_d = detrend(data,T)
```
where the second input argument T stores the offset values as its properties.