tuff

Time until first failure test for value-at-risk (VaR) backtesting

Syntax

``TestResults = tuff(vbt)``
``TestResults = tuff(vbt,Name,Value)``

Description

example

````TestResults = tuff(vbt)` generates the time until first failure (TUFF) test for value-at-risk (VaR) backtesting.```

example

````TestResults = tuff(vbt,Name,Value)` adds an optional name-value pair argument for `TestLevel`.```

Examples

collapse all

Create a `varbacktest` object.

```load VaRBacktestData vbt = varbacktest(EquityIndex,Normal95)```
```vbt = varbacktest with properties: PortfolioData: [1043x1 double] VaRData: [1043x1 double] PortfolioID: "Portfolio" VaRID: "VaR" VaRLevel: 0.9500 ```

Generate the `tuff` test results.

`TestResults = tuff(vbt)`
```TestResults=1×9 table PortfolioID VaRID VaRLevel TUFF LRatioTUFF PValueTUFF FirstFailure Observations TestLevel ___________ _____ ________ ______ __________ __________ ____________ ____________ _________ "Portfolio" "VaR" 0.95 accept 1.7354 0.18773 58 1043 0.95 ```

Use the `varbacktest` constructor with name-value pair arguments to create a `varbacktest` object.

```load VaRBacktestData vbt = varbacktest(EquityIndex,... [Normal95 Normal99 Historical95 Historical99 EWMA95 EWMA99],... 'PortfolioID','Equity',... 'VaRID',{'Normal95' 'Normal99' 'Historical95' 'Historical99' 'EWMA95' 'EWMA99'},... 'VaRLevel',[0.95 0.99 0.95 0.99 0.95 0.99])```
```vbt = varbacktest with properties: PortfolioData: [1043x1 double] VaRData: [1043x6 double] PortfolioID: "Equity" VaRID: ["Normal95" "Normal99" "Historical95" ... ] VaRLevel: [0.9500 0.9900 0.9500 0.9900 0.9500 0.9900] ```

Generate the `tuff` test results using the `TestLevel` optional input.

`TestResults = tuff(vbt,'TestLevel',0.90)`
```TestResults=6×9 table PortfolioID VaRID VaRLevel TUFF LRatioTUFF PValueTUFF FirstFailure Observations TestLevel ___________ ______________ ________ ______ __________ __________ ____________ ____________ _________ "Equity" "Normal95" 0.95 accept 1.7354 0.18773 58 1043 0.9 "Equity" "Normal99" 0.99 accept 0.36686 0.54472 173 1043 0.9 "Equity" "Historical95" 0.95 accept 1.5348 0.2154 55 1043 0.9 "Equity" "Historical99" 0.99 accept 0.36686 0.54472 173 1043 0.9 "Equity" "EWMA95" 0.95 accept 0.13304 0.7153 28 1043 0.9 "Equity" "EWMA99" 0.99 accept 0.14596 0.70243 143 1043 0.9 ```

Input Arguments

collapse all

`varbacktest` (`vbt`) object, contains a copy of the given data (the `PortfolioData` and `VarData` properties) and all combinations of portfolio ID, VaR ID, and VaR levels to be tested. For more information on creating a `varbacktest` object, see `varbacktest`.

Name-Value Arguments

Specify optional pairs of arguments as `Name1=Value1,...,NameN=ValueN`, where `Name` is the argument name and `Value` is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose `Name` in quotes.

Example: ```TestResults = tuff(vbt,'TestLevel',0.99)```

Test confidence level, specified as the comma-separated pair consisting of `'TestLevel'` and a numeric between `0` and `1`.

Data Types: `double`

Output Arguments

collapse all

`tuff` test results, returned as a table where the rows correspond to all combinations of portfolio ID, VaR ID, and VaR levels to be tested. The columns correspond to the following information:

• `'PortfolioID'` — Portfolio ID for the given data

• `'VaRID'` — VaR ID for each of the VaR data columns provided

• `'VaRLevel'` — VaR level for the corresponding VaR data column

• `'TUFF'` — Categorical array with the categories `accept` and `reject` that indicate the result of the `tuff` test

• `'LRatioTUFF'` — Likelihood ratio of the `tuff` test

• `'PValueTUFF'` — P-value of the `tuff` test

• `'FirstFailure'` — Number of periods until the first failure

• `'Observations'` — Number of observations

• `'TestLevel'` — Test confidence level

Note

For `tuff` test results, the terms `accept` and `reject` are used for convenience, technically a `tuff` test does not accept a model. Rather, the test fails to reject it.

collapse all

Time Until First Failure (TUFF) Test

The `tuff` function performs Kupiec's time until first failure test.

The TUFF test is a likelihood ratio test proposed by Kupiec (1995) to assess if the number of periods until the first failure is consistent with the VaR confidence level.

Algorithms

The likelihood ratio (test statistic) of the `tuff` test is given by

`$LRatioTUFF=-2\mathrm{log}\left(\frac{pVaR{\left(1-pVaR\right)}^{n-1}}{\left(\frac{1}{n}\right){\left(1-\frac{1}{n}\right)}^{n-1}}\right)=-2\left(\mathrm{log}\left(pVaR\right)+\left(n-1\right)\mathrm{log}\left(1-pVaR\right)+n\mathrm{log}\left(n\right)-\left(n-1\right)\mathrm{log}\left(n-1\right)\right)$`

where n is the number of periods until the first failure and pVaR = 1 - VaRLevel. By the properties of the logarithm (if n = `1`),

`$LRatioTUFF=-2\mathrm{log}\left(pVaR\right)$`

This is asymptotically distributed as a chi-square distribution with 1 degree of freedom.

The p-value of the `tuff` test is the probability that a chi-square distribution with 1 degree of freedom exceeds the likelihood ratio LRatioTUFF

`$PValueTUFF=1-F\left(LRatioTUFF\right)$`

where `F` is the cumulative distribution of a chi-square variable with 1 degree of freedom.

The result of the test is to accept if

`$F\left(LRatioTUFF\right)`

and reject otherwise, where F is the cumulative distribution of a chi-square variable with 1 degree of freedom.

If the sample has no failures, the test statistic is not defined. However, there are two cases distinguished here:

• If the number of observations is large enough that no matter when the first failure occurred it would be too late to pass the test, then the model is rejected. Technically, this happens if the number of observations N is larger than `1`/pVaR (large enough relative to the VaR confidence level) and if the test fails when n = N + `1` (the earliest observation for the first VaR failure). In this case, the likelihood ratio is reported for n = N + `1`, and the corresponding p-value.

• In all other cases, it is not possible to tell with certainty whether the result of the test would eventually be to accept or reject the model. There are ranges of possible first failure values that would result in accepting or rejecting the model. In these cases, the `tuff` function accepts the model and reports undefined (`NaN`) values for the likelihood ratio and p-value.

References

[1] Kupiec, P. "Techniques for Verifying the Accuracy of Risk Management Models." Journal of Derivatives. Vol. 3, 1995, pp. 73 – 84.

Version History

Introduced in R2016b