Note: This page has been translated by MathWorks. Please click here

To view all translated materals including this page, select Japan from the country navigator on the bottom of this page.

To view all translated materals including this page, select Japan from the country navigator on the bottom of this page.

For VaR backtesting, the possibilities every day are two: either there is a VaR failure or not. If the VaR confidence level is 95%, VaR failures should happen approximately 5% of the time. To backtest VaR, you only need to know whether the VaR was exceeded (VaR failure) or not on each day of the test window and the VaR confidence level. Risk Management Toolbox™ VaR backtesting tools support “frequency” (assess the proportion of failures) and “independence” (assess independence across time) tests, and these tests work with the binary sequence of "failure" or "no-failure" results over the test window.

For expected shortfall (ES), the possibilities every day are infinite: The VaR may be exceeded by 1%, or by 10%, or by 150%, and so on. For example, there are three VaR failures in the following example:

On failure days, the VaR is exceeded on average by 39%, but the estimated ES exceeds VaR by an average of 27%. How can you tell if 39% is significantly larger than 27%? Knowing the VaR confidence level is not enough, you must also know how likely are the different exceedances over the VaR according to the VaR model. In other words, you need some distribution information about what happens beyond the VaR according to your model assumptions. For thin-tail VaR models, 39% vs. 27% may be a large difference. However, for a heavy-tail VaR model where a severity of twice the VaR has a non-trivial probability of happening, then 39% vs. 27% over the three failure dates may not be a red flag.

A key difference between VaR backtesting and ES backtesting is that most ES
backtesting methods require information about the distribution of the returns on each
day, or at least the distribution of the tails beyond the VaR. One exception is the
“unconditional” test (see `unconditionalNormal`

and `unconditionalT`

) where you
can get approximate test results without providing the distribution information. This is
important in practice, because the “unconditional” test is much simpler to use and can
be used in principle for any VaR or ES model. The trade-off is that the approximate
results may be inaccurate, especially in borderline accept, or reject cases, or for
certain types of distributions.

The toolbox supports the following tests for expected shortfall backtesting for
table-based tests for the unconditional Acerbi-Szekely test using the `esbacktest`

object:

The toolbox also supports the following simulation-based tests for expected shortfall
backtesting using the `esbacktestbysim`

object:

For the simulation-based tests, you must provide the model distribution information as
part of the imputs to `esbacktestbysim`

.

The *conditional * test statistic is based on the conditional relationship

$$E{S}_{t}=-{E}_{t}\left[{X}_{t}|{X}_{t}<-Va{R}_{t}\right]$$

where

`X`

_{t} is the portfolio outcome, that is,
the portfolio return or portfolio profit and loss for period
*t*.

`VaR`

_{t} is the estimated VaR for period
*t*.

`ES`

_{t} is the estimated expected shortfall
for period *t*.

The number of failures is defined as

$$NumFailures={\displaystyle \sum _{t=1}^{N}{I}_{t}}$$

where

`N`

is the number of periods in the test window
(*t* = `1`

,…,`N`

).

`I`

_{t} is the VaR failure indicator on
period *t* with a value of 1 if
`X`

_{t} < -VaR, and 0 otherwise.

The conditional test statistic is defined as

The conditional test has two parts. A VaR backtest must be run for the number of
failures (`NumFailures`

), and a standalone conditional test is
performed for the conditional test statistic
`Z`

_{cond}. The conditional test accepts
the model only when both the VaR test and the standalone conditional test accept the
model. For more information, see `conditional`

.

The *unconditional * test statistic is based on the
unconditional relationship,

$$E{S}_{t}=-{E}_{t}\left[\frac{{X}_{t}{I}_{t}}{{p}_{VaR}}\right]$$

where

`X`

_{t} is the portfolio outcome, that is,
the portfolio return or portfolio profit and loss for period
*t*.

`P`

_{VaR} is the probability of VaR failure
defined as 1-VaR level.

`ES`

_{t} is the estimated expected shortfall
for period *t*.

`I`

_{t} is the VaR failure indicator on
period *t* with a value of 1 if
`X`

_{t} < -VaR, and 0 otherwise.

The unconditonal test statistic is defined as

The critical values for the unconditional test statistic are stable across a range
of distributions, which is the basis for the table-based tests. The `esbacktest`

class runs the
unconditional test against precomputed critical values under two distributional
assumptions, namely, normal distribution (thin tails, see `unconditionalNormal`

), and *t* distribution with 3
degrees of freedom (heavy tails, see `unconditionalT`

).

A sample estimator of the expected shortfall for a sample
`Y`

_{1},…,`Y`

_{N}
is given by

$$\stackrel{\u2322}{ES}(Y)=-\frac{1}{\lfloor N{p}_{VaR}\rfloor}{\displaystyle \sum _{i=1}^{\lfloor N{p}_{VaR}\rfloor}{Y}_{\left[i\right]}}$$

where

`N`

is the number of periods in the test window
(*t* = `1`

,…,`N`

).

`P`

_{VaR} is the probability of VaR failure
defined as 1-VaR level.

`Y`

_{1},…,`Y`

_{N}
are the sorted sample values (from smallest to largest), and $$\lfloor N{p}_{VaR}\rfloor $$ is the largest integer less than or equal to
`Np`

_{VaR}.

To compute the quantile test statistic, a sample of size `N`

is
created at each time *t* as follows. First, convert the portfolio
outcomes to `X`

_{t} to ranks $${U}_{1}={P}_{1}({X}_{1}),\mathrm{...},{U}_{N}={P}_{N}({X}_{N})$$ using the cumulative distribution function
`P`

_{t}. If the distribution assumptions
are correct, the rank values
`U`

_{1},…,`U`

_{N}
are uniformly distributed in the interval (0,1). Then at each time
*t*:

Invert the ranks U = (

`U`

_{1},…,`U`

_{N}) to get`N`

quantiles $${P}_{t}^{-1}(U)=({P}_{t}^{-1}({U}_{1}),\mathrm{...},{P}_{t}^{-1}({U}_{N}))$$.Compute the sample estimator $$\stackrel{\u2322}{ES}({P}_{t}^{-1}(U))$$.

Compute the expected value of the sample estimator

where

`V`

= (`V`

_{1},…,`V`

_{N}is a sample of`N`

independent uniform random variables in the interval (0,1). This can be computed anaytically.

The quantile test statistic is defined as

$${Z}_{quantile}=-\frac{1}{N}{\displaystyle \sum _{t=1}^{N}\frac{\stackrel{\u2322}{ES}({P}_{t}^{-1}(U))}{E[\stackrel{\u2322}{ES}({P}_{t}^{-1}(V))]}+1}$$

The denominator inside the sum can be computed analytically as

$$E[\stackrel{\u2322}{ES}({P}_{t}^{-1}(V))]=-\frac{N}{\lfloor {N}_{pVaR}\rfloor}{\displaystyle {\int}_{0}^{1}{I}_{1-p}(N-\lfloor {N}_{pVaR}\rfloor},\lfloor {N}_{pVaR}\rfloor ){P}_{t}^{-1}(p)dp$$

`I`

`z`

,`w`

)
is the regularized incomplete beta function. For more information, see `betainc`

and `quantile`

.[1] Basel Committee on Banking Supervision, *Supervisory framework
for the use of “backtesting” in conjunction with the internal
models approach to market risk capital requirements.* January 1996,
http://www.bis.org/publ/bcbs22.htm.

[2] Acerbi, C., and B. Szekely. *Backtesting Expected
Shortfall.* MSCI Inc. December, 2014.

`conditional`

| `esbacktest`

| `esbacktestbysim`

| `quantile`

| `runtests`

| `runtests`

| `simulate`

| `summary`

| `summary`

| `unconditional`

| `unconditionalNormal`

| `unconditionalT`

Was this topic helpful?