Documentation

This is machine translation

Translated by Microsoft
Mouseover text to see original. Click the button below to return to the English verison of the page.

Note: This page has been translated by MathWorks. Please click here
To view all translated materals including this page, select Japan from the country navigator on the bottom of this page.

Overview of Expected Shortfall Backtesting

For VaR backtesting, the possibilities every day are two: either there is a VaR failure or not. If the VaR confidence level is 95%, VaR failures should happen approximately 5% of the time. To backtest VaR, you only need to know whether the VaR was exceeded (VaR failure) or not on each day of the test window and the VaR confidence level. Risk Management Toolbox™ VaR backtesting tools support “frequency” (assess the proportion of failures) and “independence” (assess independence across time) tests, and these tests work with the binary sequence of "failure" or "no-failure" results over the test window.

For expected shortfall (ES), the possibilities every day are infinite: The VaR may be exceeded by 1%, or by 10%, or by 150%, and so on. For example, there are three VaR failures in the following example:

On failure days, the VaR is exceeded on average by 39%, but the estimated ES exceeds VaR by an average of 27%. How can you tell if 39% is significantly larger than 27%? Knowing the VaR confidence level is not enough, you must also know how likely are the different exceedances over the VaR according to the VaR model. In other words, you need some distribution information about what happens beyond the VaR according to your model assumptions. For thin-tail VaR models, 39% vs. 27% may be a large difference. However, for a heavy-tail VaR model where a severity of twice the VaR has a non-trivial probability of happening, then 39% vs. 27% over the three failure dates may not be a red flag.

A key difference between VaR backtesting and ES backtesting is that most ES backtesting methods require information about the distribution of the returns on each day, or at least the distribution of the tails beyond the VaR. One exception is the “unconditional” test (see unconditionalNormal and unconditionalT) where you can get approximate test results without providing the distribution information. This is important in practice, because the “unconditional” test is much simpler to use and can be used in principle for any VaR or ES model. The trade-off is that the approximate results may be inaccurate, especially in borderline accept, or reject cases, or for certain types of distributions.

The toolbox supports the following tests for expected shortfall backtesting for table-based tests for the unconditional Acerbi-Szekely test using the esbacktest object:

The toolbox also supports the following simulation-based tests for expected shortfall backtesting using the esbacktestbysim object:

For the simulation-based tests, you must provide the model distribution information as part of the imputs to esbacktestbysim.

Conditional Test

The conditional test statistic is based on the conditional relationship

ESt=Et[Xt|Xt<VaRt]

where

Xt is the portfolio outcome, that is, the portfolio return or portfolio profit and loss for period t.

VaRt is the estimated VaR for period t.

ESt is the estimated expected shortfall for period t.

The number of failures is defined as

NumFailures=t=1NIt

where

N is the number of periods in the test window (t = 1,…,N).

It is the VaR failure indicator on period t with a value of 1 if Xt < -VaR, and 0 otherwise.

The conditional test statistic is defined as

The conditional test has two parts. A VaR backtest must be run for the number of failures (NumFailures), and a standalone conditional test is performed for the conditional test statistic Zcond. The conditional test accepts the model only when both the VaR test and the standalone conditional test accept the model. For more information, see conditional.

Unconditional Test

The unconditional test statistic is based on the unconditional relationship,

ESt=Et[XtItpVaR]

where

Xt is the portfolio outcome, that is, the portfolio return or portfolio profit and loss for period t.

PVaR is the probability of VaR failure defined as 1-VaR level.

ESt is the estimated expected shortfall for period t.

It is the VaR failure indicator on period t with a value of 1 if Xt < -VaR, and 0 otherwise.

The unconditonal test statistic is defined as

The critical values for the unconditional test statistic are stable across a range of distributions, which is the basis for the table-based tests. The esbacktest class runs the unconditional test against precomputed critical values under two distributional assumptions, namely, normal distribution (thin tails, see unconditionalNormal), and t distribution with 3 degrees of freedom (heavy tails, see unconditionalT).

Quantile Test

A sample estimator of the expected shortfall for a sample Y1,…,YN is given by

ES(Y)=1NpVaRi=1NpVaRY[i]

where

N is the number of periods in the test window (t = 1,…,N).

PVaR is the probability of VaR failure defined as 1-VaR level.

Y1,…,YN are the sorted sample values (from smallest to largest), and NpVaR is the largest integer less than or equal to NpVaR.

To compute the quantile test statistic, a sample of size N is created at each time t as follows. First, convert the portfolio outcomes to Xt to ranks U1=P1(X1),...,UN=PN(XN) using the cumulative distribution function Pt. If the distribution assumptions are correct, the rank values U1,…,UN are uniformly distributed in the interval (0,1). Then at each time t:

  • Invert the ranks U = (U1,…,UN) to get N quantiles Pt1(U)=(Pt1(U1),...,Pt1(UN)).

  • Compute the sample estimator ES(Pt1(U)).

  • Compute the expected value of the sample estimator

    where V = (V1,…,VN is a sample of N independent uniform random variables in the interval (0,1). This can be computed anaytically.

The quantile test statistic is defined as

Zquantile=1Nt=1NES(Pt1(U))E[ES(Pt1(V))]+1

The denominator inside the sum can be computed analytically as

E[ES(Pt1(V))]=NNpVaR01I1p(NNpVaR,NpVaR)Pt1(p)dp

where Ix(z,w) is the regularized incomplete beta function. For more information, see betainc and quantile.

References

[1] Basel Committee on Banking Supervision, Supervisory framework for the use of “backtesting” in conjunction with the internal models approach to market risk capital requirements. January 1996, http://www.bis.org/publ/bcbs22.htm.

[2] Acerbi, C., and B. Szekely. Backtesting Expected Shortfall. MSCI Inc. December, 2014.

See Also

| | | | | | | | | | |

Related Topics

Was this topic helpful?