MATLAB Examples

Expected Shortfall (ES) Backtesting Workflow with No Model Distribution Information

This example shows an expected shortfall (ES) backtesting workflow and the use of ES backtesting tools. The esbacktest class supports two tests -- unconditional normal and unconditional t -- which are based on Acerbi-Szekely's unconditional test statistic (also known as the Acerbi-Szekely second test). These tests use presimulated critical values for the unconditional test statistic, with an assumption of normal distribution for the normal case and a t distribution with 3 degrees of freedom for the t case.

Step 1. Load the ES backtesting data.

Use the ESBacktestData.mat file to load the data into the workspace. This example works with the Returns numeric array. This array represents the equity returns, VaRModel1, VaRModel2, and VaRModel3, and the corresponding VaR data at 97.5% confidence levels, generated with three different models. The expected shortfall data is contained in ESModel1, ESModel2, and ESModel3. The three model distributions used to generate the expected shortfal data in this example are normal (model 1), t with 10 degrees of freedom (model 2), and t with 5 degrees of freedom (model 3). However, this distribution information is not needed in this example because the esbacktest object does not require it.

```load('ESBacktestData') whos ```
``` Name Size Bytes Class Attributes Data 1966x13 224018 timetable Dates 1966x1 15729 datetime ESModel1 1966x1 15728 double ESModel2 1966x1 15728 double ESModel3 1966x1 15728 double Returns 1966x1 15728 double VaRLevel 1x1 8 double VaRModel1 1966x1 15728 double VaRModel2 1966x1 15728 double VaRModel3 1966x1 15728 double ```

Step 2. Generate an ES backtesting plot.

Use the plot function to visualize the ES backtesting data. This type of visualization is a common first step when performing an ES backtesting analysis. For illustration purposes only, visualize the returns, together with VaR and ES, for a particular model.

The resulting plot shows some large violations in 1997, 1998, and 2000. The violations in 1996 look smaller in absolute terms, however relative to the volatility of that period, those violations are also significant. For the unconditional test, the magnitude of the violations and the number of violations make a difference, because the test statistic averages over the expected number of failures. If the expected number is small, but there are several violations, the effective severity for the test is larger. The year 2002 is an example of a year with small, but many VaR failures.

```figure; plot(Dates,Returns,Dates,-VaRModel1,Dates,-ESModel1) legend('Returns','VaR','ES') title('Test Data, Model 1, VaR level 95%') grid on ```

Step 3. Create an esbacktest object.

Create an esbacktest object using esbacktest.

```load ESBacktestData ebt = esbacktest(Returns,[VaRModel1 VaRModel2 VaRModel3],[ESModel1 ESModel2 ESModel3],... 'PortfolioID',"S&P",'VaRID',["Model1","Model2","Model3"],'VaRLevel',VaRLevel) ```
```ebt = esbacktest with properties: PortfolioData: [1966×1 double] VaRData: [1966×3 double] ESData: [1966×3 double] PortfolioID: "S&P" VaRID: ["Model1" "Model2" "Model3"] VaRLevel: [0.9750 0.9750 0.9750] ```

Step 4. Generate the ES summary report.

Generate the ES summary report. The ObservedSeverity column shows the average ratio of loss to VaR on periods when the VaR is violated. The ExpectedSeverity column shows the average ratio of ES to VaR for the VaR violation periods.

```S = summary(ebt); disp(S) ```
``` PortfolioID VaRID VaRLevel ObservedLevel ExpectedSeverity ObservedSeverity Observations Failures Expected Ratio Missing ___________ ________ ________ _____________ ________________ ________________ ____________ ________ ________ ______ _______ "S&P" "Model1" 0.975 0.97101 1.1928 1.4221 1966 57 49.15 1.1597 0 "S&P" "Model2" 0.975 0.97202 1.2652 1.4134 1966 55 49.15 1.119 0 "S&P" "Model3" 0.975 0.97202 1.37 1.4146 1966 55 49.15 1.119 0 ```

Step 5. Run a report for all tests.

Run all tests and generate a report only on the accept or reject results.

```t = runtests(ebt); disp(t) ```
``` PortfolioID VaRID VaRLevel UnconditionalNormal UnconditionalT ___________ ________ ________ ___________________ ______________ "S&P" "Model1" 0.975 reject reject "S&P" "Model2" 0.975 reject accept "S&P" "Model3" 0.975 accept accept ```

Step 6. Run the unconditional normal test.

Run the individual test for the unconditional normal test.

```t = unconditionalNormal(ebt); disp(t) ```
``` PortfolioID VaRID VaRLevel UnconditionalNormal PValue TestStatistic CriticalValue Observations TestLevel ___________ ________ ________ ___________________ _________ _____________ _____________ ____________ _________ "S&P" "Model1" 0.975 reject 0.0054099 -0.38265 -0.2403 1966 0.95 "S&P" "Model2" 0.975 reject 0.044967 -0.25011 -0.2403 1966 0.95 "S&P" "Model3" 0.975 accept 0.149 -0.15551 -0.2403 1966 0.95 ```

Step 7. Run the unconditional t test.

Run the individual test for the unconditional t test.

```t = unconditionalT(ebt); disp(t) ```
``` PortfolioID VaRID VaRLevel UnconditionalT PValue TestStatistic CriticalValue Observations TestLevel ___________ ________ ________ ______________ ________ _____________ _____________ ____________ _________ "S&P" "Model1" 0.975 reject 0.018566 -0.38265 -0.28242 1966 0.95 "S&P" "Model2" 0.975 accept 0.073292 -0.25011 -0.28242 1966 0.95 "S&P" "Model3" 0.975 accept 0.17932 -0.15551 -0.28242 1966 0.95 ```

Step 8. Run ES backtests for a particular year.

Select a particular calendar year and run the tests for that year only by creating an esbacktest object and passing only the data of interest.

```Year = 1996; Ind = year(Dates)==Year; PortID = ['S&P, ' num2str(Year)]; PortfolioData = Returns(Ind); VaRData = [VaRModel1(Ind) VaRModel2(Ind) VaRModel3(Ind)]; ESData = [ESModel1(Ind) ESModel2(Ind) ESModel3(Ind)]; ebt = esbacktest(PortfolioData,VaRData,ESData,... 'PortfolioID',PortID,'VaRID',["Model1","Model2","Model3"],'VaRLevel',VaRLevel); disp(ebt) tt = runtests(ebt); disp(tt) ```
``` esbacktest with properties: PortfolioData: [262×1 double] VaRData: [262×3 double] ESData: [262×3 double] PortfolioID: "S&P, 1996" VaRID: ["Model1" "Model2" "Model3"] VaRLevel: [0.9750 0.9750 0.9750] PortfolioID VaRID VaRLevel UnconditionalNormal UnconditionalT ___________ ________ ________ ___________________ ______________ "S&P, 1996" "Model1" 0.975 reject reject "S&P, 1996" "Model2" 0.975 reject reject "S&P, 1996" "Model3" 0.975 reject accept ```