A novel robust method for estimating the covariance matrix of financial returns with applications to risk management

Abstract

This study introduces the dynamic Gerber model (DGC) and evaluates its performance in the prediction of Value at Risk (VaR) and Expected Shortfall (ES) compared to alternative parametric, non-parametric and semi-parametric methods for estimating the covariance matrix of returns. Based on ES backtests, the DGC method produces, overall, accurate ES forecasts. Furthermore, we use the Model Confidence Set procedure to identify the superior set of models (SSM). For all the portfolios and VaR/ES confidence levels we consider, the DGC is found to belong to the SSM.

Introduction

In a globalized economy, turbulence in financial markets has become more frequent. Recent events like the COVID-19 pandemic, the surge in inflation, and the Russian-Ukrainian conflict have all triggered considerable losses for investors worldwide. Providing methods that accurately measure financial market risk is therefore an increasingly crucial task. Over the past two decades, Value at Risk (VaR)—a measure related to the quantile of the conditional portfolio return distribution—has become the standard measure of market risk. While VaR is a measure of risk easy to understand for laymen, it has a number of shortcomings. First, it provides no information about the magnitude of the losses that exceed the quantile (see for instance Christoffersen 2012, Ch. 2). Second, VaR is not a coherent risk measure (see Artzner et al. 1999, for the properties of coherent risk measures). In particular, VaR is not subadditive, implying that the portfolio VaR could be larger than the sum of the VaRs of its components (see for instance Dhaene et al. 2006). VaR has been used in the context of credit scoring by Xu et al. (2024), who introduce the worst-case conditional value-at-risk metric to measure the loss incurred from employing a classification model in credit scoring under the deterioration of cost parameters. Unlike VaR, Expected Shortfall (ES)—defined as the conditional expectation of exceedances beyond the VaR—is a coherent risk measure (Acerbi and Tasche 2002). Several methods to estimate risk measures are available (see Nieto and Ruiz 2016, forasurveyofestimationmethodsconcerningVaR). A well-known non-parametric method is historical simulation (HS), which estimates VaR and ES using the empirical counterparts under the assumption of i.i.d. portfolio returns. Parametric methods typically involve estimating GARCH-type models; a few examples include Brooks and Persand (2003), Chu et al. (2017), and Long et al. (2020). Common alternatives are models based on extreme value theory, recent examples of which are represented by Bekiros et al. (2019) and Echaust and Just (2020). A semi-parametric approach to VaR/ES estimation combining a parametric model and HS is the filtered historical simulation (FHS) method of Barone-Adesi et al. (2002). The FHS method exploits the idea of using the empirical quantile of random draws obtained with replacement from the standardized residuals of a parametric model. VaR is then obtained by rescaling this quantile using the predicted volatility from the parametric model. When the focus is only on forecasting VaR, a popular approach is to directly model the conditional quantile using quantile regression, as in the conditional autoregressive VaR model of Engle and Manganelli (2004). Alternative semi-parametric methods that jointly estimate VaR and ES models include Patton et al. (2019) and Taylor (2019). Recent works are concerned with the VaR/ES risk measures in the context of financial (James et al. 2023), insurance (Fan et al. 2023), or commodity markets (Vancsura et al. 2023).

As in Lopez and Walter (2000) and Skintzi and Xanthopoulos-Sisinis (2007), who discuss the importance of covariance matrix forecasting for risk management, in this study we consider alternative methods to estimate the covariance matrix of returns. In particular, beside HS, we compare—in terms of accuracy in forecasting VaR or jointly VaR and ES—a number of different methods (parametric, non-parametric, and semi-parametric) for obtaining the correlation and hence the covariance matrix. In addition to methods employing the standard Pearson correlation, we rely on methods based on a static or dynamic version of the robust correlation proposed by Gerber et al. (2022). The measure is an extension of Kendall’s Tau robust measure of pairwise movements of two series of returns. In particular, it is built based on the proportions of co-movements in the series of interest (i.e., on how many times the series simultaneously pierce some pre-specified thresholds). We contribute to the literature by introducing a dynamic version of the Gerber correlation matrix that we call the dynamic Gerber model (DGC). Like dynamic conditional correlation (DCC) models, the DGC model relies in the first stage of the estimation on univariate GARCH models for the volatility of each asset return. In the second stage, based on the marginally standardized residuals, a dynamic Gerber matrix is established. The evolution of such a matrix depends on six parameters only, even when the number of assets in the portfolio, k, is large. Like DCC models, but contrary to BEKK models, our DGC model cannot be used to directly capture volatility spillovers between different markets (like for instance in Kondoz et al. 2019). However, our model can easily be estimated when k is large, whereas BEKK models become problematic as the number of parameters grows with the square of k. Yet another way of obtaining a time-varying correlation matrix would be one based on Markov switching models¹. Again, unless parsimonious specifications are considered that rarely are effective in practical situations, such an approach could suffer from a curse of dimensionality. An alternative study that considers dynamic dependence via threshold exceedances is the one of Gong and Huser (2022). To have time-varying copula models², the authors estimate a family of parameters for each time point and replace the likelihood function by a family of weighted local likelihoods to be maximized. However, in their approach, the choice of the bandwidth associated with the weight functions (kernel) is crucial. Small bandwidths lead to parameter estimates that are very variable, while large bandwidths lead to smooth estimates with low variability over time. Our approach, instead, does not require specifying a bandwidth. Furthermore, the approach presented in Gong and Huser (2022) is bivariate, whereas our methodology is valid for a generic number of assets.

A study concerning robust forecasting of conditional correlations is that of Boudt et al. (2013), where an extension of the DCC model to reduce the biases in the volatility and correlation dynamics caused by large one-off events is presented. The authors apply their model to forecast the covariance matrix of the daily EUR/USD and Yen/USD return series, and they find that their model guarantees more precise out-of-sample covariance forecasts than the DCC model. In the same line of research, Jarjour and Chan (2020) introduce the concept of angular correlation for estimating the instantaneous correlation matrix and then generalize the DCC model to the dynamic conditional angular correlation (DCAC) model. They illustrate the better performance of the DCAC model compared to the DCC model in portfolio construction. A further study of robust forecasting of conditional correlations is conducted by Fiszeder et al. (2023), who suggest a new version of the DCC model based on daily opening, high, low, and closing prices. Using two different datasets, five exchange-traded funds and five currencies, they show that their model significantly outperforms the DCC model when forecasting conditional covariance matrices. Our paper can be placed in this context of empirical research, where more robust estimates of the conditional correlations than those obtained using the DCC model are proposed and tested.

In our empirical application, we consider three different portfolios consisting of developed equities (S &P 500 index), emerging equities (MXEF index), bonds (LBUSTRUU index), and gold. For different probability levels, we derive out-of-sample VaR and ES for the three portfolios. We use a recently proposed procedure to backtest ES and the Model Confidence Set (MCS) procedure to identify the superior set of models (SSM). For both the risk measures considered in the study, we find that models based on the DGC approach are part of the SSM across portfolios and confidence levels. A practical implication of our study is that the proposed DGC approach should be taken into consideration by risk managers, investors, and regulators when evaluating the risk of asset portfolios. The DGC model is indeed never eliminated from the SSM and, in the case of joint VaR and ES forecasting, is the only model not to be rejected by the MCS procedure. A model that offers a more robust estimate of conditional correlations represents a valuable tool. Indeed, as some empirical studies show (see, e.g., Jiang et al. 2017 and Athari and Hung, 2022), adverse market conditions usually make assets more interdependent, and a model that accurately describes these dynamics, like the DGC model, could ensure more reliable risk estimates. Potentially, our method can also be employed in the context of bankruptcy prediction (Kou et al. 2021) in the case of loan portfolios as an alternative to Pearson correlation-based methods (Düllmann et al. 2007, 2010). Indeed, our method could be applied to the time series of Moody’s KMV asset values as the correlation of unobservable asset returns is a key component for the measurement of portfolio risk. The remainder of the paper is organized as follows. Section “Methodology” presents the methods used in the estimations of VaR and ES, introduces the DGC model, and describes the MCS procedure. Section “Empirical analysis” presents the data used in this study and the results of the empirical analysis. Section “Conclusions” concludes.

Methodology

In this section, we first present the static version of the robust measure of correlation introduced by Gerber et al. (2022). Next, we discuss DCC models and introduce the novel DGC method. Finally, we explain how the risk measures of interest are derived and how to evaluate their predictions via the MCS procedure.

The Gerber statistic

We denote by $r_{i,t}$ and $r_{j,t}$ the returns of asset i and asset j at time t. The Gerber statistic, introduced by Gerber et al. (2022), is a robust measure of pairwise movements of the two series of returns defined as

$$\begin{aligned} g(i,j)=\frac{ n_{ij}^c - n_{ij}^d }{ n_{ij}^c + n_{ij}^d }, \end{aligned}$$

(1)

where

$$\begin{aligned} n_{ij}^c= & {} \sum _{t=1}^T I\left( r_{i,t}\ge Q_i \right) I\left( r_{j,t}\ge Q_j \right) + \sum _{t=1}^T I\left( r_{i,t}\le -Q_i \right) I\left( r_{j,t}\le -Q_j \right) =n_{ij}^{UU}+n_{ij}^{DD} \\ n_{ij}^d= & {} \sum _{t=1}^T I\left( r_{i,t}\ge Q_i \right) I\left( r_{j,t}\le -Q_j \right) + \sum _{t=1}^T I\left( r_{i,t}\le -Q_i \right) I\left( r_{j,t}\ge Q_j \right) =n_{ij}^{UD}+n_{ij}^{DU}. \end{aligned}$$

Here, T is the number of observations, $Q_i$ and $Q_j$ are thresholds, and I(A) denotes the indicator function for the event A. Hence, $n_{ij}^c$ denotes the number of concordant pairs (i.e., the number of times both returns pierce their thresholds while moving in the same direction). Indeed, $n_{ij}^c$ is equal to the sum of $n_{ij}^{UU}$—the number of pairs for which both returns are larger than their threshold—and $n_{ij}^{DD}$, the number of pairs for which both returns are smaller than their threshold times minus one. Conversely, $n_{ij}^d=n_{ij}^{UD}+n_{ij}^{DU}$ represents the number of discordant pairs in the sample, that is, the number of times both returns pierce their thresholds while moving in the opposite direction.

When several pairs of returns are involved, using (1) to construct a correlation matrix may lead to covariance matrices that are not positive semidefinite. Therefore, Gerber et al. (2022) define $n_{ij}^{NN}= \sum _{t=1}^T I\left( |r_{i,t}|\le Q_i \right) I\left( |r_{j,t}|\le Q_j \right)$ and propose replacing (1) with

$$\begin{aligned} g(i,j) =\frac{n_{ij}^c - n_{ij}^d }{ T- n_{ij}^{NN}}, \end{aligned}$$

(2)

which instead yields positive semidefinite covariance matrices.

We now consider the case of k different assets and denote by ${\varvec{U}}$ and ${\varvec{D}}$ the two $T\times k$ matrices with generic element $u_{t,j}=I\left( r_{j,t}\ge Q_j \right)$ and $d_{t,j}=I\left( r_{j,t}\le -Q_j \right)$, respectively, $t=1,\ldots ,T$ and $j=1,\ldots ,k$. The $k \times k$ matrix ${\varvec{G}}$ with element in position (i, j) given by the Gerber correlation (1) is then given by

$$\begin{aligned} {\varvec{G}}=\left( {\varvec{U}}'{\varvec{U}} + {\varvec{D}}'{\varvec{D}} - {\varvec{U}}'{\varvec{D}} -{\varvec{D}}'{\varvec{U}} \right) \oslash \left( {\varvec{U}}'{\varvec{U}} + {\varvec{D}}'{\varvec{D}} + {\varvec{U}}'{\varvec{D}} +{\varvec{D}}'{\varvec{U}} \right) , \end{aligned}$$

(3)

where $\oslash$ means elementwise division.

With the further definition of the $T\times k$ matrix ${\varvec{N}}$ with generic element $n_{t,j}=I\left( |r_{j,t}|\le Q_j \right)$, $t=1,\ldots ,T$ and $j=1,\ldots ,k$, we can express the $k \times k$ matrix ${\varvec{G}}$ with generic element g(i, j) of equation (2) as

$$\begin{aligned} {\varvec{G}}=\left( {\varvec{ \mathcal {C}}} - {\varvec{ \mathcal {D}}} \right) \oslash \left( {\varvec{1}}-{\varvec{ \mathcal {N}}} \right) , \end{aligned}$$

(4)

where ${\varvec{ \mathcal {C}}} = \left( {\varvec{U}}'{\varvec{U}} + {\varvec{D}}'{\varvec{D}} \right) / T$, ${\varvec{ \mathcal {D}}}= \left( {\varvec{U}}'{\varvec{D}} + {\varvec{D}}'{\varvec{U}} \right) /T$, ${\varvec{ \mathcal {N}}}= {\varvec{N}}'{\varvec{N}} / T$, and ${\varvec{1}}$ denotes the $k \times k$ matrix with all entries 1.

DCC models

DCC models assume that the multivariate time series of returns ${\varvec{r}}_t=(r_{1,t},\ldots ,r_{k,t})'$ is described by

$$\begin{aligned} {\varvec{r}}_t = {\varvec{\mu }}_t+ {\varvec{\Sigma }}_t^{1/2} {\varvec{z}}_t, \end{aligned}$$

(5)

where $\{{\varvec{z}}_t\}_t$ is a sequence of independent and identically distributed random vectors such that $\mathbb {E}({\varvec{z}}_t) = 0$ and $\textrm{cov}({\varvec{z}}_t) = {\varvec{I}}_k$, with ${\varvec{\Sigma }}_t^{1/2}$ denoting the positive-definite square-root matrix of the conditional covariance matrix of the returns, ${\varvec{\Sigma }}_t$.

In DCC models (Engle 2002), the ${\varvec{\Sigma }}_t$ matrix, whose generic element is $\sigma _{ij,t}$, is decomposed as

$$\begin{aligned} {\varvec{\Sigma }}_t= {{\,\textrm{diag}\,}}( \sigma _{11,t}^{1/2}, \ldots , \sigma _{kk,t}^{1/2} ) {\varvec{R}}_t {{\,\textrm{diag}\,}}( \sigma _{11,t}^{1/2}, \ldots , \sigma _{kk,t}^{1/2} ) , \end{aligned}$$

(6)

where ${\varvec{R}}_t$ is the positive definite conditional correlation matrix. In this way, if $\rho _{ij,t}$ is the element of position (i, j) of the correlation matrix ${\varvec{R}}_t$, then the corresponding element of ${\varvec{\Sigma }}_t$ is found to be $\rho _{ij,t}\sqrt{\sigma _{ii,t} \sigma _{jj,t}}$.

Let $\hat{u}_{i,t}=r_{i,t}-\mu _{i,t}$ be the time t residual from the mean equation of asset i. We denote by $\hat{{\varvec{\eta }}}_t=(\hat{\eta }_{1,t},\ldots ,\hat{\eta }_{k,t})'$ the marginally standardized innovation vector:

$$\begin{aligned} \hat{\eta }_{i,t}=\frac{\hat{u}_{i,t}}{\sqrt{\sigma _{ii,t}}} \quad i=1,\ldots ,k. \end{aligned}$$

In this way, ${\varvec{R}}_t$ is the covariance matrix of ${\varvec{\eta }}_t$. Engle (2002) proposes modelling the correlation matrix as

$$\begin{aligned} \begin{aligned} {\varvec{S}}_t&= (1-a-b) {\varvec{\bar{S}}} + a \hat{{\varvec{\eta }}}_{t-1} \hat{{\varvec{\eta }}}_{t-1}' + b {\varvec{S}}_{t-1}\\ {\varvec{R}}_t&= {\varvec{J}}_t {\varvec{S}}_t {\varvec{J}}_t, \end{aligned} \end{aligned}$$

(7)

where ${\varvec{\bar{S}}}$ is the unconditional covariance matrix of $\hat{{\varvec{\eta }}}_t$, and ${\varvec{J}}_t = {{\,\textrm{diag}\,}}(s_{11,t}^{-1/2},\ldots ,s_{kk,t}^{-1/2})$, with $s_{ii,t}$ denoting the element of position (i, i) of ${\varvec{S}}_t$. In the first stage of the DCC model estimation, k univariate GARCH models are independently estimated for each of the return series. In the second stage, the marginally standardized innovation vectors are derived and the parameters a and b in (7) are estimated.

A dynamic Gerber model

We propose a dynamic Gerber correlation (DGC) model based on a two-stage estimation like DCC models. In the first stage, the volatility of each asset is independently modelled via a GARCH model. Hence, as in DCC models, in the first stage k univariate GARCH models are independently estimated. In the second stage, based on the the marginally standardized innovation vectors, a dynamic is given to the Gerber correlation matrix (see Eq. (4)) rather than the Pearson correlation matrix. To be more precise, we identify the thresholds $Q_i^{\eta }$ for each of the k time-series of marginally standardized innovation $\hat{\eta }_{i}$ ($i=1,\ldots ,k)$ and consider the following dynamics:

$$\begin{aligned} \begin{aligned} {\varvec{ \mathcal {C} }}_t&= (1-a_C-b_C) {\varvec{\overline{\mathcal {C}}}} + a_C \hat{{\varvec{\mathcal {I}}}}_{C,t-1} + b_C {\varvec{ \mathcal {C} }}_{t-1}\\ {\varvec{ \mathcal {D} }}_t&= (1-a_D-b_D) {\varvec{\overline{\mathcal {D}}}} + a_D \hat{{\varvec{\mathcal {I}}}}_{D,t-1} + b_D {\varvec{ \mathcal {D} }}_{t-1}\\ {\varvec{ \mathcal {N} }}_t&= (1-a_N-b_N) {\varvec{\overline{\mathcal {N}}}} + a_N \hat{{\varvec{\mathcal {I}}}}_{N,t-1} + b_N{\varvec{ \mathcal {N} }}_{t-1}\\ {\varvec{G}}_t&= \left( {\varvec{ \mathcal {C} }}_t - {\varvec{ \mathcal {D} }}_t \right) \oslash \left( {\varvec{1}}- {\varvec{ \mathcal {N} }}_t \right) . \end{aligned} \end{aligned}$$

(8)

The $\hat{{\varvec{\mathcal {I}}}}$ matrices appearing in (8) are obtained as follows:

$$\begin{aligned} \hat{{\varvec{\mathcal {I}}}}_{C,t}= & {} \hat{{\varvec{u}}}_t \hat{{\varvec{u}}}_t' + \hat{{\varvec{d}}}_t \hat{{\varvec{d}}}_t'\\ \hat{{\varvec{\mathcal {I}}}}_{D,t}= & {} \hat{{\varvec{u}}}_t \hat{{\varvec{d}}}_t' + \hat{{\varvec{d}}}_t \hat{{\varvec{u}}}_t'\\ \hat{{\varvec{\mathcal {I}}}}_{N,t}= & {} \hat{{\varvec{n}}}_t \hat{{\varvec{n}}}_t', \end{aligned}$$

where the element of position i of the $k\times 1$ vectors $\hat{{\varvec{u}}}_t$, $\hat{{\varvec{d}}}_t$, and $\hat{{\varvec{n}}}_t$ is $I\left( \hat{\eta }_{i,t}\ge Q_i^{\eta } \right)$, $I\left( \hat{\eta }_{i,t}\le -Q_i^{\eta } \right)$, and $I\left( |\hat{\eta }_{i,t}|\le Q_i^{\eta } \right)$, respectively. The matrices ${\varvec{\overline{\mathcal {C}}}}$, ${\varvec{\overline{\mathcal {D}}}}$, and ${\varvec{\overline{\mathcal {N}}}}$ are obtained as the unconditional expectations of $\hat{{\varvec{\mathcal {I}}}}_{C}$, $\hat{{\varvec{\mathcal {I}}}}_{D}$, and $\hat{{\varvec{\mathcal {I}}}}_{N}$, respectively. Contrary to the approach of Algieri et al. (2021), who make the Gerber correlation dynamic by assuming two parameters for each possible pair of assets, the proposed model employs only six parameters in total. To estimate the parameters appearing in (8), we assume the multivariate skew-Student distribution used in the context of DCC models by Bauwens and Laurent (2005). The model again consists of equations (5)–(6) with the difference that i) the innovations follow the standardized multivariate skew-Student distribution for which the density function is given in Appendix A and ii) ${\varvec{R}}_t$ is replaced by ${\varvec{G}}_t$ obtained³ from (8). With the considered density, each marginal has a specific asymmetry coefficient, related to the parameters in the vector ${\varvec{\xi }}$. Kurtosis (i.e., a measure of the thickness of the tails of the return distribution), is instead captured by the parameter $\nu$, which is assumed to be common to all the k assets, as in Bauwens and Laurent (2005). Under the assumption of a non-normal distribution for the innovations, the decomposition proposed by Engle (2002) is no longer possible. Nevertheless, to follow the spirit of the DCC model, as in Bauwens and Laurent (2005), we estimate the k univariate GARCH-type model by quasi maximum likelihood, or QML, (to estimate the vector ${\varvec{\eta }}$), and then estimate the parameters in the Gerber correlation part together with the vector ${\varvec{\xi }}$ and $\nu$ related to the skew-Student density. When estimating the parameters for the dynamic correlation, we impose the constraints $\max \left\{ |b_x|,|a_x|,|a_x+b_x| \right\} <1$, $x \in \{C,D,N \}$ that imply the stationarity of the three processes appearing in (8) (see Douc et al. 2013).

Risk measures

In this study, we make predictions for the portfolio VaR and ES assuming different forecasts for the covariance matrix (and hence for portfolio volatility). Indeed, our main aim is to assess the impact of competing methods for estimating the covariance matrix. The VaR measure is defined implicitly as

$$\begin{aligned} \textrm{P}\left( r_{p,T+1}\le - \textrm{VaR}_{T+1|T}(\tau ) |\mathcal {F}_T\right) =\tau , \end{aligned}$$

where $r_{p,T+1}$ is the portfolio return at time $T+1$ and $\mathcal {F}_T$ is the information available up to time T. Expected shortfall is instead defined as

$$\begin{aligned} \textrm{ES}_{T+1|T}(\tau )= -\frac{1}{\tau } \int _{-\infty }^{- \textrm{VaR}_{T+1|T}(\tau )} y f(y)\textrm{d}y, \end{aligned}$$

where f(y) is the predicted density for $r_{p,T+1}$ conditional on $\mathcal {F}_T$. Given ${\varvec{\omega }}$, the $k\times 1$ vector of portfolio weights, the forecasts for $\tau$-VaR and $\tau$-ES in the normal case are given by

$$\begin{aligned} \textrm{VaR}_{T+1|T}(\tau )= & {} -{\varvec{\omega }}'{\varvec{\mu }}_{T+1|T} - \sqrt{{\varvec{\omega }}'{\varvec{\Sigma }}_{T+1|T} {\varvec{\omega }}} \Phi ^{-1}(\tau ) \end{aligned}$$

(9)

$$\begin{aligned} \textrm{ES}_{T+1|T}(\tau )= & {} -{\varvec{\omega }}'{\varvec{\mu }}_{T+1|T} + \sqrt{{\varvec{\omega }}'{\varvec{\Sigma }}_{T+1|T} {\varvec{\omega }}} \frac{ \phi (\Phi ^{-1}(\tau )) }{\tau }, \end{aligned}$$

(10)

where ${\varvec{\mu }}_{T+1|T}$ and ${\varvec{\Sigma }}_{T+1|T}$ are the predictions we make based on the information up to time T for the mean vector and covariance matrix at time $T+1$, respectively. In the case of models involving the multivariate skew-Student distribution, no analytic formula is available for VaR and ES. We therefore rely on Monte Carlo simulations that entail simulating portfolio returns for time $T+1$ as $r_{p,j}={\varvec{\omega }}'\left( {\varvec{\mu }}_{T+1|T} + {\varvec{\Sigma }}_{T+1|T}^{1/2} {\varvec{z}}_j\right)$, where ${\varvec{z}}_j$ is the j-th simulation from the standardized skew-Student distribution and $j=1,\ldots ,J$. Using this approach, the $\tau$-VaR for time $T+1$ is estimated as minus one times the $\tau$ empirical quantile of $r_{p,j}$ over the J simulations. $\textrm{ES}_{T+1|T}(\tau )$ can be estimated as minus one times the average of simulations that are smaller than the $\tau$ empirical quantile. In our empirical application, we set $J= 100,000$.

Evaluating VaR and ES predictions

In this study, we employ the MCS procedure of Hansen et al. (2011) to classify the models based on their out-of-sample performance. The procedure is based on an optimality criterion such that the resulting superior set of models $M^*$ will contain the best model with a given confidence level $1-\alpha$.

It uses the idea of sequential testing, for which the generic set $M^0$, containing $m_0$ competing models, gets reduced in the number of elements by an elimination rule if the Equal Predictive Ability (EPA) null hypothesis is rejected. The procedure is iterated until the EPA hypothesis is not rejected for all the models left in the set, constituting the optimal model confidence set $M^*_{1-\alpha }$.

We use a loss function to compare forecasts from different models. In particular, the smaller is the value of the loss function for a given model, the more accurate are the predictions from the model. We denote by $l_{i,t}$ the loss associated with model i at time t. To evaluate VaR forecasts, we use the following loss function (see for instance González-Rivera et al. 2004):

$$\begin{aligned} l_{i,t}\left( r_{p,t}, \textrm{VaR}^i_{t|t-1}(\tau )\right) = \rho _{\tau }\left( r_{p,t} +\textrm{VaR}^i_{t|t-1}(\tau ) \right) , \end{aligned}$$

(11)

where $\textrm{VaR}^i_{t|t-1}(\tau )$ is the predicted $\tau$-VaR at time t based on model i, $r_{p,t}$ is the realized portfolio return, and $\rho _{\tau }(u)=u(\tau -I(u<0))$.

ES is not elicitable⁴ on its own, but it is jointly elicitable together with VaR using a suitable scoring function. Hence, we jointly assess VaR and ES forecasts considering the following functional form proposed by Fissler et al. (2015):

$$\begin{aligned}{} & {} l_{i,t}(r_{p,t}, \textrm{VaR}^i_{t|t-1}(\tau ),\textrm{ES}^i_{t|t-1}(\tau )) \nonumber \\{} & {} \quad =\rho _{\tau }\left( r_{p,t} + \textrm{VaR}^i_{t|t-1}(\tau ) \right) -\tau r_{p,t} - \frac{\textrm{ES}^i_{t|t-1}(\tau )}{ 1 + \exp \left( -\textrm{ES}^i_{t|t-1}(\tau ) \right) }\times \nonumber \\{} & {} \quad \left( \textrm{VaR}^i_{t|t-1}(\tau )-\textrm{ES}^i_{t|t-1}(\tau ) - I(r_{p,t} \le -\textrm{VaR}^i_{t|t-1}(\tau )) \times \frac{\textrm{VaR}^i_{t|t-1}(\tau ) + r_{p,t}}{\tau }\right) \nonumber \\{} & {} \quad +\log \left( \frac{2}{1+ \exp \left( -\textrm{ES}^i_{t|t-1}(\tau ) \right) }\right) , \end{aligned}$$

(12)

where $\textrm{ES}^i_{t|t-1}(\tau )$ is the prediction model i makes for $\tau$-ES at time t. The remaining details of the MCS procedure are given in Appendix B.

Empirical analysis

The dataset we use consists of weekly data for the indices described in Table 1. The data spans the period from January 22, 1999 to January 7, 2022 (1199 observations) and has been downloaded from Bloomberg. The motivation for choosing these four indices is that investors may build highly diversified portfolios by purchasing passive funds mimicking them.

Table 1 Data description

For index j, we compute returns as $r_{j,t}=P_{j,t}/P_{j,t-1}-1$, where $P_{j,t}$ is the index price at the end of week t. Table 2 presents descriptive statistics for the simple returns of the four indices, and Fig. 1 plots their time series. In Fig. 2 we report the Pearson and Gerber correlation matrices calculated on the four time-series of returns. In the case of Gerber correlations, the thresholds are set to half the unconditional standard deviations.

Table 2 Summary statistics for the returns of the four indices

Fig. 1

Time-series plots for the returns of the four indices. The return of index j is calculated as $r_{j,t}=P_{j,t}/P_{j,t-1}-1$, where $P_{j,t}$ is the index price at the end of week t. The horizontal blue lines represent the thresholds used in the calculation of Gerber correlations

Fig. 2

Correlation plots for the returns of the four indices. The thresholds for the Gerber statistic are $Q_j=0.5\times \sigma _j$, $j=1,\ldots ,4$, where $\sigma _j$ is the standard deviation of the returns of index j

In sample analysis

In this section, we use the whole sample to estimate the DGC model. For the first stage, we use an ARMA(1,1)-GARCH(1,1) model for each of the return series. The thresholds for the Gerber statistic are assumed to be half the unconditional volatility. The estimated parameters are reported in Table 3. It is interesting to notice that the persistence⁵, measured by the sum of the parameters a and b, is lower for the equation related to the dynamics of the ${\varvec{\mathcal {N}}}$ matrix and higher for the dynamics of the ${\varvec{\mathcal {C}}}$ and ${\varvec{\mathcal {D}}}$ matrices. The estimated parameters for the part of the model involving the skew-Student distribution point to a negative skewness ($\hat{\xi }_j<1$ for $j=1,\ldots ,4$) and excess kurtosis due to the relatively small estimate for the degrees of freedom parameter ($\hat{\nu }=6.91$). Fig. 3 plots the Gerber correlations obtained from the estimated DGC model via the recursions (8) using the estimated parameters of Table 3. Some interesting features can be gleaned from the figure. For instance, it is possible to observe an increase in the correlation between equities and gold after 2019 (see panels c and d). Furthermore, the correlation between bonds and gold became negative in the aftermath of the financial crisis of 2008 and became again positive around 2015.

Table 3 Univariate GARCH Models (Panel A), skew-Student (Panel B), and DGC Model Estimates (Panel C)

Fig. 3

Time-Series Plots of the Gerber correlations based on the estimated DGC model

Out-of-sample analysis

Here, the focus is on estimating VaR and ES for the three portfolios of Table 4. The first one is the equally-weighted portfolio. The remaining two portfolios both invest 10% in gold. However, they differ in the equity and bond allocation. Indeed, portfolio 2 invests approximately 60% of the allocation (excluding gold) in equities. On the contrary, portfolio 3 invests approximately 60% of the allocation (excluding gold) in bonds. The rationale for choosing the last two sets of weights is that many investors hold a 60/40 or 40/60 portfolio with the addition of a relatively small share of gold⁶.

Table 4 Weights of the three portfolios

We assess how well the proposed DGC model predicts VaR and ES relative to a number of alternative methods. We consider non-parametric, parametric, and semi-parametric methods. The first benchmark is a non-parametric method, historical simulation (HS). The method considers the last T portfolio returns and estimates $\tau$-VaR as the negative of their $\tau$-quantile. Instead, $\tau$-ES is estimated under HS as minus one times the mean of portfolio returns that are less than the $\tau$-quantile. The next two methods we consider, labelled PearsonHist and GerberHist, are for the ‘static’ models relying on the Pearson and Gerber correlation matrices, respectively. In particular, they use the last T observations of the returns on the k assets to estimate the (Pearson or Gerber) correlation matrix, ${\varvec{\Omega }}_{T+1|T}$, and estimate the covariance matrix as ${\varvec{\Sigma }}_{T+1|T} ={\varvec{\Lambda }}_{T+1|T} {\varvec{\Omega }}_{T+1|T} {\varvec{\Lambda }}_{T+1|T}$, where ${\varvec{\Lambda }}_{T+1|T}$ is the matrix with the sample volatilities in the main diagonal and zero elsewhere. In the case of the GerberHist method, Eq. (2) is used and the parameters of the multivariate skew-Student are estimated. With this method, VaR and ES are derived using Monte Carlo simulations as described at the end of section “Risk measures”. However, for the PearsonHist method, VaR and ES are estimated using (9)–(10), with ${\varvec{\mu }}_{T+1|T}$ equal to the vector of sample means for the returns on the k assets. The next alternative method we consider is the DCC model (see Sect. 2.2). In this case, we use ARMA(1,1)-GARCH(1,1) for the univariate specifications. VaR and ES are based on the mean prediction from each univariate ARMA(1,1) model and on the predicted covariance matrix ${\varvec{\Sigma }}_{T+1|T} ={\varvec{\Lambda }}_{T+1|T} {\varvec{R}}_{T+1|T} {\varvec{\Lambda }}_{T+1|T}$, where ${\varvec{\Lambda }}_{T+1|T}$ is the diagonal matrix consisting of the predicted volatilities from each univariate GARCH(1,1) model. For the proposed DGC model, the covariance matrix we use in the estimation of VaR and ES is instead ${\varvec{\Sigma }}_{T+1|T} ={\varvec{\Lambda }}_{T+1|T} {\varvec{G}}_{T+1|T} {\varvec{\Lambda }}_{T+1|T}$. Note that in the DCC and DGC models, we use the same univariate specification—namely, the ARMA(1,1)-GARCH(1,1) model—for each of the return series. Furthermore, to have a meaningful comparison of the DGC and DCC models, for the latter we assume that the innovations follow the multivariate skew-Student distribution as well. Hence, similarly for DCC models, the risk measures of interest are derived using Monte Carlo simulations. Finally, we consider the semi-parametric method filtered historical simulation (FHS) together with the DCC or DGC model. We implement the method as follows. We first estimate an ARMA(1,1)-GARCH(1,1) model on the last T portfolio returns. Denote by $\{z_t\}_{t=1,\ldots ,T}$ the standardized portfolio returns, that is, $z_t=y_t/\sigma _t$, where $y_t$ is the portfolio return at time t, and $\sigma _t$ the time t volatility from the estimated GARCH(1,1) model. VaR and ES are then estimated by modifying (9)–(10) in the following way: i) $\Phi ^{-1}(\tau )$ is replaced by the sample $\tau$-quantile of the z series, $q_z(\tau )$, and ii) $\frac{ \phi (\Phi ^{-1}(\tau )) }{ \tau }$ is replaced by minus one times the mean of the $z_t$ that are smaller than $q_z(\tau )$.

Before presenting the results of the out-of-sample analysis, we first report the results of a Monte Carlo experiment we ran to evaluate the considered methods. To generate scenarios for the assets of interest, we use the moment-matching method of Høyland et al. (2003). We chose an agnostic approach to identify the data-generating process for the simulations to have a fair comparison of the alternative methods for deriving the covariance matrix. For each simulation, we generate 501 observations for the returns of the four assets and use the first 500 observations⁷ to estimate all the models and predict VaR and ES for the three portfolios described in Table 4. We consider three different values of $\tau$—1%, 5%, and 10%—when predicting VaR and ES. The last observation is used to derive the simulated portfolio return and hence to compute the losses (11)–(12) together with the predicted risk measures. The number of simulations in our experiment is 1,000. The results of the Monte Carlo experiment are given in Table 5. From the table, we can see that in the considered experiments the proposed DGC and DGC FHS methods are often the ones for which the two losses are minimized.

Table 5 Average of the loss functions (11)–(12) over the 1000 Monte Carlo simulations. The minimum loss is in bold

In the out-of-sample analysis based on real-world (rather than simulated) returns, we move a window of length $T=500$ to estimate the parameters (for parametric models) or to derive the empirical VaR or ES under the HS method. Consequently, the first prediction we make is for August 22, 2008. As in the Monte Carlo experiments, $\tau$ is equal to 1%, 5%, or 10%.

Results

In this section, we first present the results of the ES-backtesting procedure⁸ based on the results of Khalaf et al. (2021) (see Appendix C). We opt for this recently proposed procedure because it allows backtesting ES only based on a sequence of violations for an appropriately chosen sequence of VaRs. Therefore, contrary to the Du and Escanciano (2017) tests, it does not require knowing the entire conditional cumulative density function, which is difficult to derive, for instance, in the case of FHS methods. We consider conditional tests based on lags corresponding to one week, one month, and two months, that is, $m=1$, $m=5$, and $m=10$. The results of the tests are reported for all the alternative models we consider in Table 6. Overall, models based on the dynamic Gerber correlation seem to have the best performance. Indeed, they perform well for portfolio 3 (Panel C) for all the three values of $\tau$. For the first two portfolios (Panels A and B), instead, the null of “accurate” ES predictions (see Eq. (24)) is rejected for some values of m when $\tau =1\%$ but not for the two remaining values of $\tau$. Despite the fact that it is based on a distribution that captures skewness and excess kurtosis in the returns—like all the alternative models—in this particular case, the DGC model fails to accurately capture the tail expectation for small values of $\tau$ if portfolios consist mainly of highly volatile assets such as equities (such as portfolio 1 and even more so portfolio 2).

Table 6 p values ($\times 100$) for the backtesting procedure of Khalaf et al. (2021), whole sample

As a robustness check, we run the backtests for two sub-samples: the first one covers a relatively tranquil period (January 2017–November 2019), while the second one covers a more turbulent period (February 2020–January 2022). Regarding the results for the two sub-samples, reported in Tables 7 and 8, two observations are in order. First, the proposed methods are not rejected taken as a whole; second, the results for the full sample, not surprisingly, seem to be driven by the more turbulent period that includes the COVID-19 pandemic and the beginning of the Russian-Ukrainian conflict.

Table 7 p values ($\times 100$) for the backtesting procedure of Khalaf et al. (2021), January 2017–November 2019

Table 8 p values ($\times 100$) for the backtesting procedure of Khalaf et al. (2021), February 2020–January 2022

Before presenting the results of the MCS procedure, we run a number of Diebold-Mariano (DM) tests (Diebold and Mariano 2002) to assess the accuracy of the proposed models in predicting VaR alone and VaR and ES jointly. We focus on the three portfolios of Table 4 and test the null hypothesis $H_0:\mathbb {E}(d_{ij})=0$, where $d_{ij}$ is the loss differential between model i and model j. We assume that model i is one of the proposed models (i.e., DGC or DGC FHS) and model j is one of the competing models (i.e., HS, PearsonHist, GerberHist, DCC, or DCC FHS). The results of the DM tests for the forecasting of VaR are given in Table 9, whereas the results for the case of joint VaR/ES forecasting can be found in Table 10. It is evident from both tables that often (and especially in the case of joint forecasting of the two risk measures) the null hypothesis is rejected and the test statistic is negative. When this is the case, the proposed methods (DGC or DGC FHS) are more accurate in predicting the two risk measures than the competing methods. In the few cases in which the test statistic is positive, we instead observe a lack of stars in the table, meaning that the null is not rejected and hence the proposed methods are as accurate as the alternative methods.

Table 9 Test statistics for the Diebold-Mariano testing procedures, $\tau$-VaR forecasting

Table 10 Test statistics for the Diebold-Mariano testing procedures, joint $\tau$-VaR and $\tau$-ES forecasting

Next, we perform the MCS procedure using $\alpha =5\%$ and 5,000 bootstrap replications to derive the p values associated with the test statistics involved. We report the SSMs for the three portfolios we consider in Tables 11, 12, and 13 for the case of VaR predictions and in Tables 14, 15, and 16 for the case of joint VaR/ES predictions. Each table reports, for each model i belonging to the identified SSM, the statistics $\max _{j} T_{ij}$ and $T_{i\cdot }$, the p values associated with the test statistics ${T_R}$ and $T_{max}$, the ranking of the models in the SSM based on ${T_R}$ and $T_{max}$, and the average of the loss function (11) when we forecast VaR or the average of (12) when we jointly forecast VaR and ES.

In the tables involving VaR predictions alone, there are just a few cases where the SSM does not include all the considered models. When that happens, just one model is excluded. For portfolio 1, we have that all the models belong to the SSM when the VaR confidence level, $\tau$, is equal to 1% or 5%, and that only model PearsonHist gets excluded when $\tau =10\%$. For portfolio 2, all the models belong to the SSM when $\tau =1\%$, only the HS model gets excluded when $\tau =5\%$, and only the PearsonHist model gets excluded when $\tau =10\%$. Finally, for portfolio 3, all the models belong to the SSM for all three levels of $\tau$ we consider. Hence, the results regarding VaR predictions do not evidence much difference between static and dynamic models in terms of inclusion in the SSM, although the few excluded ones are static. It is worth highlighting that the newly proposed models, DGC and DGC FHS, can produce VaR predictions that allow them to belong to the SSM. From the second and fifth columns in each panel of Tables 11–13, we see that dynamic models usually rank high among the considered models. Based on the statistic $\max _j T_{ij}$, the models DCC, DCC FHS, DGC, and DGC FHS are always in the first four positions. Based instead on the statistics $T_{i\cdot }$, there are more cases where some of the static models are ranked in the first four positions, but, in any case, dynamic models are always in the first two positions. However, at least one of the newly proposed models, DGC and DGC FHS, usually ranks high with the statistic $T_{i\cdot }$. To justify these results, we observe that the out-of-sample analysis uses a period—August 22, 2008 to January 7, 2022—characterized by episodes of turmoil in the markets. Indeed, the period includes the final part of the global financial crisis of 2007–2008, the European sovereign debt crisis of 2009 through the mid-to-late 2010s, 2011 Black Monday, and the stock market crash of 2020 caused by the COVID-19 pandemic. Dynamic models rank high in Tables 11–13 because they describe financial time series in the presence of turmoil better than static models do. However, the rolling-window approach applied to make VaR predictions allows static models to stay in the SSM.

As far as the joint prediction of VaR and ES is concerned, from Tables 14–16 we see that all the SSM consist only of the newly proposed DGC model. The results are consistent across portfolios and confidence levels of the two risk measures we forecast. This means that, for the three portfolios consisting of the four assets considered in our study and for the period of investigation, the DGC model produces joint VaR/ES predictions that are more accurate than the competing models.

Table 11 Superior Set of Models, $\tau$-VaR forecasting, Portfolio 1

Table 12 Superior Set of Models, $\tau$-VaR forecasting, Portfolio 2

Table 13 Superior Set of Models, $\tau$-VaR forecasting, Portfolio 3

Table 14 Superior Set of Models, joint $\tau$-VaR and $\tau$-ES forecasting, Portfolio 1

Table 15 Superior Set of Models, joint $\tau$-VaR and $\tau$-ES forecasting, Portfolio 2

Table 16 Superior Set of Models, joint $\tau$-VaR and $\tau$-ES forecasting, Portfolio 3

Conclusions

Methods that produce accurate forecasts of risk measures like VaR and ES are essential in an environment where market turmoil and substantial losses for investors are becoming increasingly frequent. This study introduces a new method, DGC, to predict the two risk measures based on the dynamic version of the robust correlation proposed by Gerber et al. (2022) that extends Kendall’s Tau. As in DCC models, the proposed model is based—in a first stage of the estimation process—on univariate GARCH models. In an efficient way, the parameters in the recursions for the dynamic robust correlation matrix are estimated in the second stage. The distribution we use for estimating the DGC model captures asymmetries and fat tails in the returns, hence making a realistic assumption—especially in the presence of market turbulence. In an out-of-sample exercise, we tested the performance of the proposed DGC method in accurately forecasting only VaR or VaR and ES jointly for portfolios consisting of four assets. For three different diversified portfolios realistically selected by many investors, we first backtested ES for the alternative models under scrutiny. With the exception of the portfolios consisting mainly of equities for the case of the probability level $1\%$, we did not reject the null of accurate ES predictions from models based on the DGC method. Finally, for VaR and ES corresponding to different probability levels, we derived the superior set of models using the Model Confidence Set procedure. We showed that for all the portfolios and VaR/ES confidence levels we consider, the DGC model is part of the superior set of models. Because the DGC method is based on realistic assumptions about financial returns and, compared to existing methods, makes accurate predictions of market risk, it represents a valid tool for investors and financial regulators, both of whom would be concerned about the impact of losses in a turbulent market environment. Additionally, the results of our empirical application provide insight useful for investors who wish to diversify their portfolios across different asset classes. A possible limitation of the proposed methodology is that it implies the same thickness of tails for each marginal density. Another limitation is that the skewness parameters are assumed to be static. Future research may delve into these issues. Indeed, it would be interesting to extend the model by including the skewness parameters evolving according to a Generalized Autoregressive Score model. Another interesting development of the paper could be the use of the proposed method for portfolio selection, which we also leave for future research.

Appendices

Appendices

Appendix A: The standardized skew-student distribution

The standardized skew-Student density (Bauwens and Laurent 2005) is given by

$$\begin{aligned} f({\varvec{z}}\left| {\varvec{\xi }},\nu \right. )= \left( \frac{2}{\sqrt{\pi (\nu -2)}} \right) ^k \left( \prod _{i=1}^k \frac{ \xi _i s_i}{ 1+ \xi _i^2} \right) \frac{ \Gamma ((\nu +k)/2 )}{ \Gamma (\nu /2) } \left( 1+ \frac{{\varvec{z}}^{*'} {\varvec{z}}^*}{\nu -2} \right) , \end{aligned}$$

(13)

where the i-th component of the vector ${\varvec{z}}^*$ is $z_i^*=(s_iz_i+m_i)\xi _i^{I_i}$, with

$$\begin{aligned} m_i= & {} \sqrt{\frac{ \nu -2 }{ \pi } } \frac{ \Gamma ((\nu -1)/2) }{ \Gamma (\nu /2)} \left( \xi _i-\xi _i^{-1}\right) \\ s_i= & {} \left( \xi _i^{2}+\xi _i^{-2}-1\right) - m_i^{2}, \end{aligned}$$

and $I_i = 2\times I\left( z_i < -m_i/s_i \right) -1$.

Appendix B: The MCS procedure

The relative performance between models i and j is obtained via the differential

$$\begin{aligned} d_{ij, t} = l_{i,t} - l_{j,t}\qquad \forall i, j \in M_0\quad t=1, \dots , n , \end{aligned}$$

(14)

and the simple average loss of model i relative to the other models $j \in M$ at time t as

$$\begin{aligned} d_{i\cdot , t} = (m-1)^{-1} \sum _{j\in M\setminus {i} } d_{ij,t}. \end{aligned}$$

(15)

For the elimination of inferior elements within the set $M_0$, two alternative sets of hypotheses are available to test the EPA:

$$\begin{aligned} {\left\{ \begin{array}{ll} H_0 :\mathbb {E}(d_{ij})=0 \quad \forall i, j= 1, \dots , m, \qquad \text {against}\\ H_1 :\mathbb {E}(d_{ij})\ne 0 \end{array}\right. } \end{aligned}$$

(16)

or

$$\begin{aligned} {\left\{ \begin{array}{ll} H_0 :\mathbb {E}(d_{i\cdot })=0 \quad \forall i= 1, \dots , m, \qquad \text {against}\\ H_1 :\mathbb {E}(d_{i\cdot })\ne 0 \end{array}\right. } \end{aligned}$$

(17)

Two statistics are then constructed to test the above hypotheses:

$$\begin{aligned} T_{ij} = \frac{\bar{d}_{ij} }{\sqrt{ \widehat{ {\text {var}}} ( \bar{d}_{ij})}};\qquad \qquad T_{i\cdot } = \frac{\bar{d}_{i\cdot }}{\sqrt{ \widehat{ {\text {var}}} ( \bar{d}_{i\cdot })}}, \end{aligned}$$

(18)

where $\bar{d}_{ij}$ constitutes the relative average loss between models i and j and $\bar{d}_{i\cdot }$ represents the average loss of model i relative to the average loss across the models belonging to the set M:

$$\begin{aligned} \bar{d}_{ij} = n^{-1} \sum _{t=1}^{n} d_{ij, t}; \qquad \bar{d}_{i\cdot } = (m-1)^{-1} \sum _{j\in M\setminus {i}} \bar{d}_{ij}. \end{aligned}$$

(19)

The standard errors in the denominators of (18) are constructed by block bootstrap.

The two hypotheses from (16) and (17) are mapped into two test statistics:

$$\begin{aligned} T_{R} = \max _{i,j \in M} T_{ij}; \qquad T_{max} = \max _{i \in M} T_{i\cdot }. \end{aligned}$$

(20)

Because their distributions under the null are not known, they are also simulated by the bootstrap method. When the null hypothesis is rejected, the following elimination rules establish which of the models can be discarded:

$$\begin{aligned} e_{R} = \mathop {\mathrm {arg\,max}}\limits _i \left\{ \sup _{j \in M} T_{ij} \right\} ; \qquad e_{max} = \mathop {\mathrm {arg\,max}}\limits _i \left\{ T_{i\cdot } \right\} . \end{aligned}$$

(21)

Bernardi and Catania (2016) summarized the algorithm for the procedure as follows:

• 1. Set $M = M_0$.

• 2. Compute the test statistics under the null EPA hypothesis. If it is not rejected, set $M^*_{1-\alpha } = M$ and terminate the algorithm. If it is rejected, use the elimination rule to determine the worst model.

• 3. Discard the model and repeat step 2.

The elimination rule defines a sequence of sets $M = M_0 \supset M_1\dots \supset M_m$, where $M_i = (e_{M_i}, \dots , e_{M_m})$, each of which has a p value associated with an EPA test. Let $P_{H_0, M_i}$ be the p value associated with the null hypothesis $H_{0, M_i}$. The MCS p value for model $e_{M_j}\supset M$ is defined as $\hat{p}_{e_{M_j}} = \max _{i\le j} P_{H_0, M_i}$.

Appendix C: Backtesting ES

In this appendix we provide the details of the procedure we implement to backtest ES.

Du and Escanciano (2017) proposed a procedure to backtest $\tau$-ES based on the so-called cumulative violations (CV) process. The time-t value of this process is given by

$$\begin{aligned} H_{t}(\tau )=\frac{1}{\tau }\left[ \tau -u_{t}\right] I\left( u_{t}\le \tau \right) , \end{aligned}$$

(22)

where $u_{t}=G(\left. r_{p,t}\right| \mathcal {F}_{t-1})$ is the Probability Integral (PIT) transform (Rosenblatt 1952) and $G(\left. \cdot \right| \mathcal {F}_{t-1})$ is the conditional cumulative distribution function of the portfolio return $r_{p,t}$.

To avoid estimating the portfolio distribution function, Khalaf et al. (2021) consider K equally spaced VaR levels with the larger one coinciding with $\tau$, i.e. $\tau _j=(K-j+1)\frac{\tau }{K}$ for $j=1,\ldots ,K$. Their ES backtesting procedures are based on the sum of VaR violations

$$\begin{aligned} N_{t}^{K}(\tau ) = \sum _{j=1}^{K} I\left( r_{p,t}\le -\textrm{VaR}_{t|t-1}\left( j\frac{\tau }{K }\right) \right) , \end{aligned}$$

(23)

since they show that, when returns are absolutely continuous, $\frac{N_{t}^{K}(\tau )}{K}\xrightarrow {D}H_{t}(\tau )$. Khalaf et al. (2021) hence consider the null

$$\begin{aligned} H_0 : {\left\{ \begin{array}{ll} N_{t}^{K}(\tau ) =j \text { with probability }\theta _{j} \text { for } j=0,\ldots ,K, \\ N_{t}^{K}(\tau ) \perp \!\!\! \perp N_{t-h}^{K}(\tau ) {, \forall h\ne 0,} \end{array}\right. } \end{aligned}$$

(24)

where $\perp \!\!\! \perp$ denotes independence, and

$$\begin{aligned} \theta _{0}=1-\tau ,\qquad \theta _{j}=\frac{\tau }{K},\quad j=1,\ldots ,K. \end{aligned}$$

(25)

Note that $\mathbb {E}\left[ N_{t}^{K}(\tau )\right] =\frac{K+1}{2}\tau$ under (24)–(25). As a consequence, in this paper we use a conditional backtest based on the idea that under the null, $\left\{ N_{t}^{K}(\tau )- \frac{K+1}{2}\tau \right\} _t$ is a martingale difference sequence.

To this end, we use, for a sample of length n for $N_{t}^{K}(\tau )$, the Box-Pierce test statistic

$$\begin{aligned} C_m(\tau )= n\sum _{i=1}^{m} \rho _i^2(\tau ), \end{aligned}$$

(26)

where

$$\begin{aligned} \gamma _i(\tau )=\frac{1}{n-i} \sum _{t=i+1}^{n} \left( N_{t}^{K}(\tau )- \frac{K+1}{2}\tau \right) \left( N_{t-i}^{K}(\tau )- \frac{K+1}{2}\tau \right) , \end{aligned}$$

and $\rho _i(\tau )=\frac{\gamma _i(\tau )}{\gamma _0(\tau )}$. The p values associated with the test statistic (26) can be obtained using the Monte Carlo test technique (Dufour 2006) since, under the null (24)–(25), it is easy to simulate (23) and hence the test statistic (26), see Khalaf et al. (2021) for further details.