Do U.S. economic conditions at the state level predict the realized volatility of oil-price returns? A quantile machine-learning approach

Abstract

Because the U.S. is a major player in the international oil market, it is interesting to study whether aggregate and state-level economic conditions can predict the subsequent realized volatility of oil price returns. To address this research question, we frame our analysis in terms of variants of the popular heterogeneous autoregressive realized volatility (HAR-RV) model. To estimate the models, we use quantile-regression and quantile machine learning (Lasso) estimators. Our estimation results highlights the differential effects of economic conditions on the quantiles of the conditional distribution of realized volatility. Using weekly data for the period April 1987 to December 2021, we document evidence of predictability at a biweekly and monthly horizon.

Introduction

In the wake of the severe global financial crisis (GFC) of 2007–2009 and a series of crises that followed, such as the European sovereign debt crisis, Brexit, and the ongoing COVID-19 pandemic, risks associated with portfolios comprising conventional financial assets have received considerable attention in recent empirical research (see, e.g., Balcilar et al. 2017a, 2020; Muteba Mwamba et al. 2017). However, because investors search for diversification opportunities, these crises have resulted in a noticeable trend towards alternative investment opportunities, including investments in commodities, in general, and oil, in particular (Bampinas and Panagiotidis 2015, 2017). This trend has led financial market participants to supplement their traditional portfolios with positions in commodities (Bahloul et al. 2018; Bonato 2019), and the resulting financilization of the commodity sector has been reflected in an increased participation of hedge funds, pension funds, and insurance companies in commodity markets. Crude oil is now considered a profitable alternative instrument in the portfolio decisions of financial institutions, implying that modeling and predicting the volatility of oil price movements has become a key issue in the financial industry and academic research (Degiannakis and Filis 2017). Considering this, the market size of crude-oil investments is $1.7 trillion per year at current spot prices, with 34 billion barrels produced each year and over 1.7 trillion barrels of crude oil in remaining reserves (U.S. Energy Information Administration (EIA); BP Statistical Review of World Energy), making it by far the most actively traded commodity.

The volatility of asset prices is an important input for investment decisions and portfolio choices; hence, accurate predictions of the volatility of oil price returns are of paramount importance to oil traders.¹ Therefore, it is not surprising that a large and ever-burgeoning body of literature has considered the predictive value for the volatility of oil price returns of a large number of macroeconomic, financial, and behavioral variables, based on a wide spectrum of linear and non-linear models.² Given this wide array of predictors, Guo et al. (2022) and Salisu et al. (2022) use the global economic conditions (GECON) index developed by Baumeister et al. (2020)³ to forecast the realized or conditional (generalized autoregressive conditional heteroskedasticity, i.e., GARCH) volatility of movements of the West Texas Intermediate (WTI) and Brent crude oil price, in addition to heating oil and natural gas, as well as exchange-traded funds (ETFs) of the global clean energy stock market (see also Wang et al. (2022) in this regard). These studies show that GECON, which is based on a set of 16 variables covering multiple dimensions of the global economy,⁴ outperforms the other popular predictors associated with global economic activity.⁵ Salisu et al. (2022) suggest that economic conditions are expected to affect oil price volatility based on the present-value model of asset prices (e.g., Shiller 1981a, b), given the financialization of commodity markets, whereby oil price return volatility depends on the volatility of cash flows and the discount factor (Conrad et al. 2014). In this regard, a worsening of global economic conditions (such as crisis periods) affects the volatility of variables that reflect future cash flows by generating economic uncertainty (Bernanke 1983) and the discount factor (Schwert 1989); hence, (a possibly negative) relationship between economic conditions and the volatility of oil price returns can be hypothesized.

Given the importance of global economic conditions in predicting the volatility of oil price returns, we extend this line of research by comparing the role of aggregate versus state-level metrics of economic conditions in the United States (U.S.) in predicting the subsequent realized volatility of WTI oil price returns over the weekly period from April 1987 to December 2021. In this regard, we rely on a novel dataset of weekly economic-condition indexes for the 50 U.S. states that cover multiple dimensions of the overall and state economies of the U.S.⁶ While the decision to analyze the predictive value of the aggregate U.S. economic conditions emanates from the works of Guo et al. (2022) and Salisu et al. (2022), the intuition to look at state-level economic conditions in predicting the realized volatility of oil price returns is straightforward, given the exceptional degree of heterogeneity at the state level in terms of oil dependency (calculated as oil consumed minus oil produced as a percentage of oil consumed). In the process, the strengths of their status as oil suppliers and demanders (De Michelis et al. 2020), as reflected by their underlying economic conditions. Understandably, if measures of state-level economic conditions produce better predictions relative to aggregate economic conditions, this finding is of considerable value to investors, as well as for academics, investigating the possibility of new factors that drive the volatility of oil price returns. Simultaneously, because the volatility of oil price returns has historically been shown to have predictive value for slowdowns in economic growth (van Eyden et al. 2019), policymakers can use relatively more precise estimates of future movements in the volatility of oil price returns to design macroeconomic policies ahead of time to prevent possible economic downturns. This could be achieved, for example, by feeding high-frequency predictions of the volatility of oil price returns into mixed data sampling (MIDAS) models associated with nowcasting of slow-moving, that is, low-frequency macroeconomic variables (Bańbura et al. 2011).

For our empirical research, from an econometric perspective, we use a machine-learning approach to analyze the predictive value of a large number of economic-conditions-based predictors associated with U.S. states. In particular, we rely on a quantiles-based version of the least absolute shrinkage and selection operator (Lasso) estimator (Tibshirani 1996). The idea underlying the Lasso estimator is to reduce the dimension of a predictive regression model in a data-driven manner to improve the interpretability of the model and the accuracy of predictions derived from the regularized model. However, rather than adhering to the standard linear Lasso estimator, we adopt a nonlinear setting and estimate the quantile-regression version of the Lasso estimator to study the predictive value of the economic conditions of the 50 states, in addition to a corresponding small-scale quantile-predictive regression model involving the overall U.S. economic conditions as a predictor. Pan et al. (2017) discuss the need to model nonlinearity in the relationship between the volatility of oil price returns and macroeconomic conditions . An advantage of our quantiles-based approach is that it enables us to develop a more complete characterization of the conditional distribution of the volatility of oil price returns through a set of conditional quantiles. A quantiles-based approach is more flexible than standard parametric approaches, such as linear regressions, Markov switching, and threshold regression models, and is robust to deviations from normality, including the presence of outliers (Gebka and Wohar 2019). Moreover, modeling only the conditional mean of the volatility of oil price returns through a linear or complex nonlinear regression model may hide interesting characteristics and lead us to conclude that predictors have poor predictive performance, while they are actually valuable for predicting certain quantiles of volatility (Gupta et al. 2017). In particular, our approach allows us to capture any potential asymmetric effect (nonlinear relationship) of economic conditions on the distribution of volatility, which renders it possible track different “types” of predictability.

At this stage, it is important to clarify two additional issues. First, we model the weekly realized volatility of returns of the WTI oil price, where we capture the realized volatility as the square root of the sum of daily squared returns over a week (following Andersen and Bollerslev 1998), which, in turn, yields an observable and unconditional measure of volatility, an otherwise latent process. Traditionally,⁷ researchers have studied the time-varying volatility of oil price returns using various models belonging to the GARCH family, under which conditional variance is a deterministic function of model parameters and past data. Alternatively, in recent studies, some researchers have considered stochastic volatility (SV) models, wherein volatility is depicted as a latent variable that follows a stochastic process. In this regard, whether a researcher uses GARCH or SV models, the resulting estimate of volatility is not unconditional (or model-free), as is the case with realized volatility. Second, while oil is a global commodity, because we focus on state-level economic conditions, we consider the WTI as our proxy for the world oil price. However, this should not be an issue, as the U.S. is a major player in both the demand and supply fronts of the oil market.

To the best of our knowledge, this is the first study to compare the role of aggregate and state-level measures of U.S. economic conditions to predict the realized volatility of oil price returns, using quantiles-based small-scale (involving only the national metric of economic conditions as a predictor) predictive regressions and a large-scale machine-learning quantile Lasso approach. By taking a regional versus aggregate perspective of economic conditions within the U.S., we build on the works of Guo et al. (2022) and Salisu et al. (2022), who focus on the role of global economic conditions in forecasting oil market volatility. The only other study that has analyzed the role of state-level variables in forecasting oil market volatility is that by Çepni et al. (2022), wherein the authors depict the importance of state-level uncertainty. Their study, however, is at a monthly frequency, unlike the weekly frequency in our case, which should be of more importance to investors and policymakers, in addition to dealing with a wide array of information capturing general economic conditions rather than just one aspect of regional economies, namely uncertainty. In other words, our study is more general than that of Çepni et al. (2022), especially when one realizes that the newspapers-based metrics of uncertainty employed by Çepni et al. (2022) may be endogenously driven by the economic conditions prevailing in the states (Mumtaz 2018; Mumtaz et al. 2018).

The remainder of our research is organized as follows. We describe our data in "Data" section, while we lay out our empirical methods in "Methods" section. We discuss our empirical results in "Empirical results" section, and conclude the paper in "Concluding remarks" section.

Data

To construct our measure of the realized volatility (RV) of oil price returns, we first compute the daily log-returns (i.e., the first difference of the natural logarithm) of the West Texas Intermediate (WTI) oil price. In the second step, we compute the sum of the daily squared log returns over a specific week. In the third step, we obtain weekly realized volatility by taking the square root of this sum. The daily WTI crude oil nominal price data were derived from the Energy Information Administration (EIA) of the U.S.⁸ Because of the large peak in realized volatility at the end of the sample period, which is associated with the outbreak of the COVID-19 pandemic, we work with the (natural) logarithmic value of realized volatility. Working with log-realized volatility also avoids negativity issues and brings data closer to a normal distribution. Figure 1 plots the resulting time series of (log) realized volatility and its associated autocorrelation function. The slowly decaying pattern of the latter shows that the variants of the HAR-RV model that we lay out in detail in "Methods" section are natural candidates for studying the realized volatility of oil price returns.⁹

Fig. 1

Properties of the Realized Volatility of Oil-Price Returns. Note: The dashed vertical line in Panel B denotes the 95% significance level

Regarding our main predictors, we analyze the role of the weekly economic-conditions indices (ECIs) of the overall U.S., as well as its 50 states. These indices are based on the work of Baumeister et al. (2022), who derive the indexes from mixed-frequency dynamic factor models with weekly, monthly, and quarterly variables that cover multiple dimensions of aggregate and state economies.¹⁰ Specifically, Baumeister et al. (2022) group variables into six broad categories: mobility measures, labor market indicators, real economic activity, expectations measures, financial indicators, and household indicators. Tables 8 and 9 at the end of the study (“Appendix”) provide details of the variables used in the construction of the weekly ECIs under each category at the state level and for the aggregate U.S., respectively. The indices are scaled to 4-quarter growth rates of U.S. real gross domestic product (GDP) and normalized such that a value of zero indicates national long-run growth.

Baumeister et al. (2022) find considerable cross-state heterogeneity in the length, depth, and timing of business cycles, which in turn provides a strong motivation to study the predictive value of not only aggregate but also state-level ECIs for the realized volatility of oil price returns. Based on data availability, our analysis covered the first week of April 1987 to the last week of December 2021.

Methods

The heterogeneous autoregressive realized volatility (HAR-RV) model developed by Corsi (2009) is extensively used in earlier empirical research to study the realized volatility of oil price returns (see, for example, Degiannakis and Filis 2017; Gkillas et al. 2020a). Accordingly, we used the HAR-RV model as the nucleus in our predictive regression models. In the context of our empirical analysis, we formulate the HAR-RV model as follows:¹¹

$$\begin{aligned} RV_{t+h} = \beta _0 + \beta _1 RV_t + \beta _2 RV_{bw,t} + \beta _3 RV_{m,t} + \epsilon _{t+h}, \end{aligned}$$

(1)

where $\epsilon _{t+h}$ denotes the disturbance term, RV denotes the realized weekly volatility of oil price returns, $RV_{bw,t}$ denotes the average biweekly RV from week $t - 2$ to week $t - 1$, and $RV_{m,t}$ denotes the average monthly RV from week $t - 4$ to week $t - 1$, with this structure motivated by the nature of the decay of the autocorrelation function of RV in Fig. 1. Parameter h denotes the horizon over which the subsequent realized volatility of oil price returns is studied. For $h > 1$, we compute $RV_{t+h}$ as the average realized volatility over the relevant horizon, where we study weekly ($h=1$), biweekly ($h= 2$), and monthly ($h = 4$) horizons. For example, in the case where $h= 2$, we have $RV_{t+h} = ( RV_{t+1} + RV_{t+2} ) / 2$. Equation (1) formalizes the basic idea behind the heterogeneous market hypothesis (Müller et al. 1997), according to which different groups of traders populate asset (commodity) markets, where traders belonging to the various groups differ with respect to their sensitivity to information flows at different time horizons.

As a first extension of the baseline model given in Eq. (1), we consider the possibility that aggregate economic conditions, EC,  in the U.S., a major player in the international oil market, may have predictive value for realized volatility. Therefore, we specify the HAR-RV-US model as follows:

$$\begin{aligned} RV_{t+h} = \beta _0 + \beta _1 RV_t + \beta _2 RV_{bw,t} + \beta _3 RV_{m,t} + \gamma ECI_{t, US} + \epsilon _{t+h}. \end{aligned}$$

(2)

As our second extension, we study a version of the baseline model that incorporates predictors, not the aggregate economic conditions in the U.S., but rather the economic conditions as measured at the level of individual states. This extension leads to the HAR-RV-states model:

$$\begin{aligned} RV_{t+h} = \beta _0 + \beta _1 RV_t + \beta _2 RV_{bw,t} + \beta _3 RV_{m,t} + \sum _{i=1}^{50} \gamma _i ECI_{t, i} + \epsilon _{t+h}, \end{aligned}$$

(3)

where index i denotes one of the 50 states. Given the large number of parameters of the HAR-RV-states model, it is preferable to estimate the predictive regression model given in Eq. (3) using parameter shrinkage and model regularization techniques.¹² To this end, we used the least absolute shrinkage and selection operator (Lasso) proposed by Tibshirani (1996). The purpose of the Lasso estimator is to select a parsimonious version of the HAR-RV-states model by minimizing the following expression (see also the discussion in the textbook by Hastie et al. (2009)):

$$\begin{aligned} \sum _{t=1}^T \left( RV_{t+h} - \beta _0 - \beta _1 RV_t - \beta _2 RV_{bw,t} - \beta _3 RV_{m,t} - \sum _{i=1}^{50} \gamma _i ECI_{t, i} \right) ^2 + \lambda \left( \sum _{i=1}^{50} | \gamma _i | \right) \end{aligned}$$

(4)

where T denotes the number of observations and $\lambda$ denotes a shrinkage parameter. Equation (4) clarifies that the LASSO estimator adds to the standard quadratic loss function a penalty term that increases the absolute value of the coefficients to be estimated. Hence, the Lasso estimator implies that it is preferable to select coefficients that are small in absolute value or even zero, where the effect of model shrinkage must be balanced against its effect on the quadratic loss function. It should be noted that according to Eq. (4), we apply the Lasso model shrinkage only to shrink the coefficients of the states, not the intercept or coefficients of the classic HAR-RV model. The extent of shrinkage in the HAR-RV-states model depends on the magnitude of the shrinkage parameter. If the shrinkage parameter is sufficiently large, the Lasso estimator is set to zero for some or all coefficients. In our empirical research, we used tenfold cross-validation to optimize the value of the shrinkage parameter, where we used the check function to evaluate the cross-validated error.

A drawback of the predictive regression models given in Eqs. (1)–(4) is that they do not account for the possibility that the predictive value of economic activity for the subsequent realized volatility may depend on the quantile of the conditional distribution of the realized volatility oil price returns; that is, the predictive value of economic activity may depend on whether the oil market is in a state of low, intermediate, or high levels of volatility. To account for this possibility of nonlinearity, we study quantile-regression versions of the predictive regression models formalized in Eqs. (1)–(4) (see also Gkillas et al. (2020b), and Bonato et al. 2021, and for the seminal paper on quantile regressions, see Koenker and Bassett 1978). The quantile-regression versions of the HAR-RV model are given by

$$\begin{aligned} {\hat{\mathbf{b}}}_\alpha = \arg \min \sum _t^T \rho _\alpha \left( RV_{t+h} - \beta _{0,\alpha } - \beta _{1,\alpha } RV_t - \beta _{2,\alpha } RV_{bw,t} - \beta _{3,\alpha } RV_{m,t} \right) , \end{aligned}$$

(5)

where $\alpha$ denotes the quantile being studied, and ${\hat{\mathbf{b}}}_\alpha$ denotes the quantile-dependent vector of coefficients to be estimated (a hat denotes an estimated parameter). Function $\rho _\alpha$ is the check function, defined as $\rho _\alpha = \alpha \; \epsilon _{t+h}$ if $\epsilon _{t+h} > 0$ and $\rho _\alpha = ( \alpha - 1 ) \; \epsilon _{t+h}$ if $\epsilon _{t+h} < 0$. The quantile-regression version of the HAR-RV-US model can be derived by adding the aggregate U.S. economic activity to Eq. (2) as an additional predictor

The predictive regression model in Eq. (3) can be extended to a quantile-based predictive regression model in an analogous manner. However, given the large number of coefficients to be estimated, we do not estimate the quantile version of the HAR-RV-states model as a standard quantile-regression model, but rather as a penalized Lasso quantile-regression model (see Li and Zhu 2008; for a recent application of variants of the penalized quantile-regression techniques to a problem in energy economics, see Ren et al. 2022; for an analysis of the quantile Lasso approach in the context of a fixed-effects model (see also Koenker 2004). Accordingly, the quantile-regression version of the HAR-RV-states model is given by:

$$\begin{aligned} {\hat{\mathbf{b}}}_\alpha= & {} \arg \min \sum _t^T \rho _\alpha \left( RV_{t+h} - \beta _{0,\alpha } - \beta _{1,\alpha } RV_t - \beta _{2,\alpha } RV_{bw,t} - \beta _{3,\alpha } RV_{m,t} - \sum _{i=1}^{50} \gamma _{i, \alpha }ECI_{t, i} \right) \nonumber \\{} & {} + \lambda _\alpha \left( \sum _{i=1}^{50} | \gamma _{i, \alpha } | \right) \end{aligned}$$

(6)

where the shrinkage parameter was optimized given the quantile being analyzed.

To assess the fit of the various predictive regression models, we used a relative performance statistic (see also Koenker and Machado 1999; Pierdzioch et al. 2014, 2016). The relative performance RP is given by:

$$\begin{aligned} RP_{B, R | \alpha } = 1 - \frac{ \sum _{ t=1 }^T \rho _\alpha \left( e_{t, R} \right) }{ \sum _{ t=1 }^T \rho _\alpha \left( e_{t, B} \right) } \end{aligned}$$

(7)

where $e_{t,B}$ denotes the prediction error implied by the benchmark model and $e_{t,R}$ denotes the prediction error implied by the rival model. The summation in Eq. (7) runs over the entire sample when studying the full sample of the data. When we study the out-of-sample predictive values of the models, the summation runs over the relevant out-of-sample period.

It follows from the definition of the relative performance statistic given in Eq. (7) that given a quantile, the rival model performs better than the benchmark model when $RP > 0$. In turn, the benchmark model outperformed the rival model when $RP < 0$.¹³ It should be noted that, as is made explicit by Eq. (7), we evaluate the predictions under the same loss (check) function that we use to estimate the quantile (Lasso) regression models. Hence, as discussed by Koenker and Machado (1999), the relative performance statistic measures the relative predictive value of the benchmark and rival model at the quantile being studied in terms of a loss-function-weighted sum of absolute prediction errors. Therefore, the relative performance statistic is a quantile-specific local measure of relative predictive performance rather than a global measure evaluated over the entire conditional distribution of realized volatility. Such a local approach is a natural choice in the context of our empirical analysis because, as emphasized in "Introduction" section, we are interested in recovering the differential and potentially asymmetric effects of (state-level) economic conditions on different quantiles of the conditional distribution of realized volatility, rather than in inferring their global impact on predictive model performance over the entire conditional distribution.

We use the R language and environment for statistical computing (R Core Team 2021) to conduct our empirical research, where we use the R add-on package “rqPen” (Sherwood and Maidman 2020) to estimate the quantile (Lasso) regression models.

Empirical results

Table 1 summarizes the baseline results. The table shows, for the three horizons being studied, the relative performance statistic, where we compare the classic HAR-RV model with the HAR-RV-US and HAR-RV-states models and the HAR-RV-US model with the HAR-RV-states model. Three main results were obtained: First, the relative performance statistics are close to zero for the weekly horizon, indicating that there are hardly any differences in the predictive values of the three models. This could be an indication that the information regarding the ECIs could not instantaneously impact demand and supply decisions in the oil market and took time to feed into oil price movements, as some production decisions were likely made ahead of time. Second, the relative performance statistic increases in the horizon when we compare the HAR-RV and HAR-RV-US models with the HAR-RV-states model. Hence, the incremental predictive value of state-level economic conditions strengthens in the biweekly and monthly horizons. This observation is in line with the one drawn above in terms of a time-lag, but it is also indicative of our initial motivation of investigating state-level ECIs, which allows us to better capture, relative to that of the overall economic conditions of the U.S., the demand- and supply-side dynamics of the oil market in line of the heterogeneity associated with oil dependence across the U.S. states. Third, accounting for state-level economic conditions at the biweekly and monthly horizons leverages relative performance, especially in the upper and lower quantiles of the conditional distribution of realized volatility. This effect was particularly pronounced in the monthly forecast horizon. Consequently, accounting for the impact of state-level economic conditions is especially useful for predicting the subsequent low and high realized volatility of oil price returns at the lower (5%) quantiles, especially at the upper (95%) quantiles.¹⁴ This result of detecting gains at extreme ends of oil market variability should not come as a surprise and can be explained following the works of Balcilar et al. (2017b) and Bonaccolto et al. (2018). In this regard, the median is indicative of normal levels of uncertainty prevailing in the oil market, and hence, does not require investors to utilize the information content of the ECIs for volatility. However, when the oil market is characterized by low or high degrees of volatility, it is understandable that oil market traders will want to use ECIs to predict where the future path of volatility is headed, that is, whether it is going to increase or decrease conditional on demand and supply conditions, so that they can make optimal portfolio decisions.

Table 1 Baseline results

Further results (not reported but available from the authors upon request) show that the average absolute size of the coefficients estimated for the various state-level economic conditions increases in the forecast horizon, especially at the monthly horizon. Moreover, at the monthly horizon, the average absolute size of the coefficients estimated for state-level economic conditions increases in the quantiles. Furthermore, the proportion of state-level economic conditions included in the penalized models increases as we move from the weekly to the monthly horizon. These three results should not come as a surprise, given the findings reported earlier, and indicate that economic conditions, especially for the states, gain importance over investment horizons, and are of more relevance to oil market players when uncertainty, that is, volatility in the oil market is already high, compared to situations where it is low or normal. In the weekly horizon, the proportion of state-level economic conditions included in the penalized models is relatively high (above 40%) at the median (which explains why the results of the permutation tests for the weekly horizon reported in Table 2 are significant at the median).¹⁵

The results of the permutation tests reported in Table 2 show that the increase in predictive performance resulting from extending the forecasting model to include state-level economic conditions to the vector of predictors is statistically significant, which is in line with our initial premise for the need for disaggregated information that can be derived from state-level ECIs. We implement the permutation tests as follows. We sample without replacement 500 times the state-level economic conditions. We then estimate the HAR-RV-states model using the quantile Lasso estimator on the simulated data and store the model prediction errors. Next, we compute the relative performance statistics for every simulated dataset, where the prediction errors of the benchmark HAR-RV-US model are based on the estimates reported in Table 1. Finally, we compute the p value of the permutation test as the proportion of the relative performance statistics computed for the simulated data, which exceeds the relative performance statistics reported in Table 1. If the state-level economic conditions contribute to the predictive performance of the model, the simulated relative performance statistics should fall short of the relative performance statistics documented in Table 1 most of the time.

Table 2 Results of permutation tests

The results of the permutation tests show that at the weekly horizon, predictive performance due to state-level economic conditions increases in a statistically significant way, mainly at the median. At the biweekly and monthly forecast horizons, all the permutation tests yielded significant results. In other words, we find strong evidence that state-level economic conditions help to improve in a statistically significant way predictions of the subsequent realized volatility of oil-price returns at the biweekly and monthly horizons. This finding supports the basic motivation of looking at state-level economic conditions in addition to the overall condition, as we expect the former to better capture the demand and supply of oil, particularly as the forecast horizon increases, by accounting for heterogenous oil dependency across the states.

Table 3 Results for RV

Next, we report the robustness check results for realized volatility (rather than its logarithm) in Table 3. There were no changes in the general picture. The HAR-RV-US model does not add predictive value over and above the predictive value of the classic HAR-RV model, accounting for state-level economic conditions, boosts relative predictive performance, especially at the biweekly and monthly horizons. Moreover, the impact of state-level economic conditions on relative performance in the monthly horizon is again strongest in the lower and upper quantiles. These findings are in line with the underlying intuition presented above in terms of time lags, heterogeneity of oil dependency, market states affecting investment decisions, and the fact that it remains consistent irrespective of the scaling of the process of volatility, confirming the robustness of our understanding of how oil market volatility is affected by economic conditions, even though we are using an atheoretical approach here to forecast oil realized volatility.

Table 4 Results of an out-of-sample analysis

It is also interesting to analyze predictive performance in a quasi-out-of-sample context. To this end, we bootstrap the data 500 times without replacement, fixing the fraction of out-of-sample data for every bootstrap sample at 30%. We then estimate all three models on the bootstrapped data and make forecasts of the “out-of-sample data” (also known as the out-of-bag data in the machine-learning literature; it should be noted that sampling without replacement implies that the out-of-bag data are not included in the sample of data on which we train the model). For every bootstrap sample, we compute the relevant relative performance statistics. Finally, we compute the mean of the resulting sampling distributions of the relative performance statistics and study the proportion of negative relative performance statistics (which indicates that the benchmark model is superior to the rival model). We document the results in Table 4. As expected, the performance statistics were smaller than those summarized in Table 1. At the weekly horizon, the relative performance statistics are negative or close to zero, on average, for all three model combinations. Not surprisingly, the p values demonstrate that neither the HAR-RV-US nor the HAR-RV-states model exceeds the HAR-RV model in terms of predictive value. At the biweekly horizon, while the relative performance statistics for the HAR-RV-US remain negative on average, the mean values of the relative performance statistics for the HAR-RV-states model mostly take a positive but small value. There is some evidence that accounting for state-level economic conditions helps significantly increase predictive performance for the 75% quantile. Finally, for the monthly horizon, the p values for the HAR-RV-US model remain well above conventional significance levels, but the p values for the HAR-RV-states model show that state-level economic conditions significantly boost the predictive performance for all five quantiles being studied. Hence, we find evidence of the ability of state-level economic conditions, as with the in-sample tests, to predict gains for oil market volatility, particularly in the medium (bi) to the long run. While these findings can benefit oil market investors in their portfolio decisions, they tend to corroborate our underlying explanation of the in-sample results discussed above, especially with time lags and oil dependency across states.

Table 5 Results for components of state-level economic conditions

In Table 5, we document that the results that we find for state-level economic conditions also hold when we study the components of state-level economic condition indexes (expectations, financials, households, labor market, mobility, and real activity).¹⁶ The components of state-level economic conditions contribute to predictive performance (relative to the HAR-RV and HAR-RV-US models) mainly at the biweekly and monthly horizons and at the lower and, especially, at the upper (95%) quantiles, demonstrating the robustness of our results. In other words, the use of the overall ECIs of the states can convey the same information that can be obtained from its disaggregated component. This implies that the usage of all underlying information that goes in the construction of state-level ECIs is important, whether in an aggregate manner or with the separate components considered simultaneously, indicating the importance of the various economic categories of variables considered in appropriately capturing the price dynamics of the oil market.

Table 6 Results for the weakness index

As a further illustration of the robustness of our results, we report in Table 6 the results that we obtained when we replaced the data on the economic conditions index of the overall U.S. with the economic weakness index (EWI).¹⁷ The EWI is a summary measure of national business cycle dynamics and is constructed using state-level recession probabilities extracted from a Markov-switching model that allows for heterogeneous recessions and expansions (see Baumeister et al. 2022, for further details). The general pattern of our results remained unchanged. State-level economic activity again contributes to the predictive performance at the biweekly and monthly horizons, where this contribution is particularly strong at the upper quantile of the conditional distribution of the realized volatility of oil price returns. Hence, we can safely say that our economic explanation for the obtained econometric results is not sensitive to the choice of the metric of economic conditions involving the entire U.S., which again highlights the importance of the economic conditions at the state level in better capturing the underlying heterogenous nature of demand and supply of oil.

Table 7 Additional results based on intraday data calculations

Finally, Table 7 reports the additional results for data on the realized volatility of returns of crude oil, heating oil, and natural gas prices, whereby, instead of daily data to obtain the weekly values of realized volatility, we rely on underlying intraday data for the estimations, because intraday data contains rich information that can lead to more accurate estimates of volatility (McAleer and Medeiros 2008). The daily realized volatility data are derived from Risk Lab.¹⁸ For our empirical research, we sum-up over a week the daily realized volatility estimates based on 5-min subsampled returns of the NYMEX light crude oil, NYMEX heating oil No. 2, and NYMEX natural gas futures, with the sample period covering the fourth week of December 2000 to the fourth week of December 2021. It is reassuring to observe that our main results also apply not only to the realized volatility of crude oil derived using an alternative approach but also to heating oil and natural gas. In other words, the intuitive explanation of the results provided above based on weekly RV computed from daily data is robust to the use of an alternative data frequency to derive metrics of volatility for oil and the general energy market, which also includes heating oil and natural gas.

Concluding remarks

We have shown for the U.S. that state-level economic activity as measured has quantile-dependent predictive value for the subsequent realized volatility of oil price returns. While predictability is weak and hardly existent at a weekly horizon, evidence of predictability strengthens at biweekly and monthly horizons. Using the popular HAR-RV model as the starting point of our empirical analysis, we recovered robust evidence that predictability is particularly strong at the upper (95%) and lower (5%) quantiles of the conditional distribution of realized volatility. Given that the U.S. is a major player in the international oil market, and given that the results of much significant earlier empirical research clearly demonstrate that movements in the price of oil predict subsequent macroeconomic fluctuations at business cycle frequencies (Salisu et al. 2021), we believe that the results documented in this research are of paramount importance for policymakers. In addition to the policy implications of our findings, the role of state-level economic conditions in predicting the volatility of oil price returns also assists in the portfolio allocation decisions of oil traders. Finally, we consider our observations to be important from the perspective of academics studying the determinants of fluctuations in oil prices. Our results clearly demonstrate that state-level economic activity, in addition to that associated with the U.S. economy considered as a single entity, should be added to the list of potentially influential determinants of the volatility of oil price returns in future research.

Recent studies by Bouri et al. (2021) and Gupta and Pierdzioch (2021b) highlight the role of global and climate risks of the overall U.S. in predicting oil price return volatility. Given the results documented by these researchers, as part of future research, it would be interesting to compare the relative importance of state-level climate risks with that of the aggregate U.S. in predicting the variability of movements of the price of crude oil, natural gas, and heating oil, because climate risks have been shown to drive state-level economic conditions in the U.S. (Sheng et al. 2022, forthcoming).

Appendix

See Tables 8 and 9.

Table 8 Dataset for state-level economic conditions index

Table 9 Dataset for U.S. economic conditions index