Robust monitoring machine: a machine learning solution for out-of-sample R $$^2$$ -hacking in return predictability monitoring

Financial Innovation

Table 1 Out-of-sample $\mathbf {R^2}$ $\mathbf {(\%)}$ of Traditional Combination Forecasts

	First year in evaluation sample
	1947	1957	1967	1977	1987	1997	2007
Proposed forecast: $f^{(a)}$	0.50	0.37	0.36	0.14	− 0.09	− 0.10	− 0.24
Alternative combination forecasts
Median	0.40	0.37	0.38	0.21	0.09	0.08	0.04
DMSFE (60 months, $\delta =1.0$)	0.50	0.37	0.37	0.15	− 0.08	− 0.09	− 0.24
DMSFE (24 months, $\delta =1.0$)	0.49	0.36	0.37	0.14	− 0.04	− 0.03	− 0.19
DMSFE (12 months, $\delta =1.0$)	0.56	0.43	0.42	0.18	− 0.03	− 0.00	− 0.14
DMSFE (1 month, $\delta =1.0$)	1.17	1.09	1.18	1.13	− 0.31	− 0.34	− 1.26
DMSFE (60 months, $\delta =0.5$)	0.57	0.45	0.43	0.14	− 0.08	− 0.01	− 0.08
DMSFE (24 months, $\delta =0.5$)	0.57	0.45	0.43	0.14	− 0.08	− 0.01	− 0.08
DMSFE (12 months, $\delta =0.5$)	0.57	0.45	0.43	0.14	− 0.08	− 0.01	− 0.08

This table reports the forecasting performance of the proposed forecast and its variations, in terms of out-of-sample $R^2$ (%). The main proposed forecast is a equal-weighted combination forecast from Rapach et al. (2010). Its variational forms include a combination as a median and combinations using discounted mean square forecast error (DMSFE) from Stock and Watson (2004). We explore different holdout windows of 1, 12, 24 and 60 months and discount factors of 0.5 and 1. Each column corresponds to a different sample split year from 1947 to 2007. All evaluation period ends in December 2017