14.7 EMPIRICAL EVIDENCE

14.7 EMPIRICAL EVIDENCE

Stock and Watson (1999) provide an extensive empirical comparison of the performance of a range of forecast methods, including linear and nonlinear ones, along with simple equal-weighted combined forecasts and combinations weighted by the inverse MSE values. The equal-weighted average of forecasts across all methods is found to produce the most attractive forecasts at 6- and 12-month horizons.

In their influential study of predictability of a large set of macroeconomic and financial variables, Stock and Watson (1999) conclude that “pooled forecasts were found to outperform the forecasts from any single method. . . . The pooling procedures that place weight on all forecasting methods (whether equal weighting, inverse

MSE weighting, or median) proved most reliable, while those that emphasized the recently best performing methods. . . proved least reliable.”

Similarly, using a seven-country data set to forecast output growth, Stock and Watson (2004) find that combination forecasts perform better than individual autoregressive forecasts. Marcellino (2004) studies forecast pooling methods across a very large set of European macroeconomic series. He finds that pooled forecasts are often beaten by forecasts generated by nonlinear models although the combined forecasts perform well for a number of variables such as industrial production growth, inflation, and unemployment.

Related findings have been reported not just for model combination, but also for combinations of survey forecasts. For example, Genre et al. (2013) consider a range of combination schemes for the ECB Survey of Professional Forecasters and find little evidence that the simple equal-weighted average of survey forecasts is bettered by combinations using principal components, trimmed means, past-performance-based weighting, least squares estimates of the combination weights, or Bayesian shrinkage weights.

Bayesian model averaging has been used by Wright (2009) to forecast US inflation. Wright considers linear models that include the lagged inflation rate in addition to one predictor variable. Across 107 possible macroeconomic predictor variables Wright finds that BMA forecasts are more accurate than forecasts from equalweighted combinations at longer horizons between one and two years, whereas they yield virtually identical results at shorter horizons of one through three quarters.

BMA methods have also been used to predict exchange rates (Wright, 2008) and stock returns by means of large sets of prediction models; see, e.g., Avramov (2002), Cremers (2002), and Elliott, Gargano, and Timmermann (2013). These studies typically combine all possible forecast models generated from K possible predictor variables, yielding $2^K$ possible specifications.