10.5 EMPIRICAL EVIDENCE

10.5 EMPIRICAL EVIDENCE

We finally review the empirical evidence on forecasting with factor models and present an empirical application. Spurred by the important progress made in our theoretical understanding of the properties of factor models accompanied by easier access to vast data sets, recent years have seen a rapid rise in the number of empirical forecasting studies that use factor models.

Before including factors in a prediction model, it is important to ask whether the factors summarize the essence of the common variation in a given set of variables. In most empirical applications, a small number of factors seem to summarize the common factors well. For example, Giannone, Reichlin, and Sala (2005) report that a small number of factors account for a large part of the variance of most macroeconomic variables. To the extent that the role of the factors is to capture common fluctuations, this suggests as a general rule to keep the number of factors small. However, it should also be kept in mind that the ability of a common factor to explain the common variation in the conditioning variables need not be a good measure of its importance as a predictor for any one particular series.

The empirical success of factor models seems to vary significantly across different types of economic variables and it is a common finding that gains in out-of-sample predictive accuracy for factor models are stronger for real as opposed to nominal variables; see Stock and Watson (2011). For example, for real variables such as growth in industrial production, personal income, trade, or employment, Stock and Watson (2002a) find that using two common factors results in considerable improvements in predictive accuracy over simply using a univariate autoregressive specification with lag length selected by the BIC. Since the difference between real and nominal variables is the rate of inflation, this suggests that factor models have less to offer when it comes to forecasting inflation.

Pesaran, Pick, and Timmermann (2011) find that factor-augmented VAR forecasts outperform univariate forecasts at short horizons of one and three months and across both short and long horizons for four out of the five categories of economic variables studied by Marcellino, Stock, and Watson (2006). This suggests that, for most economic variables and particularly at short horizons, information beyond what is contained in the past history of the variables themselves can be helpful in producing better forecasts.

While factor models can count some important empirical successes, an important and related issue is whether there are systematic gains in predictive accuracy from using more sophisticated factor approaches. For example, given the choice between static and dynamic factor models, which approach should a researcher adopt? Empirically, Boivin and Ng (2005) find that the two approaches yield very similar forecasts and that no single approach is obviously dominant provided that the parameters are appropriately determined. Pragmatic reasons would seem to argue in favor of the approach that is easiest to implement, i.e., the static factor approach. Along similar lines, Boivin and Ng (2005) and D’Agostino and Giannone (2006) find empirically that different factor estimates lead to similar forecasts with comparable out-of-sample forecasting performance.

Stock and Watson (2012) compare forecasts from a model with five principal components to forecasts based on pre-tests, Bagging and Bayesian model averaging methods, all of which they show can be viewed as generalized shrinkage methods. Comparing forecasts across 143 macroeconomic series over the period 1985–2008, they find little evidence that the generalized shrinkage methods improve in notable ways over the dynamic factor models. Interestingly, they also find that the dynamic factor models fail to improve on a simple AR(4) model for more than half of the time series, although there is evidence of stronger forecasting performance of the dynamic forecasting model relative to this univariate benchmark in a longer sample 1960– 2008 that includes the period prior to the Great Moderation which starts around 1984.

Dobrev and Schaumburg (2013) consider robust forecasting methods that use regularization and reduced rank regression methods. They find that their method performs well relative to the five-factor principal components regression method of Stock and Watson (2012) and also relative to partial least squares methods.