10.5.1 Factor Models versus Bayesian VARs

10.5.1 Factor Models versus Bayesian VARs

In settings with large sets of predictor variables, empirical researchers have traditionally used factor models to generate forecasts. However, as we discussed in chapter 9, studies such as Banbura, Giannone, and Reichlin (2010) develop Bayesian VAR methods that work even with large sets of conditioning variables, e.g., more than 100. Empirically, they find that their Bayesian VARs produce better forecasts than factor-based approaches. Consistent with this, Koop (2013) finds that a common factor approach often produces worse forecasts than Bayesian VARs although the ranking depends on how the trade-off between incorporating more information and dampening the effect of estimation error is implemented across different Bayesian methods.

This highlights important trade-offs between factor models and Bayesian VARs. The performance of the Bayesian VARs can be quite sensitive to the choice of prior and it is clear that the same prior, and thus the degree of shrinkage, does not work well across different economic variables. For some dependent variables, small-scale BVARs work well, corresponding to ignoring the information contained in a large set of predictors. For other dependent variables, it can be important to condition on a larger set of predictors. One important advantage of Bayesian VARs is that they are set up to produce density forecasts as opposed to simple point forecasts. While density forecasts can also be generated from factor models, they require additional assumptions and will typically not account for parameter estimation error in the same way as the Bayesian approach does.

10.5.2 Application

To illustrate the factor approach empirically, we constructed a data set containing 65 variables capturing different aspects of the US economy. The variables include several categories such as industrial production growth, capacity utilization, personal consumption expenditures, retail sales, unemployment and payroll data, housing starts, inventories, inflation, average hourly earnings, commodity and stock price indexes, interest rates, and money supply. In each case we transform the series so it is stationary. The data are monthly and run from 1948:1 to 2010:12.

Figure 10.1 plots time series of the first four principal components extracted from this data set. The first three factors are quite persistent with occasional large shifts in levels, particularly for the second and third factor, while the fourth factor is far less persistent.

We next consider the stability of the factor approach when applied recursively through time. To this end, we report both the proportion of the total variation in the panel of predictor variables that can be explained by the first four factors, the number of factors selected by the forecasting models, along with the forecasts based on the factor models, and their resulting out-of-sample performance.

Figure 10.2 plots the fraction of the explained variance over time. The first factor accounts for close to 50% of the variance, while factor two accounts for between 20 and 25%. The third factor accounts for between 15 and 20% of the variation, while the fourth factor accounts for close to 12%. Thus, while the proportion of the variance explained by the first and fourth factors is quite stable through time, it is less stable for the second and third factors, although the total variation explained jointly by these two factors is fairly constant through time, adding up to a little less than 40%.

Figure 10.1: First four common factors extracted from a sample of macroeconomic and financial variables.

Figure 10.2: Recursive estimates of the proportion of the common variation explained by the first four factors.

Figure 10.3: Number of factors chosen by the Bai and Ng selection criteria applied recursively through time.

For the pure factor models, figure 10.3 shows that the two information criteria $\mathrm{I}{\mathrm{C}}_1$ and ${\mathrm{IC}}_2$ select a fairly modest number of common factors which varies between one and three.

Figures 10.4 and 10.5 show monthly forecasts for the inflation rate and unemployment rate series. We use the sample 1948:01–1969:12 for initial parameter estimation and the period 1970:01–2010:12 to evaluate the forecasts out-of-sample.

We show results for pure factor models including either models that include one or four factors, a VAR, a dynamic factor model, and two Ridge regressions with small $({\lambda }_1=300)$ and large $({\lambda }_2=5,000)$ penalty terms, respectively. For each variable, the single factor forecast is generally quite smooth, particularly when compared against the forecast with multiple factors. The VAR and dynamic factor model (DFM) forecasts are nearly identical, suggesting that once autoregressive terms are included, the factor terms are relatively less important. Among the Ridge forecasts, the effect of applying a heavy penalty term is to substantially smooth the forecasts.

Table 10.1 reports the forecasting performance of factor models selected by $\mathrm{I}{\mathrm{C}}_1$ or ${\mathrm{IC}}_2$ in addition to principal components models with one or four factors, Ridge regressions, a VAR(1), a VAR with lag length selected by the AIC, dynamic factor models, and random walk and prevailing mean benchmarks.

Figure 10.4: Recursive forecasts of the inflation rate using different combinations of the factors (top), a VAR and a factor-augmented VAR, and Ridge regression (bottom).

For the inflation rate series, the best models are the univariate autoregressive models closely followed by the VAR and DFM specifications, all of which beat the random walk benchmark. The pure factor models do comparatively worse with RMSE values above those of the best models, although the model that includes four factors still beats the random walk benchmark. This highlights the importance of including autoregressive terms for many macroeconomic variables. The performance of the Ridge regressions depends on the value of the penalty or shrinkage factor with weak amounts of shrinkage working best for this series. For stock returns, the Ridge regression with the largest amount of shrinkage is the only method that performs better than the benchmark prevailing mean method, although a number of approaches register RMSE performance similar to this benchmark. For stock returns, the most parsimonious models generate the most accurate forecasts, consistent with earlier findings for this variable.

The best unemployment rate forecasts come from the dynamic factor models, followed closely by the AR and VAR forecasts. Pure factor-based forecasts perform poorly again as they fail to pick up the very high persistence in this series. A similar conclusion holds for the T-bill rate, for which the AR, VAR, and dynamic factor models perform best, while the pure factor and Ridge models produce poor out-ofsample forecasts.

Table 10.2 reports Diebold–Mariano test statistics comparing the mean squared error performance of the forecasts in the previous table against a benchmark model. For persistent variables such as the inflation rate, unemployment, and the T-bill rate, the random walk model is a reasonable benchmark. Conversely, for

Figure 10.5: Recursive forecasts of the unemployment rate using different combinations of the factors (top), a VAR, a dynamic factor model, and Ridge regression (bottom).

TABLE 10.1:

Out-of-sample root mean squared forecast error performance for different models used to predict the monthly inflation rate, stock returns, unemployment rate, and the short interest rate. AR refers to univariate autoregressive models with lag length selected by the AIC or BIC, Ridge refers to Ridge regression, PC refers to principal components methods with the number of factors selected by the information criteria of Bai and $\mathrm{Ng}$ (2002) $({\mathrm{IC}}_1$ and ${\mathrm{IC}}_2)$ , one principal component $\left({\mathrm{PC}}_1\right)$ , four principal components $\left({\mathrm{PC}}_{1-4}\right)$ , vector autoregressions (VAR(AIC) and VAR(1)), and two dynamic factor models (DFM(AIC), DFM(1)).

TABLE 10.2:
Diebold–Mariano test statistics for quarterly out-of-sample mean squared error performance measured relative to the random walk benchmark for the inflation, unemployment, and interest rate series and relative to the prevailing mean benchmark for stock returns.

stock returns that are not serially correlated, a better benchmark is the prevailing mean. Once again, for the stock return series there is no evidence that any of the forecasting methods is capable of beating the prevailing mean forecast’s mean squared error performance. In contrast, the dynamic factor models in particular generate substantially more accurate point forecasts than the random walk forecasts for the inflation, unemployment, and interest rate series. For the inflation and unemployment rate series there is also evidence that the univariate AR models beat the benchmark random walk.

These results indicate that, for the data considered here, once autoregressive terms have been included in the forecasting model, there is little evidence that adding factors improves on the accuracy of the forecasts.