13.8 CONCLUSION

13.8 CONCLUSION

A stylized feature of many economic and financial time series is the presence of persistent volatility dynamics. Beginning with the ARCH model of Engle (1982), a large set of methods have been developed to incorporate such dynamics in density forecasts. Techniques have been developed to account for volatility persistence even for multivariate models of large dimension. As always, there is a bias–variance tradeoff when deciding whether to use flexible and less biased semi- and nonparametric techniques for estimating density forecasting models versus using parametric models which may reduce the effect of estimation errors on the density forecast but lead to biased forecasts. And, once again, the trade-off involved in the choice between different estimators will ultimately depend on the forecaster’s loss function.

Density forecasting has gained increased prominence in many areas of economic forecasting. It is now common for large institutions such as the Bank of England and the IMF to go beyond reporting simple point forecasts to also including indications of the uncertainty surrounding such forecasts. This requires a model for the full distribution of possible future outcomes. Density forecasting is also used in risk management and investment decisions as a way to track likely portfolio losses and determine optimal portfolio allocation given trade-offs between risk and returns.

Given the wide application of volatility and density forecasting in economics and finance, it is worth emphasizing that this is an area where tremendous progress has been made over the last couple of decades. We now possess sophisticated methods that allow us to generate predictive distributions for a wide variety of variables. How to assess the accuracy of such density forecasts is the topic of chapter 18.

Forecast Combinations

The idea of combining multiple forecasts of the same outcome is intuitively appealing. Individual models are likely to be misspecified and it can often be difficult to identify a single best or “dominant” forecasting model. With two available forecasts, $f_1$ and $f_2,$ , it is thus quite likely that the combined forecast $f^c(f_1,f_2)$ generates a lower expected loss than the individual forecasts:

$$E\left[L(f_i,y)\right]>\underset{f^c(\cdot )}{\min }E\left[L(f^c(f_1,f_2),y)\right],\text{\hspace{1em}(14.1)}$$

for $i=1,2$ . The notation $f^c(f_1,f_2)$ is shorthand for $f^c(f_1(z),f_2(z))$ and $f^c$ refers to the mapping from the two individual forecasts, $f_1$ and $f_2,$ to the combined forecast. Forecast combination is essentially a model selection and parameter estimation problem that arises as a special case of the issues discussed in previous chapters— albeit a case in which special constraints on the estimation problem play an important role.

Why combine forecasts in the first place? Perhaps one of the oldest understandings in statistics is that when faced with multiple measurements of the same outcome, averaging is a better approach than choosing a single estimate. Empirically, many models or forecasts are often found to have similar predictive accuracy. Taking averages or weighted averages of such forecasts is likely to improve predictive accuracy. If a single model does not separate itself from a larger set of models by producing clearly superior forecasting performance, it makes sense to expect individual models to contribute information over and above what other models offer.

Situations where forecasts are combined can be divided into two cases—those where the data underlying the forecasts are not observed and those for which such data are observed. When the underlying data are not observed, the forecaster can simply treat the observed forecasts as data in the hopes of finding a combination that improves the forecast. This is the traditional view of forecast combination and arises, for example, when forecasts come from surveys. Typically the forecaster will not have access to the underlying information sets used by survey respondents and so the only feasible strategy is to treat the individual forecasts like any other conditioning information and estimate the best possible mapping from the individual forecasts (possibly augmented with other information) to the outcome.1

When the data underlying the model forecasts are observed, “model combination” is a more accurate description of the statistical problem. This case typically arises when the researcher constructs forecasts from different models, as in the study by Stock and Watson (1999). It may seem unreasonable to combine forecasts rather than simply gather the information used to construct those forecasts and use standard model procedures to directly map the underlying data to the forecasts. Using a middle step of first constructing forecasts does limit the flexibility of the final forecasting model and hence could increase the risk that the combined forecast does not optimally use all available information. However, model combination might still be attractive. For example, it may not be possible or even reasonable to construct the underlying data. This situation is well summarized by Diebold and Pauly (1990): “While pooling of forecasts is suboptimal relative to pooling of information sets, it must be recognized that in many forecasting situations, particularly in real time, pooling of information sets is either impossible or prohibitively costly.”

Moreover, since estimation error plays a role in the final risk of any given forecasting method, model combination yields a different risk function which, through its more parsimonious use of the data, could be attractive to the forecaster. In this sense the combined forecast can be viewed simply as a different estimator of the final model, with a different—and perhaps better—risk for some parts of the parameter space. All estimation methods available when the underlying data are observed can be used in model combination. Researchers can therefore also utilize the underlying data in estimating model combination weights or as part of an extended regression model.

Forecast combination is closely related to the notion of forecast encompassing examined in chapter 17. In the context of equation (14.1), forecast $f_1$ encompasses forecast $f_2$ if the minimization results in no role being played by $f_2$ in the optimal combination; in this case (14.1) holds as an equality rather than as an inequality for i = 1. Tests for encompassing can be seen as tests for exclusion of a forecast from a forecast combination, or equivalently testing that there are no gains from taking a weighted average of the forecasts.

Conversely, if a decision is made to combine forecasts, what should be combined? Most obviously, if the information sets underlying the individual forecasts are unobserved, a generalized or “nesting” model cannot be constructed and so it is particularly likely that individual forecasts contribute distinct information. Combinations of forecasts based on different approaches—e.g., linear versus nonlinear forecasts— make it more likely that the combined forecast will be robust to changes in the datagenerating process. A third possibility is to combine survey forecasts with forecasts from econometric models. Being based on subjective judgment, survey forecasts may better incorporate true forward-looking information, while formal econometric models have the advantage that they efficiently incorporate historical information.

The outline of the chapter is as follows. Section 14.1 considers the theory on optimal forecast combination. Section 14.2 introduces a variety of estimation schemes that have been proposed in the forecast combination literature. This section also examines the “forecast combination puzzle,” i.e., the frequent empirical finding under MSE loss that averaging the forecasts is difficult to improve on. Section 14.4 examines classical model combination methods, section 14.5 examines classical density combination methods, while section 14.6 examines Bayesian model averaging which can provide both density and point forecasts. Section 14.7 provides an empirical example. Section 14.8 concludes.