14.5 DENSITY COMBINATION

14.5 DENSITY COMBINATION

So far we have examined the combination of point forecasts generated by different models or forecasters. When individual forecasts are provided as densities rather than single points, we can also consider providing a combined—or pooled—density. Linear combinations of point forecasts take the form

$$f^c=\sum _{i=1}^m{\omega }_i{f}_i.$$

Similarly, linear combinations of density forecasts take the form

$$p^c(y)=\sum _{i=1}^m{\omega }_i:p_i(y),\text{\hspace{1em}(14.27)}$$

where $p_i(y)$ are the individual density forecasts for an outcome $y,$ and ${\omega }_i$ are the combination weights. To ensure that the combined density is itself a valid density, we require that it be nonnegative over the entire support of $\mathbf{y}$ and that it integrates to 1. We achieve this by restricting the weights to sum to 1, $\begin{array}{r}{\sum }_{i=1}^m{\omega }_i=1\end{array}$ , and by imposing that the weights are nonnegative, so $0\le {\omega }_i\le 1$ for all i. The combined density is thus a mixture of the individual densities. Authors such as Wallis (2005) consider equal weighting, so ${\omega }_i=1/m$ . In the statistical literature (14.27) is known as a linear prediction pool or linear opinion pool.11

The key difference between point forecasting and density forecasting is that whereas in point forecasting the outcome is observed, in density forecasting the density of the outcome is not observed—only a single draw from this outcome is observed. It is from this difference that most of the different approaches to density forecast combination arise.

The problem of combining distributions has been a topic in the statistics literature for some time. In the prequential approach of Dawid (1984), the provision of a predictive distribution for variables rather than parameter estimation is the goal of statistical inference. This approach has been highly influential and led to the development of many of the results in this chapter.

Denote the mean of $p_i(y)$ by ${\mu }_i$ and its variance by ${\sigma }_i^2$ . It follows from simple calculation in $(14.27)$ that $p^c(y)$ has mean $\begin{array}{r}{\mu }_c={\sum }_{i=1}^m{\omega }_i{\mu }_i\end{array}$ and variance $\begin{array}{r}{\sum }_{i=1}^{\widehat{m_{}}}{\omega }_i\left({\sigma }_i^2+({\mu }_c-{\mu }_i{)}^2\right)\end{array}$ . From this it can be seen that the combined density is centered between the individual densities. If the means of the individual distributions are disparate, the spread of the combined density will be wide, potentially even wider than that of any of the individual densities. This width represents the lack of information in the individual densities in pinpointing the best location of the actual density. If the means are all similar, the spread of the combined density will be somewhere in the middle of the spreads of the individual densities.

Clemen and Winkler (1999) discuss properties of linear aggregation methods. Linear aggregates of the form in (14.27) have the property of unanimity, which is that if all densities used in the combination give the same weight to an outcome, then so does the combined density.

We can also consider nonlinear density combinations, where $\begin{array}{r}{p}^c(y)=\end{array}$ $H(p_1(y),p_2(y),\dots ,p_m(y))$ . There are strong restrictions on the form that $H(\cdot )$ can take since we require $p^c(y)$ to be a valid density that is everywhere nonnegative and integrates to 1. A popular approach is the log combination method which sets

$$p^c(y)=(\int \prod _{i=1}^m{p}_i(y{)}^{{\omega }_i}d\mu {)}^{-1}\prod _{i=1}^m{p}_i(y{)}^{{\omega }_i},\text{\hspace{1em}(14.28)}$$

where $\mu$ is the underlying counting measure. This is guaranteed to keep the density nonnegative and ensures that it integrates to 1. See Genest and Zidek (1986) for further discussion of this approach.

Many of the issues that arise in the combination of point forecasts are also important to the combination of density forecasts. First, the individual density forecasts might be the only statistics observed. Alternatively, we might actually observe the models and data used to generate the density forecasts, in which case density model combination would be the more appropriate term. The majority of combination methods rely on some form of loss function over the densities. This again brings up the question of whether the underlying data should be used rather than the forecast densities constructed from different models. Ideally, the loss function should play some role in the construction of the density combinations.