14.2.1 Estimation of Forecast Combination Weights under MSE Loss

14.2.1 Estimation of Forecast Combination Weights under MSE Loss

Many different ways exist for estimating the combination weights under MSE loss. The existence of such an extensive list of methods boils down to a number of previously discussed issues in constructing forecasts, namely, the role of estimation error, the lack of a single optimal estimation scheme, along with the empirical observation that simple methods are difficult to beat in practice.

A common baseline is to use a simple equal-weighted average of the forecasts:

$$f^{\mathrm{ew}}=\frac{1}{m}\sum _{i=1}^m{f}_i.\text{\hspace{1em}(14.12)}$$

An advantage of this approach is that it generates no estimation error since the combination weights are imposed rather than estimated. This has obvious intuitive appeal given the restrictions usually imposed on the combination weights. The method also has appeal if, when constructing forecast combinations over time, the panel of forecasts being combined changes dimension or if only a very short time series of the forecasts is available. Concerns about outlier forecasts can be addressed by using the median forecast or the trimmed mean—assuming that m is sufficiently large—instead of the mean.

The trimmed mean works as follows. Suppose the forecasts have been ranked from smallest to largest $f_1,\dots ,f_m$ . Trimming a proportion $\rho$ of the smallest and largest forecasts, the trimmed mean is computed as

$$f^{\mathrm{trim}}=\frac{1}{\lfloor m(1-2\rho )\rfloor }\sum _{j=\lfloor \rho m\rfloor +1}^{\lfloor m(1-\rho )\rfloor }{f}_j,\text{\hspace{1em}(14.13)}$$

where $\lfloor .\rfloor$ rounds to the nearest (smaller) integer.

A natural alternative to these simple robust estimation methods is to use sample estimates for the population-optimal weights in (14.9). Bates and Granger (1969) suggest simply replacing the unknown variances and covariance in the formula for the optimal weights (14.9) with the equivalent sample estimates. More generally, for an $(m\times 1)$ vector of forecasts, the estimated variance–covariance matrix could be used in the formula for the combination weights (14.9). This plug-in solution turns out to be numerically identical to the restricted least squares estimator of the weights from a regression of the outcome on the forecasts and no intercept subject to the restriction that the coefficients sum to 1 $({\hat{\omega }}_{ols}^{\prime }{\iota }_m=1)$ :

$$\begin{array}{r}{f}^c={\hat{\omega }}_{\mathrm{ols}}^{\prime }f=({\iota }^{\prime }{\hat{\Sigma }}_e^{-1}\iota {)}^{-1}{\iota }^{\prime }{\hat{\Sigma }}_e^{-1}f,\end{array}\text{\hspace{1em}(14.14)}$$

where $f$ is the vector of forecasts and ${\hat{\Sigma }}_{\epsilon }$ is the usual sample estimator of the covariance matrix, $E\left[e{e}^{\prime }\right],e.g.,T^{-1}{\sum }_{t=0}^{T-1}{e}_{t+1}{e}_{t+1}^{\prime }$

To see the equivalence between the Bates and Granger approach and restricted least squares, consider the linear regression of the outcome on the forecasts:

$$y=f^{\prime }\omega +\epsilon .\text{\hspace{1em}(14.15)}$$

Rearranging, we have

$$\begin{array}{rl}& y=(f-y{\iota }_m^{\prime }+y{\iota }_m^{\prime })\omega +\epsilon \\ & =-e\omega +y{\iota }_m^{\prime }\omega +\epsilon ,\\ s\circ & (1-{\omega }^{\prime }{\iota }_m)y=-e\omega +\epsilon ,\end{array}$$

where e is the vector of forecast errors. Under the restriction that ${\omega }^{\prime }{\iota }_m=1$ , it follows that $\epsilon =e\omega$ . Minimizing the sum of squared residuals subject to the restriction that the weights sum to 1 is therefore the same as minimizing ${\omega }^{\prime }{e}^{\prime }e\omega$ , and thus gives the same result as Bates and Granger (1969); see Granger and Ramanathan (1984).3 If there is suspicion that the individual forecasts are biased, a constant can be included in the regression.

Variants of least squares combination weights, much like variants of linear forecasting models with general data, revolve around attempts to reduce risk over some parts of the space of models, where the risk arises from the estimation error involved in constructing the weights. One possibility is to use Bayesian or empirical Bayes methods to estimate the combination weights. Clemen and Winkler (1999) propose one such approach that relies on the use of an inverse Wishart distribution on ${\Sigma }_e$

Different assumptions on priors and model parameters yield different estimators. Diebold and Pauly (1990) suggest both Bayesian and empirical Bayes shrinkage-type methods. Using the results in chapter ${}^{5,}$ it follows that for a Gaussian model and Gaussian priors the Bayesian estimator yields forecast combination weights formed as a weighted average of the prior, ${\omega }_P,$ , and the least squares estimates, ${\hat{\omega }}_{\mathrm{ols}}$ . As noted above, the natural uninformative prior is to assign equal weights to the forecasts, so suppose that the prior weights on the m forecasts are ${\omega }_P={\iota }_m/m$ . Under the g -prior of Zellner (1986b) discussed in chapter 5, where $g$ is a scalar prior that controls the degree of shrinkage with large values of $g$ implying high precision and strong shrinkage towards the prior mean of ${\iota }_m/m$ , the posterior mean of the combination weights becomes

$$\hat{\omega }=m^{-1}{\iota }_m+\frac{1}{1+g}\left({\hat{\omega }}_{\mathrm{ols}}-m^{-1}{\iota }_m\right),\text{\hspace{1em}(14.16)}$$

where ${\hat{\omega }}_{\mathrm{ols}}$ is the OLS estimate of ω in (14.14). More generally, assuming normal priors on ω, $\mathrm{N}({\omega }_p,{\tau }^2(Z_f^{\prime }{Z}_f{)}^{-1})$ , where $Z_f$ is the $(T\times m)$ matrix of regressors in (14.14), Diebold and Pauly (1990) use empirical Bayes combination weights given by

$$\hat{\omega }={\omega }_P+\frac{{\hat{\tau }}^2}{{\hat{\sigma }}^2+{\hat{\tau }}^2}\left({\hat{\omega }}_{\mathrm{ols}}-{\omega }_P\right),\text{\hspace{1em}(14.17)}$$

where

$$\begin{array}{rl}& {\hat{\sigma }}^2=(y-Z_f{\hat{\omega }}_{\mathrm{ols}}{)}^{\prime }(y-Z_f{\hat{\omega }}_{\mathrm{ols}})/T,\\ & {\hat{\tau }}^2=({\hat{\omega }}_{\mathrm{ols}}-{\omega }_{\hat{P}}{)}^{\prime }({\hat{\omega }}_{\mathrm{ols}}-{\omega }_{\hat{P}})/\mathrm{tr}(Z_f^{\prime }{Z}_f{)}^{-1}-{\hat{\sigma }}^2,\end{array}$$

where ${\hat{\sigma }}^2$ is an estimator of the variance of the residuals in the regression model, ${\sigma }^2$ . Equation (14.17) is similar to a shrinkage-type estimator for the combination weights.

Between data-free schemes such as equal weighting and methods attempting to approximate the optimal combination weights are methods that utilize data in the forecast combination scheme without attempting to asymptotically obtain the optimal combination weights. The simplest example is the trimmed mean in (14.13) which discards some fraction of the largest and smallest forecasts before taking the mean of the remaining forecasts.

Another frequently used approach simply ignores correlations across forecast errors and uses weights that are proportional to the inverse of the individual models’ MSE values, MSEi :

$${\omega }_i=\frac{{\mathrm{MSE}}_i^{-1}}{{\sum }_{i=1}^m{\mathrm{MSE}}_i^{-1}}.\text{\hspace{1em}(14.18)}$$

A variant of this method is considered by Aiolfi and Timmermann (2006) who propose a robust weighting scheme that lets the combination weights be inversely proportional to the forecast models’ rank, Ranki :

$${\omega }_i=\frac{{\mathrm{Rank}}_i^{-1}}{{\sum }_{i=1}^m{\mathrm{Rank}}_i^{-1}}.\text{\hspace{1em}(14.19)}$$

Here the model with the lowest MSE value gets a rank of 1, the model with the second lowest MSE performance gets a rank of 2, and so forth. This combination scheme again ignores correlations across forecast errors.

Aiolfi and Timmermann (2006) also consider a factor-based clustering approach. If the outcome variable has a factor structure and the individual forecasts can be clustered according to which factors they track, in some cases little is lost by pooling forecasts within clusters. Their approach is to identify a small set of clusters, form equal-weighted forecasts within each cluster, and then apply least squares combination methods such as (14.14) to these pooled forecasts.