14.2.2 Estimation Methods for Time-Varying Combination Weights

14.2.2 Estimation Methods for Time-Varying Combination Weights

If the joint distribution of the outcome and the forecasts varies through time, it seems attractive to let the combination weights change over time so that they can adapt to such changes. Bates and Granger (1969) propose several adaptive estimation schemes. One approach is to use a rolling window estimate of the models’ forecasting performance over the most recent v observations:

$${\hat{\omega }}_{it}=\frac{{\left({\sum }_{\tau =t-v+1}^t{e}_{i\tau }^2\right)}^{-1}}{{\sum }_{j=1}^m{\left({\sum }_{\tau =t-v+1}^t{e}_{j\tau }^2\right)}^{-1}},\text{\hspace{1em}(14.20)}$$

where $e_{i\tau }$ is the i th model’s forecast error at time τ . Equation (14.20) is of course just a local estimate of the weights in (14.18). Using a short value for v means putting more weight on the models’ recent track record. To account for the shorter data available for estimation, (14.20) simplifies the estimation problem by ignoring correlations between forecast errors.

Alternatively, correlations in forecast errors can be accounted for by using the following rolling window estimation scheme based on (14.9):

$$\begin{array}{c}{\hat{\omega }}_t={\hat{\Sigma }}_{et}^{-1}{\iota }_m/({\iota }_m^{\prime }{\hat{\Sigma }}_{et}^{-1}{\iota }_m),\\ {\hat{\Sigma }}_{et}[i,j]=v^{-1}\sum _{\tau =t-v+1}^t{e}_{i\tau }{e}_{j\tau }^{\prime }.\end{array}\text{\hspace{1em}(14.21)}$$

A third adaptive updating scheme uses a smoothing parameter $\lambda \in (0;1)$ to discount older forecasting performance:

$${\hat{\omega }}_{it}=\lambda {\hat{\omega }}_{it-1}+(1-\lambda )\frac{{\left({\sum }_{\tau =t-v+1}^t{e}_{i,\tau }^2\right)}^{-1}}{{\sum }_{j=1}^m{\left({\sum }_{\tau =t-v+1}^t{e}_{j,\tau }^2\right)}^{-1}}.\text{\hspace{1em}(14.22)}$$

The closer to unity is λ, the smoother the combination weights will be.

Yang (2004) proposes an algorithm that aims to identify the best forecasts by minimizing a weighted MSE measure. Assuming equal priors, normal errors, and recursively updated error variance estimates $\begin{array}{r}{\hat{\sigma }}_{it}^{2^{-}}(\mathrm{e}.\bar{\mathrm{g}}.,{\hat{\sigma }}_{it}^{\hat{2}}={\sum }_{\tau =1}^{t-1}{e}_{i\tau }^2/(t-1))\end{array}$ the weights of the aggregated forecast through exponential reweighting (AFTER) algorithm are

$${\omega }_{it}=\frac{{\omega }_{i1}\left({\prod }_{\tau =1}^{t-1}{\hat{\sigma }}_{i\tau }^{-1}\right)\exp \left(-\frac{1}{2}{\sum }_{\tau =1}^{t-1}{e}_{i\tau }^2/{\hat{\sigma }}_{i\tau }^2\right)}{{\sum }_{j=1}^m\left({\omega }_{j1}{\prod }_{\tau =1}^{t-1}{\hat{\sigma }}_{j\tau }^{-1}\exp \left(-\frac{1}{2}{\sum }_{\tau =1}^{t-1}{e}_{j\tau }^2/{\hat{\sigma }}_{j\tau }^2\right)\right)},\text{\hspace{1em}(14.23)}$$

where $i=1,\dots ,m$ and $\tau \ge 1$ . Note that this scheme weights the forecast errors relative to the estimated variance ${\hat{\sigma }}_{it}^2$ with weights that grow larger, the smaller the value of ${\hat{\sigma }}_{it}^2$ . This also means that forecast errors that are large relative to the expected variance ${\hat{\sigma }}_{it}^2$ can lead to substantial declines in the weights. In an empirical study of the Survey of Professional Forecasters, Lahiri, Peng, and Zhao (2013) find that this method adjusts weights on individual survey participants’ forecasts more aggressively following unexpectedly large squared forecast errors compared with least squares methods such as (14.14).

Forecast combinations can work well empirically because they provide insurance against model instability or the types of extraneous structural breaks considered by Hendry and Clements (2004). Empirically, Elliott and Timmermann (2005) allow for regime switching in combinations of forecasts from surveys and time-series models and find strong evidence that the relative performance of the underlying forecasts changes over time. Aiolfi and Favero (2005) study differences in forecasts of stock returns across multiple models and relate them to model uncertainty.

Further illuminating this point, the performance of combined forecasts tends to be more stable than that of individual forecasts used in the empirical combination study of Stock and Watson (2004). Interestingly, combination methods that attempt to explicitly model time variation in the combination weights often fail to perform well, suggesting that regime switching or model “breakdown” can be difficult to predict or even track through time.