19.1 BREAKS AND FORECASTING PERFORMANCE

19.1 BREAKS AND FORECASTING PERFORMANCE

To see how forecasts get affected when parameters undergo change, let ${\beta }_t$ be the regression parameters of the forecasting model where the \" { t } ^ { \ast } subscript indicates that the parameters now vary over time. Assuming that parameter changes are not random, we would prefer to construct forecasts using the parameters, ${\beta }_T,$ , at the point of the forecast (T ) which is typically at the end of the historical sample. Instead, the estimator based on full-sample information, ${\hat{\mathbf{\beta }}}_T$ , will typically converge not to ${\beta }_T$ but, rather, to the average value for the ${\beta }_t$ -values over the estimation period $t=1,\dots ,T$

Example 19.1.1 (Linear regression model with a single break). Consider the model

$$y_{t+1}={\beta }_t^{\prime }{x}_t+{\epsilon }_{t+1},t=1,\dots ,T-1,\text{\hspace{1em}(19.1)}$$

where ${\epsilon }_{t+1}$ are i.i.d. $(0,{\sigma }^2)$ and are independent of $\{x_t{\}}_{t=1}^T$ . Suppose that ${\beta }_t=\beta +d\times$ $1(t\le \tau )$ so the regression parameters change from $\beta +d$ before the break to $\beta$ after time τ . To generate a forecast of ${\gamma }_{T+1}$ at time T we would prefer to use the model ${\beta }^{\prime }{x}_T$ . However, the conditional expectation of the full-sample least squares estimates, ${\hat{\mathbf{\beta }}}_T$ , obtained by regressing $y_{t+1}$ on xt for $t=1,\dots ,T-1$ , conditional on $\{x_t{\}}_{t=1}^T$ , is equal to

$$\begin{array}{rl}& E[{\hat{\beta }}_T]=E\left[{\left(\sum _{t=1}^{T-1}{x}_t{x}_t^{{}^{\prime }}\right)}^{-1}\left(\sum _{t=1}^{T-1}{x}_t{y}_{t+1}\right)\right]\\ & =E\left[{\left(\sum _{t=1}^{T-1}{x}_t{x}_t^{{}^{\prime }}\right)}^{-1}\left(\sum _{t=1}^{T-1}{x}_t\left(x_t^{{}^{\prime }}\beta +x_t^{{}^{\prime }}d1(t\le \tau )+{\epsilon }_{t+1}\right)\right)\right]\\ & =\beta +{\Sigma }_{X,T-1}^{-1}{\Sigma }_{X,\tau }d,\end{array}\text{\hspace{1em}(19.2)}$$

where $\begin{array}{r}{\Sigma }_{X,t}={\sum }_{s=1}^t{x}_s{x}_s^{\prime }\end{array}$ . If $E\left[{\Sigma }_{X,t}\right]=t{\Sigma }_X$ for all t, then (19.2) reduces to $\beta +$ $(\tau /(T-1))d$ . In general, the parameters of the forecasting model will be biased by ${\Sigma }_{X,T-1}^{-1}{\Sigma }_{X,\tau }d$ and the bias gets larger the larger the break size, d, and the later in the sample the break occurred (τ ).

Model instability can give rise to a sudden “breakdown” in forecasting performance because the parameters of the data-generating process may have shifted while the estimated parameters put a heavy weight on data occurring prior to such a change, generating biased forecasts. For most loss functions such biases will increase the loss; for example, the expected loss is increased by the squared bias under MSE loss.

Example 19.1.2 (Linear regression with a single break, continued). Using the setup from Example 19.1.1, note that, conditional on $\{x_t{\}}_{t=1}^T$

$$\begin{array}{rlr}& & {\gamma }_{T+1}-{\hat{\beta }}_T^{\prime }{x}_T={\epsilon }_{T+1}-({\hat{\beta }}_T-\beta {)}^{\prime }{x}_T\\ & & ={\epsilon }_{T+1}+x_T^{\prime }{\Sigma }_{X,T-1}^{-1}{\Sigma }_{X,\tau }d+\left({\sum }_{t=1}^{T-1}{\epsilon }_{t+1}{x}_t^{\prime }\right){\Sigma }_{X,T-1}^{-1}{x}_T\\ & & \sim \left(x_T^{\prime }{\Sigma }_{X,T}^{-1}{\Sigma }_{X,\tau }d,{\sigma }^2\left(1+x_T^{\prime }{\Sigma }_{X,T-1}{x}_T\right)\right).\end{array}$$

It follows from this that the MSE is given by

$$\begin{array}{r}E[y_{T+1}-{\hat{\beta }}_T^{\prime }{x}_T{]}^2=d^{\prime }{\Sigma }_{X,\tau }{\Sigma }_{X,T}^{-1}{x}_T{x}_T^{\prime }{\Sigma }_{X,T}^{-1}{\Sigma }_{X,\tau }d+{\sigma }^2\left(1+x_T^{\prime }{\Sigma }_{X,T-1}{x}_T\right).\end{array}\text{\hspace{1em}(19.3)}$$

Assuming $E\left[{\Sigma }_{X,t}\right]=t{\Sigma }_X$ for all t and taking expectations over $\{x_t{\}}_{t=1}^T$ , then (19.3) simplifies to $(\tau /(T-1){)}^2{d}^{\prime }{\Sigma }_{X}d+{\sigma }^2(1+(k/T))$

Another consequence of a single break in the parameters within the historical sample is that a forecasting model that uses a good predictor might end up generating poor forecasts because the parameter estimates are a hybrid of the pre- and postbreak models or, more generally, an average of time-varying coefficients. This can lead to situations where valuable predictor variables appear to be uninformative to the forecast because the hybrid parameter estimate is close to 0. Rossi (2013a)

gives a simple example similar to (19.1) in which $\begin{array}{r}\beta =-d\frac{\tau }{T-1}\end{array}$ so that the fullsample test that $x_t$ Granger causes $y_{t+h}$ yields an OLS estimate that converges to 0 even though $\beta \ne 0$ and the predictor is useful for forecasting at each point in the sample. This is, of course, a very special case in which the effect of parameter instability exactly eradicates any full-sample evidence of predictability. Nevertheless, the point holds more broadly that predictability can be present locally in time in a way that conventional full-sample tests may not have power to detect. The converse is also possible: full-sample tests might indicate that certain predictors are useful even though, at the time of the forecast, ${\beta }_T$ is close to 0. This situation would occur in our example if $\beta =0$ , but d $\ne 0$