15.5 OPTIMALITY TESTS THAT DO NOT RELY ON MEASURING THE OUTCOME

15.5 OPTIMALITY TESTS THAT DO NOT RELY ON MEASURING THE OUTCOME

Conventional forecast efficiency tests based on forecast errors such as (15.22) require that we observe the realizations of the outcome variable so we can compute $e=y-f$ . Such data may not always be available. For example, many macroeconomic variables are subject to data revisions and so first release, second release, or the latest data vintage could be used to measure the outcome. This raises some fundamental questions related to whether forecasters are trying to predict the first data release or some underlying “true” variable that may never be known with certainty.

These issues can be ignored when evaluating forecasts under MSE loss provide that multiple forecasts of the same “event,” Yt, produced at different points in time, are available. As shown by Nordhaus (1987), the forecast revision should itself be unpredictable and follow a martingale process. To see this under MSE loss, note that the rational forecast satisfies $f_{t+h|t}=E\left[Y_{t+h}|Z_t\right]$ and $f_{t+h|t+1}=E\left[Y_{t+h}|Z_{t+1}\right]$ . By the law of iterated expectations, $E\left[f_{t+h|t+1}|Z_t\right]=E\left[E\left[Y_{t+h}|Z_{t+1}\right]|Z_t\right]=E\left[Y_{t+h}|Z_t\right]=$ $f_{t+h|t}$ . Hence the optimal forecast of a fixed event $Y_{t+h}$ is itself predicted not to change over time:

$$E\left[f_{t+h|t+1}-f_{t+h|t}|Z_t\right]\equiv E\left[\Delta f_{t+h}|Z_t\right]=0.\text{\hspace{1em}(15.39)}$$

Stated differently, the best current forecast of next period’s prediction is the current forecast. If we expect next period’s forecast to change from the current value in a manner that is predictable, this information should lead us to revise the current forecast which therefore could not have been rational in the first instance.

The martingale difference condition in (15.39) can be tested by means of a simple regression that does not involve the outcome:

$$\Delta f_{t+h|t+1}={\alpha }_h+{\delta }_h{v}_t+u_{t+h},\text{\hspace{1em}(15.40)}$$

where $\Delta f_{t+h|t+1}=f_{t+h|t+1}-f_{t+h|t},v_t=g(z_t)\in Z_t$ and, under the null of forecast rationality, $\alpha =\delta =0$

Patton and Timmermann (2012) suggest an alternative regression test of this property which replaces the actual outcome with the short-run forecast in the Mincer–Zarnowitz regression:

$$f_{t|t-h_S}=\alpha +\beta f_{t|t-h_L}+{\epsilon }_t,\text{\hspace{1em}(15.41)}$$

where $h_S<h_L$ and $\alpha =0,\beta =1$ under the null of forecast rationality. If there are more than two forecast horizons, this regression will take the form of a stacked set of regressions or, alternatively, previous forecasts or forecast revisions can all be included on the right-hand side of equation (15.41).