3.1.2 Interpretation of Forecast Optimality

3.1.2 Interpretation of Forecast Optimality

The notion of an optimal forecast that minimizes the expected loss in (3.2) has several implications. First, taking expectations over Y in (3.2), it follows that optimal point forecasts aim to work well on average rather than for a particular value (single draw) of the outcome. In this sense the forecast attempts to estimate not the realization of the outcome, but rather a function of its predictive or conditional distribution. The optimal forecast itself is a function of z and can be considered a parameter of the conditional distribution, $p_Y(y|z)$ . Under MSE loss the relevant function is the conditional mean; under MAE loss the relevant function is the conditional median; for Linex loss it is a feature of the moment generating function for the conditional random variable. The optimal forecast in this sense is unique. However, as we show below, when we move to estimating this function (or parameter), there will be no unique optimal forecast since there are many estimators for these features of the conditional distribution for Y, none of which are the single optimal estimator.

That the optimal forecast minimizes the average conditional loss also highlights the distinction between statistically bad forecasts and economically bad forecasts. A statistically bad forecast comes from constructing a poor model given the available data, while an economically bad forecast provides forecasts that are too imprecise for useful decision making. Forecasts are often viewed as “poor” if they are far from the observed realization of the outcome variable. However, in practice even good forecast models occasionally produce forecasts that are far from the outcome. For example, this will happen when we use an estimate of the conditional mean as our forecast but observe an outcome, y, drawn from the tails of its conditional distribution ${\rho }_Y(y|z,\theta )$ . We do not necessarily view this as a bad forecast in the sense that such outcomes can occur even when the model is correctly specified.

Second, the optimal forecast in all these examples depends on the conditioning variables used to produce the forecast, Z. Conditioning on different predictor variables generally leads to different optimal forecasts given those predictor variables. This is true regardless of the forecasting problem. Any claims of optimality for a particular forecast model should be restricted to optimality for the given set of predictor variables used to construct the forecast.

Third, the optimization in (3.3) is with respect to a class of functions $\mathcal{F}.$ The set of possible functions is often too wide to suggest the form of a reasonable forecast. Consider, for example, MSE loss for which we know that the optimal forecast takes the form of the conditional mean. However, the class of possible models for the conditional mean is immensely large, and hence does not actually define the form of the conditional expectation, (3.5), even up to a finite set of unknown parameters. Results can sometimes be established when the search over functions is restricted, but the notion of forecast optimality is clearly weaker.

Example 3.1.5 (MSE loss and linear projections). Under MSE loss the optimal forecast takes the form $f^{\ast }=E\left[Y|z\right]$ . Computing the conditional mean requires knowledge about the true relationship between y and z. Forecasts commonly use simple regression models based on linear projection of y $onz:f(z)={\beta }^{\prime }z$ , where $\beta$ are the parameters from the linear projection. Such projections are a subset of all possible prediction models so MSE $(f(z))\ge \mathrm{MSE}(E\left[Y|z\right])$ and linear projections are generally suboptimal in population. By construction, forecast errors and projections are orthogonal within the class of linear projections:

$$E_Y[(Y-{\beta }^{\prime }z)z^{\prime }]=0.$$

Within the class of linear models, linear projection is therefore optimal.

We might limit the set of forecast models $\mathcal{F}$ to specific parametric models rather than extending it to a wide class of functions. As in the previous example, this set may or may not include the optimal model when all possible functions are considered. When this restricted class can be written in parametric form as a function of the unknown parameters $\beta$ and the data $z,$ the optimal forecast in this restricted class is given by $f(z,{\beta }^{\ast })$ where

$${\beta }^{\ast }=\arg \underset{\beta \in \mathcal{B}}{\min }{E}_Y[L(f(Z,\beta ),Y)].\text{\hspace{1em}(3.10)}$$

We refer to ${\beta }^{\ast }$ as the pseudo-true value for $\beta {.}^5$ However, ${\beta }^{\ast }$ need not be unique because the expected value of the loss function may obtain a minimum at a number of different values for $\beta .$ Note that the value of the pseudo-true parameter, ${\beta }^{\ast }$ , depends on $\theta ,$ , the parameters of the predictive distribution ${\rho }_Y(y|z,\theta )$ , so the expected loss is still a function of $\cdot \_\theta$ rather than simply $\beta .$ .

While reduced-form expressions for the optimal forecast are often unavailable, some results can be shown for restricted classes of loss functions and conditional distributions for $Y$ given $Z=z$ . For error-based loss functions Granger (1969b) establishes that the conditional mean $E_Y[y|z]$ is the optimal forecast if the loss function is symmetric about 0 and the conditional distribution of the outcome is symmetric about $E_Y[y|z]$ so that ${\bar{p}}_Y(y)\equiv p_Y(y-E_Y[y|z])$ is symmetric about 0. This result also requires that either (i) the derivative of the loss function, $L^{\prime }.$ is strictly monotonically increasing or (ii) the conditional density $p_Y(y|z)$ is continuous and unimodal. We show the result in the first case. The expected loss is given by

$${\int }_{-\infty }^{\infty }L(y-f{)}_{PY}(y|z)dy={\int }_{-\infty }^{\infty }L(y-\alpha ){\bar{p}}_Y(y|z)dy,\text{\hspace{1em}(3.11)}$$

where $f=E_Y[y|z]+\alpha$ and $\alpha =f-E_Y[y|z]$ is the forecast bias. Differentiating (3.11) with respect to $\alpha ,$ we have

$${\int }_{-\infty }^{\infty }{L}^{\prime }(y-\alpha ){\bar{p}}_Y(y|z)dy=0.\text{\hspace{1em}(3.12)}$$

Using the assumed symmetry of ${\bar{p}}_Y(y)$ and the antisymmetry of $L^{\prime }{(y),}^6$ we have

$${\int }_{-\infty }^{\infty }{L}^{\prime }(y){\bar{p}}_Y(y|z)dy=0.\text{\hspace{1em}(3.13)}$$

This means that $\alpha =0$ is a solution to (3.12). As pointed out by Granger, the solution to (3.12) is a unique minimum. To see this, suppose that there is another value of α that satisfies (3.13), i.e.,

$${\int }_{-\infty }^{\infty }\left[L^{\prime }(y-\alpha )-L^{\prime }(y)\right]{\bar{p}}_Y(y|z)dy=0.\text{\hspace{1em}(3.14)}$$

Because $L^{\prime }$ is increasing, we have $L^{\prime }(y-\alpha )-L^{\prime }(y)>0\mathrm{if}\alpha <0,$ , while conversely $L^{\prime }(y-\alpha )-L^{\prime }(y)<0\mathrm{if}\alpha >0$ . Because ${\bar{p}}_Y(y|z)\ge 0$ and integrates to 1, this means that (3.14) cannot hold unless $\alpha =0$ . The second-order condition for the optimum forecast guarantees that $f=E_Y[y|z]$ is a minimum.