2.1.3 Existence of Expected Loss

2.1.3 Existence of Expected Loss

Restrictions must be imposed on the form of the loss function to make sense of the idea of minimizing the expected loss. Most basically, it is required that the expected loss exists. Suppose the forecast depends on data Z through a vector of parameters, ${\beta }_{:}$ which depends on the parameters of the data generating process, θ, so $f=f(z,\beta )$ From the definition of expected loss, we have

$$E_Y[L(f(z,\beta ),Y)]=\int L(f(z,\beta ),y{)}_{PY}(y|z,\theta )dy,\text{\hspace{1em}(2.6)}$$

where ${\rho }_Y(y|z,\theta )$ is the predictive density of y given z, θ . When the space of outcomes $\mathcal{V}$ is finite, this expression is guaranteed to be finite. However, for outcomes that are continuously distributed, restrictions must sometimes be imposed on the loss function to ensure finite expected loss. The existence of expected loss depends, both, on the loss function and on the distribution of the predicted variable, given the data, ${\rho }_Y(y|z,\theta )$ , where $\theta$ denotes the parameters of this conditional distribution. Existence of expected loss thus hinges on how large losses can get in relation to the tail behavior of the predicted variable, as captured by ${\rho }_Y(y|z,\theta )$

A direct way to ensure that the expected loss exists is to bound the loss function from above.6 From a practical perspective this would seem to be a sensible practice in constructing loss functions. Even so, many of the most popular loss functions are not bounded from above. In part this practice stems from not considering the loss related to the forecasting problem at hand, but instead borrowing “off-the-shelf” loss functions from estimation methods that lead to simple closed-form expressions for the optimal forecast.

It is useful to demonstrate the conditions needed to ensure that the expected loss exists. Following Elliott and Timmermann (2004), suppose that L depends only on the forecast error, $e=y-f$ , and lends itself to a Taylor-series expansion around the mean error, ${\mu }_e=E_Y[Y-f]$ :

$$L(e)=L({\mu }_e)+L_{{\mu }_e}^{\prime }(e-{\mu }_e)+\frac{1}{2}{L}_{{\mu }_e}^{\prime \prime }(e-{\mu }_e{)}^2+\sum _{k=3}^{\infty }\left(\frac{1}{k!}\right)L_{{\mu }_e}^k(e-{\mu }_e{)}^k,\text{\hspace{1em}(2.7)}$$

where $L_{{\mu }_e}^k$ denotes the kth derivative of L evaluated at ${\mu }_e.$ . Suppose there are only a finite number of points where L is not analytic and that these can be ignored because they occur with probability 0. Taking expectations in (2.7), we then get

$$\begin{array}{l}E\left[L(e)\right]=L({\mu }_e)+\frac{1}{2}{L}_{{\mu }_e}^{\prime \prime }{E}_Y[(e-{\mu }_e{)}^2]+\sum _{k=3}^{\infty }\left(\frac{1}{k!}\right)L_{{\mu }_e}^k{E}_Y[(e-{\mu }_e{)}^k]\\ \\ =L({\mu }_e)+\frac{1}{2}{L}_{{\mu }_e}^{\prime \prime }{E}_Y[(e-{\mu }_e{)}^2]+\sum _{k=3}^{\infty }\left(\frac{1}{k!}\right)L_{{\mu }_e}^k\sum _{i=0}^k(\genfrac{}{}{0ex}{}{k}{i})E_Y[e^{k-i}{\mu }_e^i]\\ =L({\mu }_e)+\frac{1}{2}{L}_{{\mu }_e}^{\prime \prime }{E}_Y[(e-{\mu }_e{)}^2]+\sum _{k=3}^{\infty }{L}_{{\mu }_e}^k\sum _{i=0}^k\frac{1}{i!(k-i)!}{E}_Y[e^{k-i}{\mu }_e^i].\end{array}\text{\hspace{1em}(2.8)}$$

This expression is finite provided that all moments of the error distribution exist for which the corresponding derivative of the loss function with respect to the forecast error is nonzero. This is a strong requirement and rules out some interesting combinations of loss functions and forecast error distributions. For example, exponential loss (or the Linex loss function defined below) and a student-t distribution with a finite number of degrees of freedom would lead to infinite expected loss since all higher-order moments do not exist for this distribution. What is required to make the higher-order terms in (2.8) vanish is that the tail decay of the predicted variable is sufficiently fast relative to the weight on these terms implied by the loss function.