2.1 CONSTRUCTION AND SPECIFICATION OF THE LOSS FUNCTION

2.1 CONSTRUCTION AND SPECIFICATION OF THE LOSS FUNCTION

Let Y denote the random variable describing the outcome of interest and let Y denote the set of all possible outcomes. For outcomes that are either continuous or can take on a very large number of possible values, typically $\mathcal{V}$ is the real line, R. In some forecasting problems the set of possible outcomes, $\mathcal{V},$ can be much smaller, such as for a binary random variable where $\mathcal{D}=\{0,1\}$ . For multivariate outcomes typically $\mathcal{V}={\mathbb{R}}^k$ for some integer $k,$ where k is the number of forecasts to be evaluated.

Point forecasts are denoted by $f$ and are defined on the set $\mathcal{F}$ . Typically we assume $\mathcal{F}=\mathcal{V}$ since in most cases it does not make sense to have forecasts that cannot take on the same values as $Y$ or, conversely, have forecasts that can take on values that the outcome $Y$ cannot. There are exceptions to this rule, however. For example, a forecast of the number of children per family could be a fraction such as 1.9, indicating close to 2 children, even though $Y$ cannot take this value. We assume that the predictors $Z$ (as well as the outcome Y and hence the forecast $f)$ are real valued. Formally, the loss function, $L(f,Y,Z)$ , is then defined as a mapping $L:\mathcal{V}\times \mathcal{V}\times \mathcal{Z}\mapsto \mathcal{L}$ , where $\mathcal{L}$ is in ${\mathbb{R}}^1$ , and $\mathcal{Z}$ contains the set of possible values the conditioning variables, $z,$ can take. Often $\mathcal{L}={\mathbb{R}}_{+}^1$ , the set of nonnegative real numbers. Alternatively, we could constrain the forecasts to lie in the convex hull of the set of all possible outcomes, i.e., $\mathcal{F}=\mathrm{conv}(\mathcal{V})$ . We discuss this further below.

A common assumption for loss functions is that loss is minimized when the forecast is equal to the outcome—min f $L(f,y,z)=L(y,y,z)$ . The idea is that if we are to find a forecast that minimizes loss, then nothing dominates a perfect forecast. In cases where the loss function does not depend on $Z,$ so $L(f,Y,Z)=L(f,Y)$ , it is natural to normalize the loss function so that it takes a minimum value at 0. This can be done without loss of generality by subtracting the loss associated with the perfect forecast $$\begin{gathered} f=y,i.e., \\ \overline{L(f,Y)}\stackrel{..}{=}\tilde{L}(f,Y)-\tilde{L}(\overline{Y,Y}) \end{gathered}$$ for any loss function, $\tilde{L}$ For $f=y$ to be a unique minimum we must have $L(f,y)>0$ for all $f\ne y{.}^1$ More generally, when the loss function $L(f,Y,Z)$ varies with Z, it may not be possible to rescale the loss function in this manner. For example, a policy maker’s loss function over inflation forecasts might depend on the unemployment rate so that losses from incorrect inflation forecasts depend on whether the unemployment rate is high or low. For simplicity, in what follows we will mostly drop the explicit dependence of the loss function on Z and focus on the simpler loss functions $L(f,Y)$ .

2.1.1 Constructing a Loss Function

Construction of loss functions, much like construction of prior distributions in Bayesian analysis, requires a careful study of the forecasting problem at hand and should reflect the actual trade-offs between forecast errors of different signs and magnitudes. Laying out the trade-off can be straightforward if the decision environment is fully specified and naturally results in a measurable outcome that depends on the forecast. For example, for a profit-maximizing investor with a specific trading strategy that requires forecasts of future asset prices, the natural choice of loss is the function relating payoffs to the forecast and realized returns. Other problems may not lead so easily to a specific loss function. For example, when the IMF forecasts individual countries’ budget deficits, both short-term considerations related to debt financing costs and long-term reputational concerns could matter.2 In such cases one can again follow a Bayesian prior selection strategy of defining a function that approximates a reasonable shape of losses associated with decisions based on incorrect forecasts.

Loss functions, as used by forecasters to evaluate their performance, and utility functions, as used by economists to assess the economic value of different outcomes, are naturally related. Both are grounded in the same decision-theoretic setup which regards the forecast as the decision and the outcome as the true state and maps pairs of outcomes (states) and forecasts $(Y,f)$ to the real line. In both cases we are interested in minimizing the expected loss or disutility that arises from the decision.

The relationship between utility and loss is examined in Granger and Machina (2006), who show that the loss function can be viewed as the negative of a utility function, although a more general relation of the following form holds:

$$U(f,Y)=k(Y)-L(f,Y),\text{\hspace{1em}(2.1)}$$

where k(Y) plays no role in the derivation of the optimal forecast.

Example 2.1.1 (Squared loss and utility). Granger and Machina (2006) show that a utility function $U(f,Y)$ generates squared error loss, $L(f,Y)=a(Y-f{)}^2,fora>0,$ if and only if it takes the form

$$U(f,Y)=k(Y)-a(Y-f{)}^2.\text{\hspace{1em}(2.2)}$$

It follows that utility functions associated with squared error loss are restricted to a very narrow set.

Academic studies often do not derive loss functions from first principles by referring to utility functions or fully specified decision-theoretic problems, though there are some exceptions. Loss functions that take the form of profit functions have been used to evaluate forecasts by Leitch and Tanner (1991) and Elliott and Ito (1999). West et al. (1993) compare utility-based and statistical measures of predictive accuracy for exchange rate models. Examples of loss functions derived from utility are provided in the final section of this chapter.