Loss Functions

A formal method for trading off potential forecast errors of different signs and magnitudes is therefore required. The loss function, L (·), describes in relative terms how costly it is to use an imperfect forecast, f, given the outcome, Y, and possibly other observed data, Z. This chapter examines the construction and properties of loss functions and introduces loss functions that are commonly used in forecasting.

A central point in the construction of loss functions is that the loss function should reflect the actual trade-offs between different forecast errors. In this sense the loss function is a primitive to the forecasting problem. From a decision-theoretic perspective the forecast is the action that must be constructed given the loss function and the predictive distribution, which we discuss in the next chapter. For example, the Congressional Budget Office must provide forecasts of future budget deficits. Their loss function in providing the forecasts should be based on the relative costs of over- and underpredicting public deficits. Weather forecasters face very different costs from underpredicting the strength of a storm compared to overpredicting it.

The choice of a loss function is important for every facet of the forecasting exercise. This choice affects which forecasting models are preferred as well as how their parameters are estimated and how the resulting forecasts are evaluated and compared against forecasts from competing models. Despite its pivotal role, it is common practice to simply choose off-the-shelf loss functions. In doing this it is important to choose a loss function that at least approximately reflects the types of trade-offs relevant for the forecast problem under study. For example, when forecasting hotel room bookings, it is hard to imagine that over- and underpredicting the number of hotel rooms booked on a particular day lead to identical losses because hotel rooms are a perishable good. Hence, using a symmetric loss function for this problem would make little sense. Asymmetric loss that reflects the larger loss from over- rather than underpredicting bookings would be more reasonable.

There are examples of carefully grounded loss functions in the economics literature. For example, sometimes a forecast can be viewed as a signal in a strategic game that is influenced by the forecast provider’s incentives. Studies such as Ehrbeck and Waldmann (1996), Hong and Kubik (2003), Laster, Bennett, and Geoum (1999), Ottaviani and Sørensen (2006), Scharfstein and Stein (1990) and Trueman (1994) suggest loss functions grounded on game-theoretical models. Forecasters are assumed to differ in their ability to predict future outcomes. The chief objective of the forecasters is to influence forecast users’ assessment of their ability. Such objectives are common for business analysts or analysts employed by financial services firms such as investment banks or brokerages whose fees are directly linked to clients’ assessment of their forecasting ability.

The chapter proceeds as follows. Section 2.1 examines general issues that arise in construction of loss functions. We discuss the mathematical setup of a loss function before relating it to the forecaster’s decisions and examining some general properties that loss functions have. Section 2.2 reviews specific loss functions commonly used in economic forecasting problems, assuming there is only a single outcome to predict, before extending the analysis in section 2.3 to cover cases with multiple outcome variables. Section 2.4 considers loss functions (scoring rules) for distributional forecasts, while section 2.5 provides some concrete examples of loss functions and economic decision problems from macroeconomic and financial analysis. Section 2.6 concludes the chapter.