1.1.1 Part I

1.1.1 Part I

The first part of the book motivates that point forecasting should be thought of simply as an application of decision theory. Since much is known about decision theory, much is also known about forecasting. This perspective makes point forecasting a special case of estimation, a field where excellent texts already exist. What makes economic forecasting interesting as a separate topic is the particular details of how decision theory is applied to the problem at hand. To apply this approach, we require a clear statement about the costs of forecast errors $e=y-f,$ where y is the outcome being predicted and f is the forecast. The trade-off between different forecasting mistakes is embodied in a loss function, $L(f,y)$ which is discussed in chapter 2, with additional material on the binary case available in chapter 12. We regard loss functions as realistic expositions of the forecaster’s objectives, and consider the specification of the loss function as an integral part of the forecaster’s decision problem.1 Different forecasters approaching the same outcome may well have different loss functions which could result in different choices of forecasting models for the same outcome.

The specification of loss functions is often disregarded in economic forecasting, and instead “standard” loss functions such as mean squared error loss tend to be employed. This can prove costly in real forecasting situations as it overlooks directions in which forecast errors are particularly costly. Nonetheless, much of the academic literature is based on these standard loss functions and so we focus much of our survey of methods throughout the second part of the book on these standard loss functions.

Chapter 3 provides a general description of the forecaster’s problem as a decision problem. It may strike some readers, more used to the “art” of forecasting, as unusual to cast point forecasting as a decision-theoretic problem. However, even readers who do not explicitly follow this approach are indeed operating within the decisiontheoretic framework. For example, most forecasting methods are motivated in one of two ways: either the methods are demonstrated to provide better performance given a loss function (or set of loss functions) through Monte Carlo simulations for reasonable data-generating processes, or alternatively, the forecasting methods are shown to work well for some loss function for a particular set of empirical data $z=(y,x)$ , where x represents the set of predictor variables used to forecast the outcome y. Both ways of measuring performance place the forecasting problem within the decision-theoretic approach.

To illustrate this point, consider Monte Carlo simulations of a data-generating process (joint density for the data) regarded as a reasonable representation of some data of interest. The simulation method suggests constructing N independent pseudo samples from this density, constructing N forecasts and evaluating $N^{-1}\overline{{\sum }_{n=1}^N}L(f^{(n)},y^{(n)})$ , where $y^{(n)}$ is the outcome we wish to forecast, $f^{(n)}$ is the forecast generated by a prediction model, and $L(f,y)$ is the loss function which measures the costs of forecast inaccuracies. Superscripts refer to the individual simulations, $n=1,\dots ,N$ . The simulated average loss is usually thought of as a measure of the performance of the forecasting method or model for this datagenerating process. This is reasonable since as N gets large, the sample average is, by standard laws of large numbers, a consistent estimate of the risk at the point of the parameter space for the data-generating process chosen for the Monte Carlo, i.e., as long as E $[L(f,y)]$ exists,

$$N^{-1}\sum _{n=1}^{N}L(f^{(n)},y^{(n)}){\to }^{p}E\left[L(f,y)\right],\text{\hspace{1em}(1.1)}$$

where the Monte Carlo estimates a point on the risk function and $\to \mathbf{P}$ means convergence in probability. Finding a forecast that minimizes the risk is precisely the setup of a decision-theoretic problem.

The third part of the book discusses methods for evaluation of sequential outof-sample predictions. In each case, one obtains from the data set T observations of the “realized” loss from the data. One then evaluates the time-series average $\begin{array}{r}{T}^{-1}{\sum }_{t=1}^{T}L(f_t,y_t)\end{array}$ , where the t subscript refers to time, as a measure of the expected loss. In this case the assumptions that underlie results such as (1.1) are much more stringent because the sequence of expected losses generated from data are not independently and identically distributed (i.i.d.) as in the Monte Carlo simulations. However, under suitable assumptions again this method estimates risk. When using real (as opposed to simulated) data, we do not know the true parameter values of the data-generating process. Analyzing a variety of economic variables, we get a sense of how well different forecasting methods work for different types of data.

The general setup in chapter 3 is common to forecasters basing their estimation strategies either on frequentist or on Bayesian approaches. Chapters 4 and 5 build on this setup separately for these two approaches. Chapter 4 examines the typical frequentist approaches, explaining general pitfalls that can occur as well as highlighting special cases arising later in the book. Chapter 5 does the same for the Bayesian approach.

Viewing forecasting as a decision-theoretic problem sometimes means that the best forecasting model, despite working well in practice, may actually be a model that is very difficult to interpret economically. This becomes a problem when the forecasting exercise is a step in a decision process, and the forecaster must “explain” the forecast to decision makers or forecast users. In these cases an inferior point forecast that tends to be further away from the outcome may be preferred because it is easier to explain and may be seen to be more credible. Of course in situations where we suspect a lot of overfitting or instability in the relationships between the variables, we might prefer forecasting models that conform to economic theory since they are expected to be more robust. Practically, economically motivated restrictions on forecasting models can just be seen as following the decision-theoretic approach for a restricted set of models.

The final chapter of the first part of the book, chapter 6, examines issues related to model selection. By now the econometrics literature has a very good understanding of the merits and limitations of model selection, which we discuss for general models. From the perspective of forecasting, however, we regard model selection as simply part of the model estimation process. Of interest to the forecaster is the risk of the final forecasting model computed in a way that accounts for the full estimation process. Given the complexity of the distributions of estimators obtained from models whose selection is driven by the data, this issue is difficult to address analytically although it is still of direct relevance to the forecaster.