3.6 CONCLUSION

This chapter discusses how to find an optimal forecasting model by minimizing expected loss in situations with a known parametric forecasting model for the outcome given a set of predictor variables. In situations where the forecasting model is known except for a finite set of parameters, the expected loss can be computed by taking the expected value of the loss with respect to the conditional distribution for the outcome. Under MSE loss the optimal forecast is simply the conditional mean of the outcome.

When the model is unknown, past data must be employed to construct estimates of the model. Defining risk to be the expected loss integrated over both the outcome and the data employed to construct the forecast, the chapter discusses how classical forecasting approaches attempt to minimize this risk. Even in the simplest of forecasting situations, the presence of estimated parameters means that there is no unique forecasting model that minimizes risk over the entire parameter space. Different views on the relevant parameter space motivate different forecasting models. Although the Bayesian approach conditions on the data, different priors can be appropriate for different views on the parameter space, hence again leading to a range of possible forecasting models. Thus for both the classical and Bayesian approaches, there is not a single optimal approach to constructing a forecasting model.

The next two chapters elaborate on the classical and Bayesian approaches to constructing forecast models.

Classical Estimation of Forecasting Models

t is rare for a forecaster to know the exact form of the best forecasting model or I the values of its parameters. For example, under MSE loss we want to compute the conditional mean of Y, but typically do not know the true functional form of this. In practice, going from a forecasting model to an operational forecast—i.e., a single number or a density forecast—requires consulting with data that will help in both tasks. This chapter deals with general issues related to the classical approach to forecasting when parameters must be estimated while the next chapter discusses Bayesian estimation methods. We focus on the general approaches and how they relate to notions of forecast optimality as discussed in the previous chapter. Model selection is covered in chapter 6.

Using the setup from chapter 3 we are looking for a forecasting model, $f^{\ast }(z)$ , that minimizes the expected loss

$$f^{\ast }(z)=\underset{f\in \mathcal{F}}{\mathrm{argmin}}\int L(f(z),y)p_Y(y|z,\theta )dy.\text{\hspace{1em}(4.1)}$$

Solving (4.1) involves a search over a space of functions, F. Chapter 3 gave examples of how the optimal forecast can be constructed when the conditional density, ${\rho }_Y(y|z,\theta )$ , and the parameters, θ, are known.

When the parameters of the forecasting model $\beta$ must be estimated, the risk of the forecasting model will depend on the estimation procedure even if the model, ${\rho }_Y(y|z,\theta )$ , is known. Even in the most trivial examples there will not be a uniquely optimal forecast procedure over all ranges of θ. Hence different approaches to constructing forecast methods coexist, even when the form of the joint density of Y and Z is known with only the parameters, θ , to be estimated.

More realistically, ${\rho }_Y(y|z,\theta )$ is not even known up to a set of parameters $\theta \in \Theta$ in which case the model itself is an approximation, thus further restricting notions of optimality. Often what is referred to as an “optimal” forecast in the forecasting literature is much weaker than the notion of forecast optimality given in (4.1) and instead refers to a set of possible forecasts that solve a much more restricted problem.

Classical estimation methods are the dominant approach in forecasting analysis. The simplest classical approach to constructing a forecast model is to use the sample analog of the forecast objective function, i.e., the loss function, and base the model on estimates that maximize this objective function. This makes estimation of forecast models a special case of M-estimation in statistics. An alternative and very popular approach, described in the previous chapter, is to first formulate the forecast model (which, from the previous chapter is a feature of the conditional distribution of the predicted variable) and then plug in estimates for the unknown parameters possibly obtained by minimizing some other objective function such as mean squared errors.

Sections 4.1 and 4.2 examine different estimation procedures when the forecasting model is approximated by the parametric specification $f(z_T,\beta )$ for some vector $\beta \in B$ . Many approaches have been considered, including M-estimation based on the loss function, maximum likelihood estimation, and various ad hoc methods. These methods compute an average of the loss function over all the observed data points, rather than the loss function at the time where the forecast is being generated, which is at the end of the sample, at time $T.$ . The chapter also examines popular estimation methods that have had some success in using penalized loss functions. Here the intent is not to interpret the limiting value of $\hat{\mathbf{\beta }}$ , but instead to reduce parameter estimation error and its effect on risk. Section 4.3 briefly examines issues that arise when the forecast model is not specified parametrically. This section serves primarily as a bridge to subsequent chapters that cover the material in greater detail. Finally, section 4.4 concludes.