3.4 RELATING THE BAYESIAN AND CLASSICAL METHODS

3.4 RELATING THE BAYESIAN AND CLASSICAL METHODS

For any weighting scheme, π(θ), over the possible parameter values, the Bayesian approach finds the best average method. The classical approach, due to its typically ad hoc use of plug-in estimators, may or may not achieve the same weighted average risk. Even in settings with identical loss functions and predictive densities, one might therefore find a Bayesian method with equal or smaller risk than the classical procedure for all possible values of θ. Conversely, if the two methods are equivalent, then the classical approach cannot be beaten.

Example 3.4.1 (Comparing classical and Bayes risk under Linex loss with Gaussian data). To compare the classical and Bayesian results, assume the variance is known. The classical plug-in estimator in (3.17) is

$$\hat{f}(z)={\bar{y}}_T+\frac{a{\sigma }^2}{2}.$$

For any normal prior this yields higher risk than the optimal rule. To see this, note that for the optimal rule in (3.28),

$$R(\theta ,f)=\frac{a{\sigma }^2}{2}\left(1+\frac{1}{T}\right)-1,$$

while for the classical plug-in rule,

$$R(\theta ,\hat{f})=\exp \left[\frac{a^2{\sigma }^2}{2T}\right]+\frac{a{\sigma }^2}{2}-1.$$

The difference between these two risk measures is

$$\begin{array}{c}R(\theta ,\hat{f})-R(\theta ,f)=\exp \left[\frac{a^2{\sigma }^2}{2T}\right]+\frac{a{\sigma }^2}{2}-\frac{a{\sigma }^2}{2}\left(1+\frac{1}{T}\right)\\ =\exp \left[\frac{a^2{\sigma }^2}{2T}\right]-\frac{a^2{\sigma }^2}{2T}>0.\end{array}$$

This follows because the loss function is positive for arguments away from 0.

In the classical setting the plug-in method for constructing f (z) through the estimate $\hat{\theta }(z)$ may be ad hoc rather than directed towards minimizing average loss. Often forecasters choose estimators that yield nice properties for the parameters themselves. For example, estimators that are individually consistent, asymptotically normal, and asymptotically efficient for θ are often employed. Such features of the estimators ignore potential trade-offs between parameters that could be used to generate forecasts with better risk properties. Plug-in methods may therefore not yield good forecasting rules for constructing the forecast, $f(z)$ , even for reasonable weighting functions, $\pi (\theta )$ . Ultimately, the goal of forecasting is not to estimate θ , but instead to minimize a function of these parameters.

In practice, differences between the risks of optimal and ad hoc methods need not be large. In the very simple example above, the difference is small for large values of the sample size, T . Even if a method is not strictly admissible, it might well be “close enough” that the classical method works reasonably well.

Admissibility—using the best forecast method given some weighted average over the parameters π (θ )—is a less exciting property if the particular weights, $\pi (\theta )$ , are not empirically plausible for the particular problem at hand.

Finally, all such comparisons of forecast methods require that the model ${\rho }_Y(y|z,\theta )$ be correctly specified up to the unknown parameters. This goes against both the spirit and practice of modern forecasting. Typically, the true model is unknown. Still, the Bayesian approach offers a method for forecast construction that is guaranteed to be admissible for the specified model, should it be true. Even if the true model is not necessarily the one used to construct the forecasts, provided that the forecasting model is close to the true model we are assured of using a method that works well for this particular model.

It follows from this discussion that it is difficult to find optimal solutions even for very simple forecasting problems. Hence for different combinations of distributions of the data and values of the model parameters, there is scope for alternative methods to dominate. This lack of a single dominant approach explains much of the burgeoning interest in empirical comparisons of different forecasting approaches.