3.2.3 Minimax and Average Risk

3.2.3 Minimax and Average Risk

As with estimation, we might also consider minimax forecasting strategies. Rather than choosing the forecast to minimize the expected loss, an alternative is to choose the forecast with the smallest risk across all possible values of the unknown parameters, $\theta \in \Theta$ . This is the minimax rule. Formally, one forecast, $f_1$ , is preferred to another forecast, $f_2$ , under the minimax notion of optimality if

$$\underset{\theta \in \Theta }{\sup }R(\theta ,f_1)<\underset{\theta \in \Theta }{\sup }R(\theta ,f_2).$$

More generally, $f^m$ is the minimax decision rule if it minimizes supθ $R(\theta ,f)$ among all decision rules: $\begin{array}{r}{\sup }_{\theta \in \Theta }R(\theta ,f^m)={\inf }_{f\in \mathcal{F}}\left({\sup }_{\theta \in \Theta }R(\theta ,f)\right)\end{array}$ . Minimax rules are often difficult to identify outside very restricted sets of problems.

Example 3.2.8 (MSE loss given the observed history of the outcome, continued). Consider the set of estimators $\{\gamma {\bar{y}}_T\}for0<\gamma \le 1.$ , and notice from (3.20) that for all $\gamma <1$ the risk is unbounded as $\mu$ becomes large. For $\gamma =1$ , the risk does not depend on $\mu ,$ and hence the worst case scenario over $\mu$ for any ${\sigma }^2$ is infinite risk for $\gamma <1$ or ${\sigma }^2(1+T^{-1})for\gamma =1$ . Hence choosing the forecasting method ${\bar{y}}_T$ is minimax over $\mu$ in this set.

Minimax results extend beyond this example. For instance, OLS parameter estimates often yield minimax forecasts with bounded risk among linear predictors and predictive densities with finite second moments.

Since there is typically no single optimal procedure that minimizes risk over all regions of $\theta ,$ , choosing between forecast methods comes down to a view as to which regions of $\theta$ are most likely. In the examples above where the choice is over $f_{\gamma }(z)$ we might use the sample mean if we think it likely that $\mu$ is large in absolute value. Conversely, if we think $\mu$ is unlikely to be large relative to the bounds derived in Example 3.2.4, then we might choose a shrinkage estimator. The optimal choice of the shrinkage parameter $\gamma$ depends on how close we believe $\mu$ is to $0.$ More generally, the choice between forecasting approaches critically depends on the family of models thought to be possible for any forecasting problem. “Kitchen sink” regressions that include all available variables in the forecast model and use least squares estimation offer a robust strategy over a wide set of parameters, $\theta .$ . In practice, however, this strategy tends to provide extremely poor forecasts with very high risk because much of the parameter space for which this approach would work well might reasonably be deemed irrelevant. All of the methods discussed in the second part of the book are directed at different forecast situations and can be thought of as providing models that work well for different regions of $\stackrel{\rightharpoonup }{\theta }$

It is common to run Monte Carlo experiments to compute the average loss for different values of the parameters and compare competing estimation methods. One can then choose the “least bad” method or alternatively choose the method that seems to be doing best for the majority of cases. We can also consider a more formal approach to decide between methods. Denoting the weighting function by $\pi (\theta )$ one can choose the forecast method that minimizes  $R(\theta ,f)\pi (\theta )d\theta$ , i.e., the risk averaged over all relevant parameters. The weighted average risk is a single number, enabling direct comparisons across different forecast methods.