12.3.3 Maximum Utility Estimation

12.3.3 Maximum Utility Estimation

Maximum utility estimation makes use of the M-estimator to minimize risk. Using homogeneity of the risk function, minimizing risk amounts to choosing a function, $g\in G$ ,

$$\underset{g\in G}{\max }{E}_{\tilde{Y},Z}\{b(Z)[\tilde{Y}+1-2c(Z)]\mathrm{sign}[g(Z)]\},\text{\hspace{1em}(12.20)}$$

where $G$ is the set of measurable functions from ${\mathbb{R}}^k$ to R, $b(z)=u_{1,1}(z)-u_{0,1}(z)+$ $u_{0,0}(z)-u_{1,0}(z)$ , and $\mathrm{sign}(a)=1$ for $a>0,\mathrm{sign}(a)=-1$ for $a\le 0$ . Having chosen the best function $g(Z)$ in $G$ , forecasts are constructed as $\mathrm{sign}(g(Z))$ . In practice, we choose a set of parameterized functional forms for $g(Z),g(Z,\beta )$ , and maximize over $\beta .$

The sample analog of the objective function in (12.20) is

$$Q_T(\beta )\equiv \frac{1}{T}\sum _{t=0}^{T-1}b(z_t)[{\tilde{y}}_{t+1}+1-2c(z_t)]\mathrm{sign}[g(z_t,\beta )].\text{\hspace{1em}(12.21)}$$

This method was suggested by Manski (1975, 1985) and Manski and Thompson (1989) for the case where $c(z)=0.5,b(z)$ is constant, and $g(z,\beta )=g(z^{\prime }\beta )$ . Under these conditions the estimator that maximizes (12.21) is the maximum score estimator. Elliott and Lieli (2013) extend this analysis to the general case.

Estimation of $\beta$ in (12.21) depends on the utility function since different values of $b(z_t)$ and $c(z_t)$ affect the estimates of the parameters, $\beta .$ This means that the value of each match varies as a function of the utility function through the weights $b(z_t)[{\tilde{y}}_{t+1}+1-2c(z_t)]$ . The elements $b(z_t)$ and $c(z_t)$ play different roles in the estimation: $c(z_t)$ determines the relative values of true positives versus true negatives, while $b(z_t)$ reweights the objective function, assigning relatively higher weights to those values of $z_t$ for which correct prediction yields the largest benefits.

Regardless of the specification of $g(Z,\beta )$ , the presence of the sign function in (12.21) makes the objective function nondifferentiable at points where the sign changes. This complicates both estimation and inference. For example, gradient methods will not be useful for estimation. Elliott and Lieli (2013) suggest simulated annealing methods for the general case; see also Corana et al. (1987) and Goffe, Ferrier, and Rogers (1994).

Primitive conditions that ensure convergence of $Q_T(\beta )$ to $E[b(z_t)[{\tilde{y}}_{t+1}$ $+1-2c(z_t)]\mathrm{sign}[g(z_t,\beta )]]$ are given in Elliott and Lieli (2013). The assumptions that guarantee consistency of the estimator are slightly more restrictive than those usually entertained, e.g., strict stationarity and ergodicity of $\{{\tilde{Y}}_t,Z_t\}$

12.3.4 Comparing Estimation Methods

Provided that the forecasting model is correctly specified, estimation without use of the loss function yields consistent parameter estimates with efficiency properties that are likely to be difficult to improve upon in practice. Correct specification here means not just including the correct predictors but also having the correct functional form for the conditional probability of $Y=1$ . Conversely, when the model is not correctly specified, use of the loss function will generally improve on the forecasting performance.

In the context of binary forecasts, there can often be large gains from using the correct loss function given the practice of employing the arbitrarily chosen MSE loss function for parameter estimation. Unlike the linear regression case, using MSE loss never generates maximum likelihood or pseudo maximum likelihood results. So, unless squared error loss really reflects the forecast user’s preferences, use of the correct loss function for estimation will likely yield improvements to the results.

These issues can be given a visual interpretation for the binary case. Returning to figure 12.1, we do not need very precise estimates of $p(z)$ everywhere to generate point forecasts, except at the point at which this function cuts $c(z)$ , which equals 0.5 for all values of z in the figure. Outside this region, all we require is that $p(z)-c(z)$

has the correct sign—not that it is particularly close to its true value. Maximum likelihood estimation attempts to fit $p(z)$ as closely as possible given the flexibility of the model over all values of $z,$ not just for z near the cutoff. Similar issues arise when squared loss is employed for estimation. Only use of the loss function in the estimation will focus the attention of the estimators on the cutoff.

Elliott and Lieli (2013) explore these issues analytically and through Monte Carlo analysis. In situations with misspecified models, they show that differences between loss-based and non-loss-based estimators can be quite large even when the misspecification is small enough to be difficult to detect with conventional tests. Lossbased and non-loss-based estimation methods have also been contrasted using actual data; Leung, Daouk, and Chen (2000) consider stock market returns, Srinivasan and Kim (1987) consider credit extension, and Lieli and Springborn (2013) study management of invasive plants.