12.2 DENSITY FORECASTS FOR BINARY OUTCOMES

12.2 DENSITY FORECASTS FOR BINARY OUTCOMES

As noted in the previous section, the distribution of a binary outcome is described entirely by its mean, so density estimation and estimation of the mean amount to the same estimation problem. Hence, the most common form of forecast reported for 0–1 outcomes is an estimate of the conditional probability that $Y=1$ i.e., $p(z)=E\left[Y|Z=z\right]$

Forecasters with MSE loss could also be interested in reporting the conditional probability that $Y=1$ constructed by choosing $f(Z)$ as

$$p^{\ast }(Z)=\arg \underset{p(Z)}{\min }E[(Y-p(Z){)}^2|Z],$$

whose solution is the conditional mean, $p^{\ast }(Z)=E\left[Y|Z\right]$ . This is simply a special case of the result in chapter 3 pertaining to the Bernoulli distribution. For the 0–1 case, the density forecast and the conditional mean forecast are identical.

12.2.1 Parametric Density Forecasting Models

Parametric models of the conditional mean of Y are popular since they can be constructed to ensure that the probability lies in the 0–1 interval provided that they are nonlinear functions of the data. The most popular methods are the probit and logit models. Linear models truncated to fall between 0 and 1 can also be employed.

Estimation of the parameters of a model for the conditional distribution requires some form of loss function. Typically we choose a strictly proper scoring rule as reviewed in chapter 2. Recall that scoring rules take the form $S(\mathbf{y},\mathbf{p})$ , where $E\left[S(y,p)\right]$ is finite and $\mathbf{p}$ is maximized when it is set equal to the true conditional probability that generates y. The sample analog is

$$Q_T(\beta )=\sum _{t=0}^{T-1}S(p(z_t,\beta ),y_{t+1}).$$

Sample estimates of the parameters, $\hat{\mathbf{\beta }},$ are then given by

$${\hat{\mathbf{\beta }}}_T=\arg \underset{\mathbf{\beta }\in \mathbf{B}}{\max }\sum _{t=0}^{T-1}\mathbf{S}(\mathbf{p}({\mathbf{z}}_t,\mathbf{\beta }),{\mathbf{y}}_{t+1}).$$

Logit and probit models are typically estimated by maximum likelihood (log scoring rule) methods using an index model. Given a parametric function $p(z_t^{\prime }\beta )$ for the probability that $Y=1$ and a one-step-ahead forecasting model, this approach chooses $\beta$ to maximize

$$Q_T(\beta )=\frac{1}{T}\sum _{t=0}^{T-1}(y_{t+1}\ln p(z_t^{\prime }\beta )+(1-y_{t+1})\ln (1-\bar{p}(z_t^{\prime }\beta ))).\text{\hspace{1em}(12.9)}$$

The probit specification for the linear index model sets

$$\begin{array}{r}p(z_t^{\prime }\beta )=\Phi (z_t^{\prime }\beta ),\end{array}\text{\hspace{1em}(12.10)}$$

where $\Phi (\cdot )$ is the c.d.f. of a normal distribution; the logit model sets

$$p(z_t^{\prime }\beta )=\frac{\exp (z_t^{\prime }\beta )}{1+\exp (z_t^{\prime }\beta )}.\text{\hspace{1em}(12.11)}$$

Choices between these specifications are usually motivated by some background model $y_{t+1}^{\ast }=z_t^{\prime }\beta +{\epsilon }_{t+1}$ , where $y_{t+1}^{\ast }$ is unobserved, while the indicator $y_{t+1}=$ $1(y_{t+1}^{\ast }>0)$ is observed. The probit or logit model (or other specifications) then follow from distributional assumptions on ${\epsilon }_{t+1};$ see Maddala (1986) for an overview.

Under assumptions on the data-generating process, general properties can be established for estimates of the conditional probability (12.7) using different models and scoring rules. If the data are strictly stationary and sufficiently well behaved, $\hat{\beta }\to {\beta }^{\ast }=\arg {\max }_{\beta \in B}E\left[S(y_{t+1},p(z_t,\beta ))\right]$ . Such results rely on different proofs of laws of large numbers, an example of which is given in Elliott, Ghanem, and Krüger (2014). If the data are covariance stationary rather than strictly stationary, it is still possible that the estimator converges to the value that minimizes average expected loss, $\begin{array}{r}\hat{\beta }{\beta }^{\ast }=\arg {\max }_{\beta \in B}{\sum }_{t=0}^{T-1}E[S(y_{t+1},p(z_t,\beta ))]\end{array}$ . Use of a strictly proper scoring rule is necessary to ensure identification. de Jong and Woutersen (2011) present consistency results for the probit model with lagged values for y (which are binary) added.

For all strictly proper scoring rules, $p(z_t,{\beta }^{\ast })$ is the true conditional probability provided that the model is correctly specified. Regardless of the scoring rule, it follows that the forecasts should be similar in sufficiently large samples. This gives little reason to choose between scoring rules as long as they are strictly proper. However, the log scoring rule yields an efficient estimate for $\beta$ since it is the maximum likelihood objective and so would be the preferred scoring rule in finite samples.

Correct specification of the conditional probability is a strong assumption and is difficult to defend unless there is good economic theory guiding the choice of the model. If the model is not correctly specified, ${\beta }^{\ast }$ depends on the chosen scoring rule. Section 12.1 examined how scoring rules are derived from weighting different utility functions. As the weighted average utility functions change, so do the approximate models that maximize them. Hence, the choice of scoring rule depends on which of these weighting schemes over utility functions is most relevant to the problem at hand.

Multistep forecasts of binary variables engender the same choice between using either a direct or an iterated method for constructing forecasts as discussed in chapter 7. The direct approach uses the likelihood

$$Q_T(\beta )=\frac{1}{T-h}\sum _{t=0}^{T-h}(y_{t+h}\ln p(z_t^{\prime }\beta )+(1-y_{t+h})\ln (1-p(z_t^{\prime }\beta ))),\text{\hspace{1em}(12.12)}$$

and hence computation of forecasts is straightforward. Iterated forecasts require taking expectations over multiple paths of $y_{t+1},\dots ,y_{t+h}$ . Kauppi and Saikkonen (2008) note that because of the binary nature of $\mathbf{y}$ such calculations are simpler than in the general case and provide (model-specific) formulas for constructing iterated forecasts. As noted in chapter 9, it is more likely that the direct method yields better forecasts when the model is misspecified.

When parametric specifications such as the probit or logit are employed to model the conditional probability, maximum likelihood (log scoring) is typically chosen as the objective function. Examples include Estrella and Mishkin (1998) and Wright (2006) who use probit models to forecast recessions with various financial variables.