4.2.2 Estimation Based on Functions Other Than the Loss Function

4.2.2 Estimation Based on Functions Other Than the Loss Function

An alternative to loss-based estimation is to use plug-in estimators derived from a variety of popular estimation methods. The leading case is perhaps to use maximum likelihood methods to estimate the model parameters and then plug these into the forecast model. Alternatively, often estimates obtained from least squares regression are used in forecast problems involving very different loss functions.

Maximum likelihood estimates for $\theta$ or $\beta$ are commonly used in forecasting. Maximum likelihood methods typically require the construction of a likelihood for the data, although limited information maximum likelihood methods might be employed when models are complicated or it is thought that some of the predictor variables can be treated as exogenous. We can then construct forecasts from the model $f(z_T,{\hat{\beta }}_{\mathrm{MLE}})$ , using that ${\hat{\beta }}_{\mathrm{MLE}}$ is typically easily derived from ${\hat{\theta }}_{\mathrm{MLE}}$ and often is just a subset of the latter.

Example 4.2.2 (Forecast of a binary outcome variable). Suppose Y is a binary variable that takes values of either 0 or 1 so the optimal forecast is a function of the “success” probability, $P\left[Y=1|Z\right]=E\left[Y|Z\right]$ . To generate a forecast of Y we might use a linear index model, $P(Y=1|Z)=F(z^{\prime }\beta )$ for some $c.d.f.$ , F . Popular choices are probit and logit models, the maximum likelihood estimates for which ${\hat{\beta }}_{\mathrm{MLE}}$ can be employed in a decision rule such as $1(F(z^{\prime }{\hat{\beta }}_{\mathrm{MLE}})>c)$ , which forecasts $y=1$ if the predicted probability falls above a certain threshold, c, and otherwise forecasts $y=0$

Using maximum likelihood estimates to construct a forecast model requires some knowledge of the form of $p_Y(y|z)$ and $p_z(z)$ . When the likelihood is correctly specified so the entire joint density of $Y$ and $Z$ posited by the forecaster is the true data-generating process, under fairly general conditions the maximum likelihood estimators for the parameters ${\hat{\theta }}_{\mathrm{MLE}}$ will be consistent for the true parameters of the data-generating process, θ. Assuming that the random variables from which the data are drawn have absolutely continuous distributions, and that the expected likelihood exists and is continuous in $\theta ,$ the consistency result follows provided that (i) the model is identified (so the expected likelihood has a unique maximum at the true θ for the data-generating process) and (ii) the suitably scaled log-likelihood

$$\log \left\{\prod _{t=0}^{T-1}{\hat{p}}_Y(y_{t+1}|z_t,z_{t-1},\dots ,\theta ){\hat{p}}_Z(z_t|z_{t-1},z_{t-2},\dots ,\theta )\right\}$$

satisfies a uniform law of large numbers. This result follows from White (1996, Theorem 4.2).

In practice, the assumption that the likelihood is correctly specified is unlikely to hold and we can instead view the estimator as a quasi-maximum likelihood estimator (QMLE). QMLEs may still prove to have good properties, as discussed in White (1996), although their properties must be established in relation to the forecaster’s loss function. When the kernel of the likelihood function agrees with the forecaster’s loss function, the results from above apply.

Example 4.2.3 (Quasi-maximum likelihood estimation with a normal distribution). The pseudo MLE from a model that assumes normality can be equivalent to a forecast that minimizes mean squared errors. To see this, let $\theta =(\beta ,{\sigma }^2)$ , so

$${\hat{\theta }}_{\mathrm{MLE}}=\arg \underset{\theta \in \Theta }{\max }(\ln ({\sigma }^2{)}^{-T/2}\exp \{-\frac{1}{2{\sigma }^2}\sum _{t=0}^{T-1}(y_{t+1}-f(z_t,\beta ){)}^2\left\}\right),$$

and

$$\hat{\beta }=\arg \min \sum _{t=0}^{T-1}(y_{t+1}-f(z_t,\beta ){)}^2.$$

Here the estimates of the forecasting model $f(z_t,\hat{\beta }({\hat{\theta }}_{\mathrm{MLE}}))=f(z_t,\hat{\beta })$ and so forecasts based on the MLE values may be reasonable even when the density is misspecified.

Example 4.2.4 (QMLE for tick-exponential family). Example 4.1.2 showed that the M-estimator for the relevant quantile provided an estimator for the forecast model under lin-lin loss. Komunjer (2005) examines QMLEs for the tick-exponential family whose pseudo densities take the form

$$\begin{array}{r}{p}_t^{\alpha }(y,\eta )=\exp \left[-(1-\alpha )(b(y)-a(\eta ))1(y\le \eta )+\alpha (b(y)-a(\eta ))1(y>\eta )\right],\end{array}$$

where η is the model for the conditional quantile. When the tick-exponential is an asymmetric Laplace (double exponential) density, the tick-exponential QMLEs coincide with the conventional quantile regression estimator. Setting $b(y)=y$ and $a(\eta )=\eta ={\beta }^{\prime }z$ yields a likelihood that is an order-preserving transform of the M-estimator based on the log function. It follows from Komunjer’s results that the QMLE provides the best forecast among linear models.

Equivalence results such as those in the previous examples do not hold more generally. Rather, the forecasts based on MLEs will differ from those that minimize the loss function. Under reasonable conditions forecasts based on the correct loss function have the property that—at least in large enough samples—they yield losses that are approximately the smallest possible ones given the loss function and the forecasting model. In general we therefore expect forecasts based on the correct loss function to be superior to those based on MLEs. In small samples, efficiency may be more of an issue and it is possible that utility- or loss-based estimators are more sensitive to outliers than MLE-based estimators and so could lead to worse finitesample performance. This issue has to be evaluated in each individual case.

Another case where the parameters are estimated under objective functions other than the forecaster’s loss arises when standard econometric methods such as least squares estimation are used even though the forecaster has a different loss function such as mean absolute error loss or perhaps is interested in predicting turning points of the data. Indeed, the large literature on turning point prediction often examines forecasts in the context of logit or probit models; see, e.g., Chauvet and Potter (2005). It follows directly from the fact that ${\beta }^{\ast }$ differs for different loss functions (see the examples in chapter 3) that this approach can easily lead to forecast models that do not achieve average losses that converge to the average expected loss given ${\beta }^{\ast }$ . For further discussion and examples, see Weiss (1996).

Example 4.2.5 (Least squares estimates under absolute error loss). Suppose least squares estimates are employed to generate forecasts under an absolute error loss function

$$(T-1{)}^{-1}\sum _{t=1}^{T-1}|y_{t+1}-{\hat{\beta }}_{\mathrm{ols}}^{\prime }{x}_t{|}^{p}E[|Y-f(Z,{\beta }_{\mathrm{ols}}^{\ast })|]\ge E[|Y-f(Z,{\beta }^{\ast })|],\text{\hspace{1em}(4.9)}$$

where ${\beta }^{\ast }=\arg {\min }_{\beta }$ $E_Y[|Y-f(Z,\beta )|]$ . Equality in (4.9) holds when the conditional mean and conditional median are equivalent, $e.g.$ , when the conditional distribution of Y given X is symmetric. Equality of the expected loss need not hold, however, for skewed or asymmetrical distributions.