4.1 LOSS-BASED ESTIMATORS

4.1 LOSS-BASED ESTIMATORS

A straightforward approach to constructing a forecasting model1 is to specify the model up to a set of unknown parameters, $\beta \in B,$ , where $\mathbf{B}$ is the set of possible parameter values, and then estimate these parameters directly using the sample analog for the model’s expected loss based on data $\{z_t,y_{t+1}{\}}_{t=0}^{T-1}$ . This procedure, usually described as an M-estimator, yields2

$${\hat{\beta }}_T=\arg \underset{\beta \in \mathcal{B}}{\min }{T}^{-1}\sum _{t=0}^{T-1}\mathcal{L}(f(z_t,\beta ),y_{t+1}).\text{\hspace{1em}(4.2)}$$

The forecast of next period’s outcome, $y_{T+1},$ denoted by $f(z_T,{\hat{\mathbf{\beta }}}_T)$ , is then constructed using the estimates, ${\hat{\mathbf{\beta }}}_T$ . Notice that there is no assumption (or expectation) that the forecast model $f(z_t,\beta )$ is correctly specified.

Under relatively general conditions it can be established that

T ^ { - 1 } \sum _ { t = 0 } ^ { T - 1 } L ( f ( z _ { t } , \beta ^ { * * } ) , \boldsymbol { y } _ { t + 1 } ) - T ^ { - 1 } \sum _ { t = 0 } ^ { T - 1 } L ( f ( z _ { t } , \boldsymbol { \hat { \beta }}} ) , \boldsymbol { y } _ { t + 1 } ) = o _ {P } ( 1 ) ,\tag{4.3}

where

$${\beta }^{\ast \ast }=\arg \underset{\beta \in \mathcal{B}}{\min }\underset{T\infty }{\lim }{T}^{-1}\sum _{t=0}^{T-1}E[\mathcal{L}(f(z_t,\beta ),y_{t+1})].\text{\hspace{1em}(4.4)}$$

Here we use the notation ${\beta }^{\ast \ast }$ to denote the average (over the sample) best choice for $\beta ,$ as opposed to ${\beta }^{\ast }$ defined in the previous chapter as the best choice at time T. As discussed below, these may differ. Proofs of results such as (4.4) are typically broken into two steps: first show that the objective function converges uniformly in $\beta ,$ then show that ${\hat{\mathbf{\beta }}}_T\to P{\mathbf{\beta }}^{\ast \ast }$ . A wide range of assumptions can be used to establish the result. Differences in the required assumptions reflect the form of stationarity of the data (strict, second order, or global) and type of time-series dependence assumed for the data (mixing conditions, near epoch dependence, summability assumptions, etc.), and the degree of smoothness of the objective function. Because of such differences, any attempt to write down assumptions and prove general results will be both lengthy and incomplete. For i.i.d. data, a summary of results is provided in Newey and McFadden (1994). For dependent data, Wooldridge (1994) is an excellent source for results.

Drawing on Wooldridge (1994, Theorems 4.2 and 4.3), we simply present a set of assumptions that both illustrate the character of the results and are general enough to cover many forecast situations. Assume the following.

1. $\beta \in B\subset {\mathbb{R}}^k$ , where $\mathbf{B}$ is compact.

2. $L(f(Z_t,\beta ),Y_{t+1})$ and $f(Z_t,\beta )$ are measurable functions in $\beta$ and continuous in $\{z_t,y_{t+1}\}$ on $B.$

3. For each $\beta \in B,L(f(Z_t,\beta ),Y_{t+1})$ satisfies a weak law of large numbers (LLN).

4. There exists a function $c_t(z_t,y_{t+1})\ge 0$ such that

(a) for all ${\beta }_1,{\beta }_2\in \mathcal{B}$

$$|L(f(z_t,{\beta }_1),y_{t+1})-L(f(z_t,{\beta }_2),y_{t+1})|\le c_t(z_t,y_{t+1})||{\beta }_1-{\beta }_2||.$$

(b) $c_t(Z_t,Y_{t+1})$ satisfies a weak LLN.

Under these assumptions

$$\underset{\beta \in \mathcal{B}}{\max }\left|T^{-1}\sum _{t=0}^{T-1}L(f(z_t,\beta ),y_{t+1})-T^{-1}\sum _{t=0}^{T-1}E[L(f(z_t,\beta ),Y_{t+1})]\right|{\to }^{p}0.\text{\hspace{1em}(4.5)}$$

This result gives a uniform weak LLN for the objective function over the possible parameters of the forecast model. The result is for a stronger case than that in Wooldridge (1994) because we assume that the loss function does not vary over time, which is a typical assumption for most forecast problems, although the results can be extended to allow the function itself to change.

Application of the result in (4.5) still requires proof of the two weak LLNs in the theorem, which in turn requires stationarity and dependence assumptions. Weak LLN results follow fairly directly from general assumptions, for example if (2) above holds, $[y_{t+1},z_t{]}^{\prime }$ is φ-mixing of size $-r/(2(r-1)),r\ge 1$ or $[y_{t+1},z_t{]}^{\prime }$ is α- mixing of size $-r/(r-1)$ , where $r>1$ , and $E|L(f(z_t,\beta ),y_{t+1}){|}^{r+\delta }<\Delta <\infty$ for some $\delta >0$ and all t, then (3) above holds (see White, 2001, Corollary 3.48), where the addition of assumption (2) ensures that $L(f(z_t,\beta ),y_{t+1})$ is mixing for any $\beta .$ . Given these assumptions and the assumption that there is a unique maximizer to lim $\begin{array}{r}.{}_{T\infty }{T}^{-1}{\sum }_{t=0}^{T-1}E[L(f(Z_t,\beta ),Y_{t+1})]\end{array}$ , it follows that ${\hat{\mathbf{\beta }}}_T$ in (4.2) is consistent for ${\beta }^{\ast \ast }\mathrm{in}(4.4)$

Results such as (4.5) suggest that—given enough data—the average loss from using $\hat{\mathbf{\beta }}$ in the forecasting model is close to the smallest possible value of lim $\begin{array}{r}{\Omega }_{{}^{\backslash T\infty }}{T}^{-1}{\sum }_{t=0}^{T-1}E[L(f(Z_t,\beta ),Y_{t+1})]\end{array}$ , holding the forecast model $f(z_t,\beta )$ constant. What we really want is a forecast model $f(z_t,\hat{\mathbf{\beta }})$ that comes close to obtaining the smallest possible conditional expected loss $E[L(f(Z,\hat{\mathbf{\beta }}),Y)]$ ; in our notation this is $E[L(\bar{f}(Z_T,{\hat{\beta }}_T),Y_{T+1})]$ . An alternative way to make this point is to note that we want $\hat{\mathbf{\beta }}\to P{\mathbf{\beta }}^{\ast }$ , but (4.5) establishes convergence at ${\beta }^{\ast \ast }$ which need not be equal to ${\beta }^{\ast }$

The objective we have minimized and the objective we would like to minimize are the same if the data are strictly stationary, i.e., under the assumption that the data are drawn from a strictly stationary process so $\begin{array}{r}{T}^{-1}{\sum }_{t=0}^{T-1}E\left[L(f(Z_t,\beta ),Y_{t+1})\right]=\end{array}$ ${\mathcal{E}}_{\tau }[\mathcal{L}(f({\mathcal{Z}}_{\mathcal{T}},\beta ),{\mathcal{Y}}_{\mathcal{T}+1})]$ . This is a very strong assumption, and unlikely to hold for most forecasting applications. For parameter estimators such as linear projections that depend only on the first two moments of $\{Y,Z\}$ , the assumption of covariance stationarity suffices. More likely in any forecast situation, the data are heterogeneous in the sense that the data-generating process changes over time. Using past data to estimate the forecast model in such situations yields an estimator that works “on average,” as opposed to being tailored towards the current forecasting problem. We illustrate this point in the following example.

Example 4.1.1 (Best average versus best current model under MSE loss). Suppose that $\overline{Y_{t+1}}|z_t\sim \mathrm{N}({\beta }_1{z}_t+\overline{{\beta }_2{z}_t^2},{\sigma }^2)$ and $z_t$ is mean 0 with heterogeneous second and third moments denoted ${\sigma }_{zt}^i,i=2,3.$ . Let the forecast model $f_t(z_t,\beta )=\beta z_t$ . Then under MSE loss $E[L(f(Z_T,{\hat{\beta }}_T),Y_{T+1})]=E(Y_{T+1}-\beta Z_T{)}^2$ which is minimized at $\beta =$ ${\beta }_1+{\beta }_2{\sigma }_{zT}^3/{\sigma }_{zT}^2.$ . In this model $E\left[L(f(Z_t,\beta ),Y_{t+1})\right]=C_t+({\beta }_1-\beta {)}^2{\sigma }_{zt}^2+2({\beta }_1-\beta$ $\beta ){\beta }_2{\sigma }_{zt}^3$ , where $C_t$ does not depend on β. Assuming that $\begin{array}{r}{\bar{\sigma }}_z^i={\lim }_{T\infty }{T}^{-1}\sum {\sigma }_{zt}^i\end{array}$ is finite, ${\tilde{\beta }}^{\ast }={\beta }_1+{\beta }_2{\bar{\sigma }}_z^3/{\bar{\sigma }}_z^2\ne \widehat{{\beta }_1}+{\beta }_2{\sigma }_{zT}^3/{\sigma }_{zT}^2$ at T. As usual the OLS coefficient gives the best “average” fit and here this differs from the best current model.

Note that (4.5) holds for a broad class of loss functions and forecast models, including loss functions that are not differentiable everywhere. This generality comes at a cost—some of the assumptions are stronger than we might prefer for some oftenused loss functions. For example, under MSE loss where both the loss function is differentiable and a closed-form solution for ${\hat{\mathbf{\beta }}}_T$ is available, we might not assume that the parameter space is compact. Indeed, for this simple problem, consistency results that do not require compactness are available for the least squares estimator.

For more complicated forecasting models $(e.g.$ , nonlinear models) the assumption of compactness fits well with the common forecasting approach of using an “insanity filter,” which ignores the forecast if the forecast is deemed too far from the current outcome. Given the data, this could be considered a bound on the parameters that one is willing to entertain, which essentially amounts to a compactness assumption.

Some results have been established for particular combinations of loss functions and forecast models. The following example illustrates this.

Example 4.1.2 (M-estimator under lin-lin loss). Consider the piecewise linear (linlin) loss function with $\alpha \in (0,1)$ and a linear forecast model of the form ${\beta }^{\prime }{z}_T$ :

$$\begin{array}{rl}& L(f(z_T,\beta ),y_{T+1})=\alpha (y_{T+1}-{\beta }^{\prime }{z}_T)1(y_{T+1}>{\beta }^{\prime }{z}_T)\\ & -(1-\alpha )(y_{T+1}-{\beta }^{\prime }{z}_T)1(y_{T+1}\le {\beta }^{\prime }{z}_T).\end{array}$$

Loss-based estimation of the forecasting model suggests using the analog

$$\begin{array}{c}\hat{\beta }=\arg \underset{\beta }{\min }{T}^{-1}\sum _{t=0}^{T-1}\alpha (y_{t+1}-{\beta }^{\prime }{z}_t)1(y_{t+1}>{\beta }^{\prime }{z}_t)\\ -(1-\alpha )(y_{t+1}-{\beta }^{\prime }{z}_t)1(y_{t+1}\le {\beta }^{\prime }{z}_t).\end{array}$$

This is the M-estimator for the quantile α. Komunjer (2005) gives assumptions under which the M-estimator of $\hat{\mathbf{\beta }}$ is consistent for ${\beta }^{\ast \ast }$ , including results for more general models.

When the loss function is twice differentiable everywhere, we can use expansions to establish the above results. A second-order mean value expansion yields

$$\begin{array}{rl}& T^{-1}\stackrel{\mathcal{T}\prod }{{}_{t=0}}I(f(z_t,{\beta }^{\ast }),y_{t+1})\\ & =T^{-1}\stackrel{\mathcal{T}\prod ^{r\to 1}}{{}_{t=0}}I(f(z_t,\hat{\beta }),y_{t+1})-[T^{-1}\stackrel{T\to 1}{{}_{t-0}}\frac{\partial L(\cdot )}{\partial \beta }{|}_{\beta =\hat{\beta }}](\hat{\beta }-{\beta }^{\ast \ast })\\ & +\frac{1}{2}(\hat{\beta }-{\beta }^{\ast \ast }{)}^{\prime }[T^{-1}\stackrel{T\to 1}{{}_{t=0}}\frac{{\partial }^{2}L(\cdot )}{\partial \beta \partial {\beta }^{\prime }}{|}_{\beta =\hat{\beta }}](\hat{\beta }-{\beta }^{\ast \ast })\\ & =T^{-1}\stackrel{T\to 1}{{}_{t=0}}I(f(z_t,\hat{\beta }),y_{t+1})+\frac{1}{2}(\hat{\beta }-{\beta }^{\ast \ast }{)}^{\prime }[T^{-1}\stackrel{T-1}{{}_{t=0}}\frac{{\partial }^{2}L(\cdot )}{\partial \beta \partial {\beta }^{\prime }}{|}_{\beta =\hat{\beta }}](\hat{\beta }-{\beta }^{\ast \ast }),\end{array}$$

for some intermediate value, $\hat{\beta }$ , between $\hat{\mathbf{\beta }}$ and ${\beta }^{\ast \ast }$ . Here we used the first-order optimality condition, $E\left[\partial L(\cdot )/\partial \beta \right]=0$ , when evaluated at $\beta =\hat{\mathbf{\beta }}$ . Provided that $\hat{\mathbf{\beta }}$ is consistent for ${\beta }^{\ast \ast }$ and $\begin{array}{r}{T}^{-1}{\sum }_{t=0}^{T-1}{({\partial }^{2}L(\cdot )/\partial \beta \partial {\beta }^{\prime })}_{{\beta }^{\ast \ast }}\end{array}$ does not diverge, it follows that the average loss evaluated at the pseudo-true value and estimated value, $\hat{\mathbf{\beta }},$ become similar.

The assumption of a unique maximizer may not hold in practice. For example, consider estimation of the parameters of an ARMA–GARCH model (discussed further in chapters $7$ and 13) under MSE loss. Under this type of loss, the optimal forecast is simply the conditional mean and so the GARCH volatility parameters (other than the intercept) are unidentified. It may not be obvious that a lack of identification of the model is of practical concern for the forecaster since this simply means that two forecast models are equally good in terms of their expected (or average) loss. Using an unidentified model could, however, make it difficult to explain the forecast if two models yield the same expected outcome.

Finally, notice that a key assumption here is that we can formulate a parametric forecast model. We examine nonparametric methods in chapter 11.