16.3.1 West’s Result under Squared Error Loss

16.3.1 West’s Result under Squared Error Loss

Conditions (1)–(5) in West’s theorem may appear somewhat abstract and hard to verify. To make the result more accessible, we next discuss West’s theorem for the common case with squared error loss and univariate forecasts based on a linear regression model, $y_{t+1}={\beta }^{\prime }{x}_t+{\epsilon }_{t+1}$ , whose parameters are estimated by recursive OLS. Assumption (1) in West’s theorem involves both the loss function and the method for constructing the forecast. Under squared error loss, and assuming $h=1$ ，

$$l_{t+1|t}(\beta )=(y_{t+1}-{\beta }^{\prime }{x}_t{)}^2,$$

and so

$$\begin{array}{rlr}\frac{\partial l_{t+1\mid t}(\beta )}{\partial \beta }=-2(y_{t+1}-{\beta }^{\prime }{x}_t)x_t,& & \\ & & \\ & \frac{{\partial }^2{l}_{t+1\mid t}(\beta )}{\partial \beta \partial {\beta }^{\prime }}=2x_t{x}_t^{\prime },& \end{array}$$ showing that the loss is twice differentiable.

Turning to the second assumption in West’s theorem, the parameter estimate, ${\hat{\mathbf{\beta }}}_t$ , obtained under an expanding window takes the form

\hat { \beta } _ { t } - \beta ^ { * } = \mathopen { } \mathclose \bgroup \left( ( t - 1 ) ^ { - 1 } \sum _ { s = 1 } ^ { t - 1 } x _ { s } x _ { s } ^ { \prime } \aftergroup \egroup \right) ^ { - 1 } \mathopen { } \mathclose \bgroup \left( ( t - 1 ) ^ { - 1 } \sum _ { s = 1 } ^ { t - 1 } x _ { s } (y _ { s + 1} - \beta ^ {* \prime} x _ { s}) \aftergroup \egroup \right) .

This case is covered by assumption (2) with $\begin{array}{r}B(t)=((t-1{)}^{-1}{\sum }_{s=1}^{t-1}{x}_s{x}_s^{\prime }{)}^{-1}\end{array}$ ; B (t) is the inverse of the variance–covariance matrix of $x_t$ and so has limit B under relatively mild conditions on $x_t$ . This leaves the definition of ${\eta }_{s+1}={\eta }_{s+1}({\beta }^{\ast })$ to be ${\eta }_{s+1}=x_s(y_{s+1}-{\beta }^{\ast \prime }{x}_s)$ , where the dependence on ${\beta }^{\ast }$ is explicit. We define ${\beta }^{\ast }$ as the linear projection of ${\gamma }_{s+1}$ on $x_s$ , so $E\left[\eta ({\beta }^{\ast })\right]=0$ by construction. If the regressors were nonstationary, $B(t)$ would not converge to B and so this case is ruled out by assumption (2).

Assumption (3) in West’s theorem is a smoothness condition. In the example here, the second derivative does not depend on the parameters and so this assumption follows directly given assumptions on the regressors. Assumption (4) places limited dependence and moment restrictions on the regressors and forecast errors, which again follows under weak assumptions on the DGPs. For example, in the very common case where $\{x_t,{\epsilon }_t\}$ are covariance stationary, these conditions will hold and the theorem applies.

West’s theorem shows that in the special case where $\Omega =S_y(0)$ , the covariance matrix of the sample mean can be approximated by the usual spectral density estimate, ignoring sampling variation when we construct the estimates. One such special case arises when the training (estimation) sample used to construct the forecasts $\left(T_R\right)$ overwhelms the out-of-sample period $\left(T_P\right)$ so $\pi =0$ . Intuitively it makes sense that parameter estimation error is not relevant in this situation.

A second special case is less obvious. Under squared error loss, the forecasts are generated so that the prediction error is uncorrelated with the predictors and so

$$F=E\left[{\frac{\partial l_{t+1\mid t,1}(\beta )}{\partial \beta }|}_{{\beta }^{\ast }}\right]=-2E(y_{t+1}-{\beta }^{\ast \prime }{x}_t)x_t^{\prime }=0,$$

and so the usual asymptotic results apply. This combination of mean squared error loss and forecasts based on a linear regression model is very popular in applied work. For this pairing it is straightforward to compute standard errors for the expected loss, even though this is uncommon to do in practice. One only need treat the time series of squared error losses as raw data, compute the mean and spectral density at zero frequency in the usual way (e.g., by applying the methods of Newey and West (1987) or Andrews (1991)), and report the associated standard error.

In other cases, sampling error in the parameter estimates must be taken into account. Indeed, when the second and third terms in (16.15) are nonzero, the usual variance estimator is not consistent as it omits the extra two terms in the result. For such cases, West (1996) suggests estimators for the additional terms.

We next present a simple example that illustrates the individual terms in West’s theorem.

Example 16.3.1 (Forecasting a univariate i.i.d. variable by its mean under squared error loss). Let $y_t\sim$ ind $\mathrm{N}(\mu ,{\sigma }^2)$ and suppose the one-step-ahead forecast model is based on the sample mean,

$$y_{t+1}=\mu +{\epsilon }_{t+1},$$

with $\begin{array}{r}{\hat{\mu }}_t=t^{-1}{\sum }_{\tau =1}^t{y}_{\tau }\end{array}$ . Under squared error loss, $L(\mu )=E\left[L\left(f(z_t,{\beta }^{\ast }),y_{t+1}\right)\right]=$ $E[(y_{t+1}-\mu {)}^2]={\sigma }^2$ . The model is correctly specified in this case and so ${\beta }^{\ast }=\mu$ Under squared error loss the mean value expansion is exact. Using that

$$\begin{array}{rl}& \frac{\partial l_{t+1\mid t}(\beta )}{\partial \beta }=\frac{\partial (y_{t+1}-\mu {)}^2}{\partial \mu }=-2(y_{t+1}-\mu ),\\ & \frac{{\partial }^2{l}_{t+1\mid t}(\beta )}{\partial \beta \partial {\beta }^{\prime }}=2,\end{array}$$

the three terms in (16.15) are therefore given by

$$\begin{array}{rl}& (y_{t+1}-{\hat{\mu }}_t{)}^2=([y_{t+1}-\mu ]-[{\hat{\mu }}_t-\mu ]{)}^2\\ & =(y_{t+1}-\mu {)}^2-2(y_{t+1}-\mu )({\hat{\mu }}_t-\mu )+({\hat{\mu }}_t-\mu {)}^2.\end{array}$$

Averaging over the $T_P$ out-of-sample observations, we get

$$\begin{array}{l}{T}_P^{-1}\sum _{i=1}^{T_P}(y_{T_R+i}-{\hat{\mu }}_{T_R+i-1}{)}^2={\mathcal{T}}_P^{-1}\sum _{i=1}^{T_P}(y_{T_R+i}-\mu {)}^2\\ -2{\mathcal{T}}_P^{-1}\sum _{i=1}^{T_P}(y_{T_R+i}-\mu )({\hat{\mu }}_{T_R+i-1}-\mu )\\ +{\mathcal{T}}_P^{-1}\sum _{i=1}^{T_P}({\hat{\mu }}_{T_R+i-1}-\mu {)}^2.\end{array}$$

The first of these terms is the average loss with no uncertainty about $\mu .$ . Contributions from parameter uncertainty come from the remaining terms and will disappear asymptotically.

West’s result excludes some interesting cases where the assumptions of the theorem fail to hold. One such situation arises when a vector of forecasts is considered, one forecasting model nests other models, and $\pi \ne 0$ . Suppose the training sample used for parameter estimation is large so the asymptotic approximation sets the coefficients in the model used to generate the forecasts $({\hat{\beta }}_t)$ at their pseudo-true values, ${\beta }^{\ast }$ . For nested models where the small model is the true one, these parameter values are the same and hence the two sets of forecasts are identical and the covariance matrix will be singular. This is ruled out by assumption (4) in West’s theorem since $S_y(0)$ is assumed to be nonsingular. This situation is not relevant when evaluating a single forecast, but becomes important in the comparison of multiple forecasts described in the next chapter.

For some cases the loss function is not twice differentiable. Examples of loss functions that do not satisfy this property are the MAE, lin-lin, and asymmetric quadratic loss functions, that are not differentiable at a single point. To some extent this requirement reflects the proof in West (1996) which assumes a twice continuously differentiable loss function in order to apply a second-order mean value theorem. Indeed, other methods of proof are available. McCracken (2000) relaxes the differentiability condition with a smoothness condition that requires the expected loss (or difference in losses) to be continuously differentiable around the probability limits of the parameter estimates, while maintaining other conditions similar to those in West (1996). Other loss functions, such as the binary forecasting loss function (see chapter 2) and profit functions that rely on discrete actions cannot be handled in this way, and require a separate approach to establishing the consistency of expected loss, its rate of convergence, and also conditions under which expected loss is asymptotically normal.