20.1 EXPECTED LOSS WITH TRENDING VARIABLES

20.1 EXPECTED LOSS WITH TRENDING VARIABLES

To illustrate the first point, consider the following example where an autoregressive model with a root near 1 is the true data-generating process and we use a least squares estimator to obtain the model parameters; this can be thought of as a plug-in estimator or as the estimator that minimizes the average in-sample loss.

Example 20.1.1 (MSE with near-unit root process). Consider an $AR(1)$ model $y_{t+1}=\beta y_t+{\epsilon }_{t+1}$ , where ${\epsilon }_{t+1}\sim (0,{\sigma }^2)$ . The mean squared error for a one-step-ahead forecast at time T is

$$\begin{array}{rl}& E[(y_{T+1}-{\hat{\beta }}_T{y}_T{)}^2]=E[({\epsilon }_{T+1}-({\hat{\beta }}_T-\beta )y_T{)}^2]\\ & =E[{\epsilon }_{T+1}^2+({\hat{\beta }}_T-\beta {)}^2{y}_T^2]+{\mathbf{o}}_{\mathbf{p}}(1).\end{array}\text{\hspace{1em}(20.1)}$$

An exact result for this expression is difficult to establish even for the Gaussian case. As $T\to \infty ,$ , let $\beta ={\beta }_T=1-\gamma /T_{}$ , so we are considering triangular array asymptotics. For $\gamma =0$ we have a unit root; for $\gamma >0$ we have a root local to unity. Then $\begin{array}{r}T(\hat{\beta }-\beta )\Rightarrow {\left(\int W_{\gamma }(s{)}^{2}ds\right)}^{-1}\int W_{\gamma }(s)dW(s)\end{array}$ and $T^{-1/2}{y}_T\Rightarrow \sigma W_{\gamma }(1)$ , where $W(s)$ is a standard Brownian motion and $\begin{array}{r}{W}_{\gamma }(s)={\int }_0^s{e}^{-\gamma (s-r)}dW(r).\end{array}$ Hence, the term inside the expectation in (20.1) can be approximated by

$${\sigma }^2\left(1+T^{-1}{W}_{\gamma }(1{)}^2{\left(\int W_{\gamma }(s{)}^{2}ds\right)}^{-2}{\left(\int W_{\gamma }(s)dW(s)\right)}^2\right).$$

Again the order of the estimation error is $T^{-1}$ , although for a unit root or root local to unity the magnitude of the effect is different from the case with a single stationary regressor.

This example illustrates our main points. Despite both $y_{t+1}$ and $z_t=y_t$ having either a unit root or a near unit root, the expected loss exists and can be approximated by limit results in sufficiently large samples. As in the ergodic case, the term that arises from estimation error is of order ${\bar{T}}^{-1}$ . This is true even though our estimator for the autoregressive root is converging at rate $T$ rather than $T^{1/2}$ . The rate in the expected loss calculation remains at ${\check{T}}^{-1}$ because the value of the regressor $z_T$ also diverges. The order $T^{-1}$ term no longer equals k, the number of regressors, but is instead the expectation of a function of Brownian motions. This function depends on the value for $\gamma ,$ the local-to-unity parameter, and so depends on the model parameters.

The next example shows a similar result for the case with a time trend rather than a unit root.

Example 20.1.2 (Forecasting model with a deterministic trend). Consider the model $y_t={\beta }_1+{\beta }_{2}t+u_t={\beta }^{\prime }{x}_t+u_t$ , where $u_t\sim (0,{\sigma }^2),\beta =({\beta }_1,{\beta }_2{)}^{\prime }$ , and ${\mathbf{x}}_t=(1,t)$ . The OLS estimators for the regression coefficients are jointly normal with ${{\rm Y}}_T(\hat{\beta }-\beta {)}^d\mathcal{N}(0,{\sigma }^2\Sigma )$ where ${{\rm Y}}_T=\mathrm{diag}(T^{1/2},T^{3/2})$ and $\Sigma =$ $\mathbf{p}$ lim $\begin{array}{r}({{\rm Y}}_T^{-1}[{\sum }_{t=1}^T{x}_t{x}_t^{\prime }]{{\rm Y}}_T^{-1}{)}^{-1}\end{array}$ is a $(2\times 2)$ matrix whose elements are 1 and $1/3$ on the diagonal with $1/2$ on the off-diagonal. Since $T^{-3/2}({\hat{\beta }}_2-{\beta }_2)is{O}_{\hat{P}}(1),{\hat{\beta }}_2$ converges faster than ${\hat{\beta }}_1$ and we might expect that the effect of estimation error is of lower order for ${\beta }_2$ than for ${\beta }_1$ . However, this is not true because the additional T in the convergence rate offsets the time trend regressor which is also of order T. Assuming mean squared error loss and ignoring higher-order terms, we have

$$E[(y_{T+1}-x_t^{\prime }\hat{\beta }{)}^2]={\sigma }^2(1+T^{-1}{\iota }^{\prime }\Sigma \iota )={\sigma }^2\left(1+\frac{4}{T}\right).$$

Here the order of the effects of estimation error are the same for each coefficient. However, the overall effect of estimation error is bigger than in the stationary case with two regressors, $by2(T{)}^{-1}$

In this example the term of order $T^{-1}$ differs from the corresponding term in the ergodic case. In the trended case, the term depends on the particular type of trend (a linear time trend) but not on the parameters of the model. Again, the faster rate of convergence on the coefficient on the time trend does not translate into an estimation error of lower order.

A third point to note is that when the trending behavior is not correctly specified, this typically results in a mean squared error that grows as the sample size increases, rather than converging to some expected MSE for the problem.

Example 20.1.3 (MSE with misspecified trend). Consider again the model $y_t={\beta }_1+$ ${\beta }_{2}t+u_t={\beta }^{\prime }{x}_t+u_t$ where $u_t\sim (0,{\sigma }^2),\beta =({\beta }_1,{\beta }_2{)}^{\prime }$ , and ${\mathbf{x}}_t=(1,t)$ . However, suppose we misspecify the trend by including only a constant in the forecasting model, $i.e.,$ our forecast is the sample average of ${\gamma }_t$ using all available data. In this case we use $\begin{array}{r}{\hat{\beta }}_1=\widehat{T^{-1}}{\sum }_{t=1}^T{y}_t\end{array}$ to forecast ${\gamma }_{T+1}$ which yields an MSE,

$$\begin{array}{rlr}E[y_{T+1}-{\hat{\beta }}_1{]}^2={\sigma }^2+E[{\beta }_1+{\beta }_2(T+1)-{\hat{\beta }}_1{]}^2& & \\ & & \\ & & ={\sigma }^2(1+T^{-1})+{\beta }_2^2(T+1-T^{-1}{\sum }_{t=1}^{t}t{)}^2\\ & & \\ & & ={\sigma }^2(1+T^{-1})+{\beta }_2^2{T}^2/4+o(1),\end{array}$$

and so the MSE diverges as $T$ gets large.

Similar results hold for situations where ${\gamma }_t$ has a unit root but the forecasting model regards ${\gamma }_t$ as having a time trend, or vice versa. They also extend to situations where ${\gamma }_t$ has a unit root but the model for ${\gamma }_t$ as a function of other trending variables does not adequately account for the trending behavior in $y_t,e.g.$ , if the model is not cointegrating. $As$ for the case with a correctly specified trend, each case needs to be constructed individually and there are no general results. However, a unifying theme is that the MSE diverges when the trend is misspecified as the trend comes to dominate. In these cases, typically the parameters do not converge to pseudo-true values and a subset of the parameter estimates is not consistent for any fixed value as the sample size grows.

Under an unknown trend, a simple forecasting approach that is almost always feasible and often results in a nondivergent MSE (or any loss function for which expectations are finite) is to use $y_T$ to predict ${\gamma }_{T+h}$ . This prediction is known as the random walk forecast since it is optimal for that model. In this case, even though the mean or variance of ${\gamma }_{T+h}$ is diverging in $T,$ , the forecast error, $y_{T+h}-y_T$ , does not diverge, provided that the trending behavior is due to unit roots or time trends. For other types of trending behavior, e.g., near unit roots, the divergent behavior of $y_{T+h}-y_T$ is often much smaller and hence less of a problem.