7.3.2 Finite-Sample Properties of Forecasts from AR(1) Models

7.3.2 Finite-Sample Properties of Forecasts from AR(1) Models

Consider the $\mathrm{AR}(p)$ model where p is assumed to be known and greater than 0. Since there are no MA terms $(q=0)$ , only $\mathbf{p}$ autoregressive parameters have to be estimated. This involves projecting ${\gamma }_t$ on a fixed number of its own lagged values, all of which are observed, and so lends itself to least squares estimation. The regressor matrix can be constructed by dropping the first $\mathbf{p}$ observations. For example, for an AR(1) model we regress the vector $\{y_1,...,y_T\}$ on any deterministic terms and the regressor $\{y_0,\dots ,y_{T-1}\}$ , where T is the sample size. Assuming the data are covariance stationary, least squares estimates of the coefficients ${\varphi }_1,\dots ,{\varphi }_p$ are consistent and asymptotically normal.

Provided the model is correctly specified, such estimates are also asymptotically efficient. However, even if the model is correctly specified, least squares estimates do not have optimality properties in finite samples and will be biased (e.g., Kendall (1954), Kendall and Stuart (1961), Marriott and Pope (1954)). For the $\mathrm{AR}(1)$ model, the OLS estimate ${\hat{\varphi }}_1$ is biased towards 0. Specifically, Kendall (1954) shows that ${\hat{\varphi }}_1$ has a downward bias of $(1+3{\varphi }_1)/T_{;}$ , so $E[{\hat{\varphi }}_1]={\varphi }_1-((1+3{\varphi }_1)/T$ in an $\mathrm{AR}(1)$ model with a constant. For higher-order models, the biases are complicated and can go in either direction (e.g., Shaman and Stine (1988)).

This lack of small sample optimality for least squares estimates has led to a number of alternative estimation methods being used, the most popular of which is maximum likelihood estimation.4 This method differs from least squares in its treatment of the first $\mathbf{p}$ observations. These are dropped under least squares estimation but can be considered as draws from their unconditional distributions in the construction of the likelihood for the data.

Regardless of the estimation technique, the estimated parameters are likely to be biased. Forecasters are not, however, interested in properties of the coefficient estimates themselves but rather in the overall forecast error or loss. For example, the forecast error for the AR(1) model is given by

$$y_{t+h}-f_{t+h|t}=\left({\varphi }_1^h-{\hat{\varphi }}_1^h\right)y_t+\sum _{s=0}^{h-1}{\varphi }_1^s{\epsilon }_{t+h-s},$$

where ${\varphi }_1$ and ${\hat{\varphi }}_1$ are the true and estimated AR(1) coefficients, respectively. The key observation is that the bias in the parameter estimate $\left({\varphi }_1^h-{\hat{\varphi }}_1^h\right)$ interacts with the data $\{y_t,y_{t-1},\dots ,y_1\}$ used to construct the forecast. Interestingly, in a wide range of situations, the forecast errors are unconditionally unbiased, i.e., $E[y_{t+h}-$ $f_{t+h|t}]=0$ , so that a sequence of forecasts are unbiased “on average,” i.e., across realizations of a sequence of forecasts, provided that the innovations, $\epsilon ,$ have a symmetric distribution. The result is surprising since neither of the estimates, ${\hat{\mu }}_t.$ ${\hat{\varphi }}_t$ , is unbiased, but the two biases may cancel out.

In practice, this result may not be of too great interest to forecasters. A forecaster knows precisely which set of data $\{y_t,y_{t-1},\dots ,y_{t-p}\}$ is employed to construct the forecast and is therefore more likely to be interested in the bias conditional on these values. Unfortunately, forecast errors are generally not conditionally unbiased. Phillips (1979) constructs exact results for the distribution of one-step-ahead $(h=1)$ forecast errors from an $\mathrm{AR}(1)$ model estimated by $\mathrm{OLS}{.}^6$ Consider an $\mathrm{AR}(1)$ model without an intercept,

$$y_t={\varphi }_1{y}_{t-1}+{\epsilon }_t,{\epsilon }_t\sim \mathrm{iid}(0,{\sigma }^2).$$

Phillips shows that the forecast of ${\gamma }_{T+1}$ conditional on ${\gamma }_T$ is biased with a bias that depends on ${\varphi }_1$ and $y_T:$

$$E\left[y_{T+1}-f_{T+1|T}|y_T\right]=\frac{2{\varphi }_1}{T}{y}_T+O(T^{-2}).$$

Note that the direction of the bias in the forecast error depends on the sign of $y_T$ , with the bias being positive if the final observation of y is positive, $y_T>0$ . Accounting for the statistical dependence between ${\hat{\varphi }}_{1T}$ and ${\gamma }_T$ , Phillips characterizes the effect of estimation error on the approximate conditional distribution of the one-period forecast $f_{T+1|T}$ given ${\gamma }_T$ in a sample with $T$ observations. Let $f_{T+1|T}^{\ast }={\varphi }_1{y}_T$ be the infeasible population forecast, i.e., the forecast without estimation error, while ${\hat{f}}_{T+1|T}={\hat{\varphi }}_{1T}{y}_T$ is the feasible forecast based on the estimated AR(1) coefficient $\begin{array}{r}{\hat{\varphi }}_{1T}={\left({\sum }_{\tau =1}^T{y}_{\tau -1}^2\right)}^{-1}\left({\sum }_{\tau =1}^T{y}_{\tau -1}{y}_{\tau }\right)\end{array}$ . Further, let be the normal $c.d.f.$ , while ${\Phi }^{\prime }$ is the normal density. Phillips shows that when $y_T>0$ , as an approximation, the scaled difference between the feasible and infeasible one-step-ahead forecast follows the distribution

$$\begin{array}{l}P(\sqrt{T}(f_{T+1|T}-f_{T+1|T}^{\ast })\le x|y_T)\\ =\Phi \left(\frac{x}{y_T\sqrt{1-{\varphi }_1^2}}\right)\\ +\frac{{\varphi }_1}{\sqrt{T(1-{\varphi }_1^2)}}{\Phi }^{\prime }\left(\frac{x}{y_T\sqrt{1-{\varphi }_1^2}}\right)\left(2+\frac{x^2}{y_T^2(1-{\varphi }_1^2)}-{\left(\frac{y_T}{{\sigma }_y}\right)}^2\right)+\mathcal{O}(T^{-1}),\end{array}$$

where ${\sigma }_y=\sigma /\sqrt{1-{\varphi }_1^2}$ . When $y_T<0$

$$\begin{array}{l}P(\sqrt{T}(f_{T+1|T}-f_{T+1|T}^{\ast })\le x|y_T)\\ =\Phi \left(\frac{-x}{y_T\sqrt{1-{\varphi }_1^2}}\right)\\ -\frac{\varphi }{\sqrt{T(1-{\varphi }_1^2)}}{\Phi }^{\prime }\left(\frac{x}{y_T\sqrt{1-{\varphi }_1^2}}\right)\left(2+\frac{x^2}{y_T^2(1-{\varphi }_1^2)}-{\left(\frac{y_T}{{\sigma }_y}\right)}^2\right)+\mathcal{O}(T^{-1}).\end{array}$$

Notice from (7.28) and (7.29) that, assuming ${\varphi }_1>0$ , the distribution of forecasts has a negative skew when $y_T>0$ , whereas it has a positive skew when $y_T<0.$ , so that there is a skewness towards the origin (0) which is the mean of the process. The dependence between the least squares estimate of $\begin{array}{r}{\varphi }_1,{\hat{\varphi }}_{1T}={\sum }_{t=1}^T{y}_t{y}_{t-1}\widehat{/}{\sum }_{t=1}^T{y}_{t-1}^2\end{array}$ , and ${\gamma }_T$ deepens the skewness towards the origin that is also present when ${\hat{\varphi }}_{1T}$ is based on data that are independently distributed of ${\gamma }_T$

Phillips (1979) (Theorem 3) also shows that the distribution of the forecast error conditional on ${\gamma }_T$ can be approximated by

$$\begin{array}{l}P(y_{T+1}-f_{T+1|T}\ge -x|y_T)\\ =\Phi \left(\frac{x}{\sigma }\right)\\ +{\Phi }^{\prime }\left(\frac{x}{\sigma }\right)\left(\frac{1}{T}\left[\frac{{\varphi }_1}{\sqrt{1-{\varphi }_1^2}}\right]\left(\frac{y_T}{{\sigma }_y}\right)\left(3-\frac{y_T^2}{{\sigma }_y^2}\right)-\frac{y_T^2}{2T{\sigma }_y^2}\left(\frac{x}{\sigma }\right)\right)+\mathcal{O}(T^{-2}).\end{array}$$

Hence, the approximate conditional distribution of the forecast error $y_{T+1}-f_{T+1|T}$ is positively skewed when $y_T>0$ and negatively skewed when $y_T<0$ . This is consistent with the reverse direction of the skew in $f_{T+1|T}$ discussed above.