7.3 FORECASTING WITH ARMA MODELS

7.3 FORECASTING WITH ARMA MODELS

We next cover the classical theory for forecasting with ARMA models. This area is well understood, so we provide only a few key results and formulas which will prove useful in the subsequent analysis. Next, we discuss properties of forecasts using AR models with estimated parameters, direct versus iterated procedures for generating multiperiod forecasts, and forecasts of variables that have unit roots.

7.3.1 Classical Theory of Forecasting with ARMA Models

Simple and well-understood procedures exist for using ARMA models to generate forecasts of future outcomes. Prediction is particularly straightforward for purely autoregressive, $\mathrm{AR}(p)$ , models, i.e., models without moving average terms:

$$y_{t+1}={\varphi }_1{y}_t+\cdot \cdot \cdot +{\varphi }_p{y}_{t-p+1}+{\epsilon }_{t+1},{\epsilon }_{t+1}\sim \mathrm{WN}(0,{\sigma }^2).\text{\hspace{1em}(7.17)}$$

To introduce the mechanics of how ARMA forecasts are generated, we initially treat the parameters as known and thus ignore estimation error. Using that $P\left[{\epsilon }_{t+1}|Z_t\right]=0$ and $\{y_{t-p+1},\dots ,y_t\}\in Z_t$ , the linear forecast of $y_{t+1}$ given $Z_t$ becomes

$$f_{t+1|t}(Z_t,\beta )={\varphi }_1{y}_t+\cdot \cdot \cdot +{\varphi }_p{y}_{t-p+1},\text{\hspace{1em}(7.18)}$$

with $\beta =({\varphi }_1,\dots ,{\varphi }_p)$ and $Z_t=\{y_1,\dots ,y_t\}$ . When generating forecasts multiple steps ahead, unknown values of $y_{t+i}(i\ge 1)$ can be replaced with their forecasts, $f_{t+i|t}$ , setting up a recursive system:

$$\begin{array}{rl}& f_{t+2|t}(z_t,\beta )={\varphi }_1{f}_{t+1|t}(z_t,\beta )+{\varphi }_2{y}_t+\cdot \cdot \cdot +{\varphi }_p{y}_{t-p+2},\\ & f_{t+3|t}(z_t,\beta )={\varphi }_1{f}_{t+2|t}(z_t,\beta )+{\varphi }_2{f}_{t+1|t}(z_t,\beta )+{\varphi }_3{y}_t+\cdot \cdot \cdot +{\varphi }_p{y}_{t-p+3},\\ & \\ & ⋮(\\ & f_{t+p+1|t}(z_t,\beta )={\varphi }_1{f}_{t+p|t}(z_t,\beta )+{\varphi }_2{f}_{t+p-1|t}(z_t,\beta )+{\varphi }_3{f}_{t+p-2|t}+\cdot \cdot \cdot \\ & +{\varphi }_p{f}_{t+1|t},\end{array}\text{\hspace{1em}(7.19)}$$

and so forth, out to $f_{t+h}$ . This “chain rule” provides a computationally simple approach and is equivalent to recursively expressing unknown future values $y_{t+i}$ as functions of ${\dot{\mathbf{y}}}_t$ and its past. Known values of y affect the forecasts up to horizon $t+p$ while forecasts further ahead depend on forecasts recorded at shorter horizons.

Example 7.3.1 (Forecasts generated by the $\mathrm{AR}(1)$ model). For the $AR(1)$ model $y_{t+1}={\varphi }_1{y}_t+{\epsilon }_{t+1}$ , the chain rule forecasts are

$$\begin{array}{r}{f}_{t+1|t}(y_t,{\varphi }_1)={\varphi }_1{y}_t,\\ \\ f_{t+2|t}(y_t,{\varphi }_1)={\varphi }_1^2{y}_t,\end{array}$$

$$\begin{array}{rlr}& :& \\ & & \\ & & f_{t+h|t}(y_t,{\varphi }_1)={\varphi }_1^h{y}_t.\end{array}$$

Next, consider the $\mathrm{MA}(q)$ model,

$$y_{t+1}={\epsilon }_{t+1}+{\theta }_1{\epsilon }_t+\cdot \cdot \cdot +{\theta }_q{\epsilon }_{t-q+1}.\text{\hspace{1em}(7.20)}$$

Assuming again that ${\theta }_1,\dots ,{\theta }_q$ are known and $Z_t=({\epsilon }_t,{\epsilon }_{t-1},...,{\epsilon }_0)$ , the one-stepahead forecast becomes

$$\begin{array}{rl}& f_{t+1|t}(z,\beta )={\theta }_1{\epsilon }_t+\cdot \cdot \cdot +{\theta }_q{\epsilon }_{t-q+1},\\ & f_{t+2|t}(z,\beta )={\theta }_2{\epsilon }_t+\cdot \cdot \cdot +{\theta }_q{\epsilon }_{t-q+2},\\ & \\ & ⋮\\ & f_{t+q|t}(z,\beta )={\theta }_q{\epsilon }_t,\\ & f_{t+q|t}(z,\beta )=0.\end{array}$$

The $\mathrm{MA}(q)$ model has limited memory: values of an $\mathrm{MA}(q)$ process more than q periods into the future are not predictable. In contrast, whereas the degree of predictability eventually decays for a stationary $\mathrm{AR}(p)$ process, there is not the same sharp drop off in predictability for this process.

While the sequence of shocks $\left\{{\epsilon }_t\right\}$ are not directly observable, they can be computed recursively given the model. Or, depending on the estimation procedure, they can be estimated given a set of assumptions on the initial values for ${\epsilon }_t,t=0$ $\cdots ,q-1$

Example 7.3.2 (Forecasts generated by the MA(1) model). For the $MA(1)$ model, we can use ${\epsilon }_0=0$ and set up the recursion

$$\begin{array}{rl}& {\epsilon }_1=y_1,\\ & {\epsilon }_2=y_2-{\theta }_1{\epsilon }_1=y_2-{\theta }_1{y}_1,\\ & {\epsilon }_3=y_3-{\theta }_1{\epsilon }_2=y_3-{\theta }_1(y_2-\theta y_1),\end{array}$$

and so forth. Unobserved shocks can be written entirely as functions of the parameter value, ${\theta }_{1:}$ , and current and past values $ofy.$ . Simple recursions using past forecasts can also be employed to update the forecasts. This follows since $\begin{array}{r}{\epsilon }_t=y_t-P\left[y_t|Z_{t-1}\right]=\end{array}$ $y_t-f_{t|t-1}$ . For the $MA(1)$ model we have

$$f_{t+1|t}={\theta }_1{\epsilon }_t={\theta }_1(y_t-f_{t|t-1}).\text{\hspace{1em}(7.21)}$$

For a general $MA(q)$ process the updating formula again involves only past observations of y and past forecasts.

Next, consider MA processes of infinite order. Using that the MA coefficients are time invariant according to the Wold representation theorem, the infinite-order MA

model for $y_{t+h}$ for $h\ge 1$ is

$$\begin{array}{l}{\gamma }_{t+h}=\sum _{i=0}^{\infty }{\theta }_i{\epsilon }_{t+h-i}\\ =(\underset{\mathrm{future}\mathrm{shocks}}{\underbrace{{\epsilon }_{t+h}+{\theta }_1{\epsilon }_{t+h-1}+\cdot \cdot \cdot +{\theta }_{h-1}{\epsilon }_{t+1}}}+\underset{\mathrm{current}\mathrm{and}\mathrm{past}\mathrm{shocks}}{\underbrace{{\theta }_h{\epsilon }_t+{\theta }_{h+1}{\epsilon }_{t-1}+\cdot \cdot }})\end{array}$$

where the last line assumes $h\ge 1$ . Using the lack of serial correlation in $\epsilon ,$ this allows us to decompose $y_{t+h}$ into predictable and unpredictable parts. Hence if ${\epsilon }_t$ were observed, the forecast would be

$$\begin{array}{l}{f}_{t+h|t}(z_t,\beta )={\theta }_h{\epsilon }_t+{\theta }_{h+1}{\epsilon }_{t-1}+\cdot \cdot \cdot \\ =\sum _{j=h}^{\infty }{\theta }_j{\epsilon }_{t+h-j}.\end{array}\text{\hspace{1em}(7.22)}$$

Example 7.3.3 (Wiener–Kolmogorov prediction formulas). For a process with MA representation $y_t=\theta (L){\epsilon }_t$ , the Wiener–Kolmogorov prediction formula for linear projections $P(y_{t+h}|Z_t)$ takes the form

$$P(y_{t+h}|Z_t)={\left[\frac{\theta (L)}{L^h}\right]}_{+}\frac{y_t}{\theta (L)},\text{\hspace{1em}(7.23)}$$

where the $[.{]}_{+}$ operator excludes negative exponents of the lag operator, L . For example, for the invertible $AR(1)$ model in (7.8), $\theta (L)=(1-{\varphi }_{1}L{)}^{-1}$ , so (7.23) yields

$$\begin{array}{c}P(y_{t+h}|Z_t)=\frac{{\varphi }_1^h}{1-{\varphi }_{1}L}(1-{\varphi }_{1}L)y_t\\ ={\varphi }_1^h{y}_t.\end{array}$$

Similarly, for the $MA(q)$ process in (7.20),

$${\left[\frac{1+{\theta }_{1}L+{\theta }_2{L}^2+\cdot \cdot \cdot +{\theta }_q{L}^q}{L^h}\right]}_{+}=\{\begin{array}{ll}{\sum }_{j=h}^q{\theta }_j{L}^{q-h}& ifh\le q,\\ 0& ifh>q,\end{array}$$

and so the h-step-ahead linear prediction becomes

$$\begin{array}{r}P(y_{t+h}|Z_t)=\{\begin{array}{ll}{\sum }_{j=h}^q{\theta }_j{L}^{q-h}{\epsilon }_t& ifh\le q,\\ 0& ifh>q.\end{array}\end{array}$$

For mixed $\mathrm{ARMA}(p,q)$ models

$$y_{t+1}={\varphi }_1{y}_t+{\varphi }_2{y}_{t-1}+\cdot \cdot \cdot +{\varphi }_p{y}_{t-p+1}+{\epsilon }_{t+1}+{\theta }_1{\epsilon }_t+\cdot \cdot \cdot +{\theta }_q{\epsilon }_{t-q+1},\text{\hspace{1em}(7.24)}$$

the separate AR and MA prediction steps can be combined by recursively replacing future values of $y_{t+i}$ with their predicted values and setting $E\left[{\epsilon }_{t+j}|Z_t\right]=0\mathrm{for}j\ge 1$ This works as follows:

$$\begin{array}{rlr}& f_{t+1\mid t}={\varphi }_1{y}_t+{\varphi }_2{y}_{t-1}+\cdot \cdot \cdot +{\varphi }_p{y}_{t-p+1}+{\theta }_1{\epsilon }_t+\cdot \cdot \cdot +{\theta }_q{\epsilon }_{t-q+1},& \\ & & \\ f_{t+2\mid t}={\varphi }_1{f}_{t+1\mid t}+{\varphi }_2{y}_t+\cdot \cdot \cdot +{\varphi }_p{y}_{t-p+2}+{\theta }_2{\epsilon }_t+\cdot \cdot \cdot +{\theta }_q{\epsilon }_{t-q+2},& & \\ & & \\ ⋮& (7)& \\ f_{t+h\mid t}={\varphi }_1{f}_{t+h-1\mid t}+{\varphi }_2{f}_{t+h-2\mid t}+\cdot \cdot \cdot +{\varphi }_p{f}_{t-p+h\mid t}+{\theta }_h{\epsilon }_t+\cdot \cdot \cdot +{\theta }_q{\epsilon }_{t-q+h},& & \end{array}\text{\hspace{1em}(.25)}$$

where $f_{t-j+h|t}=y_{t-j+h}\mathrm{if}j\ge h$ , and ${\theta }_h=0$ for $h>q$

Having characterized the forecasts, we next turn to the forecast errors. By the Wold representation theorem, each stationary ARMA process can be written as an MA process, possibly of infinite order. From (7.22), the forecast error of such a process is

$$y_{t+h}-f_{t+h|t}(z_t,\beta )={\epsilon }_{t+h}+{\theta }_1{\epsilon }_{t+h-1}+\cdot \cdot \cdot +{\theta }_{h-1}{\epsilon }_{t+1},$$

and so the mean squared forecast error is

$$\begin{array}{rl}& E\left[(y_{t+h}-f_{t+h\mid t}(z_t,\beta ){)}^2\right]=E{\left[{\epsilon }_{t+h}+{\theta }_1{\epsilon }_{t+h-1}+\cdot \cdot \cdot +{\theta }_{h-1}{\epsilon }_{t+1}\right]}^2\\ & ={\sigma }^2(1+{\theta }_1+\cdot \cdot \cdot +{\theta }_{h-1}{)}^2.\end{array}\text{\hspace{1em}(7.26)}$$

This can be calculated for various $\mathrm{ARMA}(p,q)$ models. The chief difficulty stems from the MA coefficients being nonlinear functions of the original $\mathrm{ARMA}(p,q)$ coefficients in (7.26).

Example 7.3.4 (MSE for the AR(1) process). For the $AR(1)$ model, ${\theta }_i={\varphi }_1^i$ and so

$$\begin{array}{l}E[(y_{t+h}-f_{t+h\mid t}(z_t,\beta ){)}^2]={\sigma }^2(1+{\theta }_1+\cdot \cdot \cdot +{\theta }_{h-1}{)}^2\\ \\ ={\sigma }^2\sum _{j=0}^{h-1}{\varphi }_1^j\\ =\frac{{\sigma }^2(1-{\varphi }_1^{2h})}{1-{\varphi }_1^2}.\end{array}\text{\hspace{1em}(7.27)}$$

This result could as easily have been obtained using the backward recursion in (7.9):

$$f_{t+h|t}={\varphi }_1^h{y}_t+\sum _{s=0}^{h-1}{\varphi }_1^s{\epsilon }_{t+h-s}.$$

The forecast is $f_{t+h|t}(z_t,\beta )={\varphi }_1^h{y}_t$ and the MSE is again $\begin{array}{r}{\sigma }^2{\sum }_{j=0}^{h-1}{\varphi }_1^j={\sigma }^2(1-\end{array}$ ${\varphi }_1^{2h})/(1-{\varphi }_1^2)$ . $Ash\to \infty$ , this converges to ${\sigma }^2/(1-{\varphi }_1^2)$ which is the unconditional variance of the process. This makes sense since, at very long horizons, predictions of a stationary AR process are simply the unconditional mean of the process.