7.2 ESTIMATION AND LAG SELECTION FOR ARMA MODELS

7.2 ESTIMATION AND LAG SELECTION FOR ARMA MODELS

The forecasting methods discussed in the previous section assume that the ARMA model and its parameters are known. In practice these need to be estimated from the data. To construct forecasts we require (i) the order of the $\mathrm{ARMA}(p,q)$ model; (ii) estimates of the parameters ${\varphi }_1,\dots ,{\varphi }_p,{\theta }_1,\dots ,{\theta }_q;$ and (iii) estimates of past shocks, ${\epsilon }_t,\dots ,{\epsilon }_{t-q}\mathrm{if}q\ge 1$ . This section discusses how to obtain parameter estimates, select the order of the lag polynomials, $p,q$ , and obtain filtered estimates of current and lagged innovations for $\mathrm{ARMA}(p,q)$ models.

7.2.1 State-Space Representations of ARMA Models

ARMA models can be estimated using state-space techniques. Writing forecasting models in their state-space representation has a number of advantages. First, if the model has a moving average component, often the state-space form and the Kalman filter provide an easy estimation strategy, i.e., one that is employed in many standard computer packages. Second, the state-space representation provides a convenient representation of the model for constructing multistep forecasts.

The state-space representation of a model consists of two equations, known as the state equation and the measurement equation. The state equation is written in the form of a (typically multivariate) first-order autoregression in “states” ξt,

$${\xi }_t=F{\xi }_{t-1}+v_t.\text{\hspace{1em}(7.14)}$$

Similarly, a typical form for the measurement equation is

$$y_t=H{\xi }_t+w_t,\text{\hspace{1em}(7.15)}$$

where the error terms $(v_t,{\omega }_t{)}^{\prime }$ are white noise and mutually uncorrelated:

$$E\left[\left(\begin{array}{c}{v}_t\\ w_t\end{array}\right)\left(v_t^{\prime }{w}_t^{\prime }\right)\right]=\left(\begin{array}{cc}Q& 0\\ 0& R\end{array}\right).$$

Here ${\gamma }_t$ denotes observed values while the states ${\xi }_t$ may or may not be observed. The parameters of the model F , H, Q, and R can be allowed to vary with time. Often either Q or R is 0, which presents no difficulties.

As an example of how ARMA models can be written in state-space form, consider the ARMA(1,1) model,

$$y_t=\rho y_{t-1}+{\epsilon }_t+\theta {\epsilon }_{t-1}.$$

This model can be written in state-space form as

$$\begin{array}{r}\left(\genfrac{}{}{0ex}{}{{\xi }_{1t}}{{\xi }_{2t}}\right)=\left(\begin{array}{ll}\rho & 1\\ 0& 0\end{array}\right)\left(\genfrac{}{}{0ex}{}{{\xi }_{1t-1}}{{\xi }_{2t-1}}\right)+\left(\begin{array}{l}1\\ \theta \end{array}\right){\epsilon }_t,\\ y_t={\xi }_{1t},\end{array}$$

which yields the ARMA(1,1) model since ${\xi }_{2t}=\theta {\epsilon }_t$ and $y_t={\xi }_{1t}=\rho {\xi }_{1t-1}+{\epsilon }_t+$ ${\xi }_{2t-1}=\rho {\xi }_{1t-1}+{\epsilon }_t+\theta {\epsilon }_{t-1}$ . The appendix at the end of the book contains additional examples.

When moving average terms are involved $(q>0)$ , plug-in forecasts require estimates of the MA parameters and of the underlying shocks $\left\{{\epsilon }_t\right\}$ . The most common approach is to use maximum likelihood estimation under the assumption of normal errors. Hamilton (1994, chapters 3 and 5) provides an excellent overview of the precise formulas for the likelihood and methods for maximizing it. Typically the Kalman filter is employed to construct the likelihood. Kalman filter recursions also provide estimates of the shocks and the forecast directly from the updating and prediction equations for the state variables, respectively. An appendix shows how the state-space form can be used for constructing maximum likelihood estimates and estimating parameters.

Taking advantage of the first-order recursive structure, state-space representations are useful for forecasting multiple periods ahead. Since ${\xi }_t=F{\xi }_{t-1}+v_t.$ it follows that $E\left[{\xi }_{t+h}|{\xi }_t\right]=F^h{\xi }_t$ . Moreover, from (7.15), we have $E\left[y_{t+h}|{\xi }_t\right]=$ $H[{\xi }_{t+h}|{\xi }_t]=H{F}^h{\xi }_t$ . This is often a much simpler approach to computing the forecast than working with the original model.