21.4.3 State-Space Approaches with Irregular Data

21.4.3 State-Space Approaches with Irregular Data

Kalman filter methods are ideally suited to handle irregularly observed data and have been used by Aruoba, Diebold, and Scotti (2009) to estimate a dynamic factor model that tracks the state of the economy at a higher frequency than that at which many variables are observed. Their approach allows for differences in observation frequencies of economic data as well as irregular patterns in the publication of observable data.

Suppose the scalar time-series process v captures the underlying state of the economy. Letting ${\xi }_t$ measure the state of the economy on day t, this is assumed to follow an $\mathrm{AR}(p)$ process,

$${\xi }_t=\sum _{i=1}^P{\rho }_i{\xi }_{t-i}+e_t,e_t\sim \mathrm{WN}(0,1).\text{\hspace{1em}(21.20)}$$

Here ${\xi }_t$ is assumed to be unobserved and is treated as a latent variable. It is linked to a vector of observable variables, $y_t=(y_{1t},\dots ,y_{nt}{)}^{\prime }$ , each of which is a linear function of ${\xi }_t$ , and a $(k\times 1)$ vector of exogenous variables $w_t=(w_{1t},\dots ,w_{kt}{)}^{\prime }$ through the equations

$$y_{it}=c_i+{\beta }_i{\xi }_t+\sum _{j=1}^k{\delta }_{ij}{w}_{jt}+\sum _{j=1}^{n_J}{\gamma }_{ij}{y}_{it-j\times D_{it}}+u_{it}.\text{\hspace{1em}(21.21)}$$

Here $D_{it}$ is the number of days per observation period for variable i at time t. For example, $D_{it}=7$ for a variable measured weekly. The value of $D_{it}$ will almost certainly vary across different variables and over time, but if we assume that $D_{it}=$ $D_{jt}=D$ , we can write (21.21) as a vector process for $y_t=(y_{1t},\dots ,y_{nt}{)}^{\prime }$

$$y_t=c+\beta {\xi }_t+\delta w_t+\sum _{j=1}^{n_J}{\gamma }_j{y}_{t-j\times D}+u_t,\text{\hspace{1em}(21.22)}$$

where $c,\beta ,$ , and $u_t$ are $(n\times 1)$ vectors, δ is $(n\times k)$ , and ${\gamma }_j$ is $(n\times n)$ . This equation is the basis for the measurement equation of a state-space model comprising (21.20) and (21.22). Aruoba, Diebold, and Scotti (2009) provide details of how the Kalman filter and smoother can be used to extract estimates of the latent state.

Two complications arise when implementing the state-space model (21.20)– (21.22). First, suppose the model for ${\gamma }_{it}$ is tailored to the daily frequency. Many variables are not observed this often. To account for this, let ${\tilde{y}}_{it}$ denote the i th variable observed at the lower frequency. For stock variables ${\tilde{y}}_{it}=y_{it}\mathrm{if}{y}_{it}$ is observed on day $t;$ otherwise ${\tilde{y}}_{it}=\mathrm{NA}$ (“not available” or missing). For flow variables, ${\tilde{y}}_{it}={\sum }_{i=0}^{D_{it}-1}{y}_{it-j}\mathrm{if}{y}_{it}$ is observed on day t, otherwise ${\tilde{y}}_{it}=\mathrm{NA}$

A second complication arises exactly because $D_{it}$ will differ across variables measured at different frequencies and across time due to changes in the number of days per month or the effect of holidays. This leads to heteroskedasticity in the error terms in (21.21) and to time variation in some of the matrices of the model. Letting $z_t^{\prime }=(w_t^{\prime }{y}_{t-1}^{\prime }\dots y_{t-JD}^{\prime }{)}^{\prime }$ , while ${\alpha }_t$ captures the latent state variables, (21.20) and (21.22) lead to a state-space model of the form

$$\begin{array}{r}{y}_t={\Psi }_t{\alpha }_t+{\Gamma }_t{z}_t+u_t,\\ {\alpha }_{t+1}=J{\alpha }_t+R{\eta }_t,\end{array}$$

for $t=1,\dots ,T$ , where $u_t\sim (0,H_t)$ , and ${\eta }_t\sim (0,Q)$ . Time variation in the matrices ${\Psi }_t,{\Gamma }_t$ , and $H_t$ reflects changes in the number of days across different months and quarters. If the model is tailored to daily data, ${\gamma }_t$ will mostly have missing observations since many variables are not observed this often. This means that the associated matrices of the state-space system (21.20)–(21.22) become sparse, a feature that can be exploited to simplify the estimation and updating equations.

Specifically, because most variables do not get observed on most days, the measurement equation on day t can be based on only the subset of those variables that are observed on this day, denoted $y_t^{\ast }=S_t{y}_t$ , where the matrix $S_t$ selects those elements of ${\gamma }_t$ that are observed on day t. Updates to the Kalman filter can then be based on a reduced subsystem for $y_t^{\ast }=S_t{y}_t$ ,

$$y_t^{\ast }={\Psi }_t^{\ast }{\alpha }_t+{\Gamma }_t^{\ast }{z}_t+u_t^{\ast },u_t^{\ast }\sim \mathrm{N}(0,H_t^{\ast }),$$

where ${\Psi }_t^{\ast }=S_t{\Psi }_t,{\Gamma }_t^{\ast }=S_t{\Gamma }_t,u_t^{\ast }=S_t{u}_t$ , and $H_t^{\ast }=S_t{H}_t{S}_t^{\prime }$ . When the number of series being modeled, n, is large, this can considerable simplify updates to the model.

These methods have proved influential as the Federal Reserve Bank of Philadelphia publishes a daily time series summarizing the state of the US economy.

21.5 CONCLUSION

Recent improvements in computer power and access to large real-time data have translated into exciting developments allowing forecasters to produce daily estimates and forecasts of the state of the business cycle in a manner that would not have been feasible just a few years ago. These advances build on methods covered throughout this book such as dynamic factor models and Kalman filtering and only begin to scratch the surface of what are likely future usages of forecasting techniques in a wide area of applications.

his Appendix covers the Kalman filter along with some probability concepts that are used throughout the book.

A.1 KALMAN FILTER

The linear Kalman filter is essentially an algorithm for linear prediction. Introduced in 1960 by Kalman for engineering applications (Kalman, 1960), the method has found widespread use in many disciplines, including economics and finance. For models with normally distributed variables, the filter can be used to write down the likelihood function or, more generally, a pseudo likelihood. Many popular models can be rewritten in a form so that they fit in this framework. Our exposition draws on Hamilton (1994, chapter 13).