21.3 REAL-TIME DATA

21.3 REAL-TIME DATA

Real-time forecasting refers to the computation of forecasts by means of data restricted to be available at the point in time when the forecast is constructed. This may seem like an obvious requirement to impose and is automatically satisfied in the case of survey forecasts. Often, however, forecasts are constructed “after the fact,” using simulated or pseudo out-of-sample methods—or backtests as they are called in finance—and it is for these exercises that the data availability restriction becomes relevant. Suppose, for example, that we find that we could have predicted some event, say the sharp decline in stock markets during the fall of 2008, by means of data that were not available at the time such as detailed information on banks’ balance sheets or counterparty risk exposures, and the delinquency rates on mortgages. Clearly, we would not conclude from this finding that, historically, markets were not efficient. At most we could conclude that, had such data been available and used to construct a forecasting model, it might have helped investors and other decision makers predict the ensuing events.

Close attention therefore has to be paid to which data are available to forecasters in real time. Many types of macroeconomic data undergo substantial revisions. This leads to the concept of data vintages, i.e., snapshots of the data series available at a given point in time, v. For example, at the monthly frequency the 2012:12 data vintage might comprise all data available at the end of December 2012. This includes data on earlier observations known at that point in time. Data vintages can be represented as an expanding triangular shape, as shown in table 21.1.

Data revisions come in different shapes. First, there are the regularly scheduled revisions for variables such as GDP. For example, in the US, an advance estimate of GDP in a given quarter is published by the Bureau of Economic Analysis near the end of the month following the end of the quarter. The following year, data from income tax records and economic census data allow this estimate to be updated to include more precise data. Second, benchmark revisions may reflect changes in how a variable is being measured. A prime example here is the change from measuring GDP using fixed weighting as opposed to chain weighting, which was introduced in 1996 for US GDP.

Let t be the date of the forecast and let $z_{tv}$ refer to date-t observations from vintage v. Only the current data vintage—data available at time t—can be used to generate forecasts. Moreover, let ${\beta }_{tv}$ be the parameter of the forecasting model associated with vintage v. Taking explicit account of data vintages, we can write the forecast for period t + h given information at time t on the latest vintage v as

$$f_{t+h|tv}=f_v(z_{tv},{\beta }_{tv}).$$

Here v refers to the vintage of data used for the model, $z_{tv}$ is the data vintage available at time t, and ${\beta }_{tv}$ are the parameters associated with this model.

As pointed out by Croushore (2006) and Stark and Croushore (2002), data revisions can affect the forecasting model, and the conditional forecast, $f_v(z_{tv},{\hat{\mathbf{\beta }}}_{tv})$ , in three ways. First, data revisions directly affect the forecast through the conditioning information, $z_{tv}$ . To illustrate this point, suppose we compare two forecasts from identical linear models, except that one forecast uses vintage $v_1$ , while the other uses vintage $v_2.$ . Then the two forecasts would be ${\hat{f}}_{t+h|t{v}_1}={\hat{\beta }}^{\prime }{z}_{t{v}_1}$ and ${\hat{f}}_{t+h|t{v}_2}={\hat{\beta }}^{\prime }{z}_{t{v}_2}$ , respectively. Second, the use of different data will almost certainly lead to different parameter estimates, ${\hat{\mathbf{\beta }}}_{tv},$ i.e., ${\hat{\mathbf{\beta }}}_{v_1}^{\prime }\ne {\hat{\mathbf{\beta }}}_{v_2}^{\prime }$ , and so ${\hat{\beta }}_{v_1}^{\prime }{z}_{t{v}_1}\ne {\hat{\beta }}_{v_2}^{\prime }{z}_{t{v}_2}$ . Finally, the forecast model $f_v(\cdot )$ could itself change with the data vintage—a point that applies both to the choice of lag order for linear models and the inclusion of nonlinear terms for nonlinear models.

These considerations appear to matter empirically. In a study of predictability of money demand, Amato and Swanson (2001) find that the latest available data on the aggregate stock of money, as measured by M1 and M2, appear to predict output growth, while real-time measures of the money stock do not have such predictive power. This suggests that measures of growth in money should perhaps be omitted from real-time forecasting models. Swanson and White (1997) examine the effect of real-time data revisions on model selection.

ABLE 21.1 Data vintages for quarterly US real GNP.

Dependent on what is being modeled, data revisions can also lead to changes in the predicted variable. If the predicted variable is subject to revisions, what is considered the “actual” or outcome is no longer so clear-cut. Are forecasters trying to predict the first measurement of a variable because this is what will generate most publicity, or are they trying to predict some underlying, unobserved, “true” measure of the outcome which, over time, is measured with increasing precision? This important issue is often difficult to address.

Stark and Croushore (2002) consider using either the latest available data vintage, the most recent vintage prior to a benchmark revision, or the vintage obtained one year after the observation date. If information is improving over time, using the latest available vintage would seem to make most sense. In this case, data on older observation dates have had more time to “settle” and so could introduce heterogeneity in the quality of the data, with more recent data being subject to the greatest measurement errors. For GDP growth, Stark and Croushore (2002) find that forecasts are not notably improved if based on the latest available data. Conversely, for inflation forecasts, models based on the latest available data seem to lead to better forecasts than models based on the real-time data vintages.