8.5 FORECASTING WITH NONLINEAR UNIVARIATE MODELS

8.5 FORECASTING WITH NONLINEAR UNIVARIATE MODELS

As is clear from the discussion of the nonlinear models above, forecasting with univariate nonlinear models generally relies on numerical methods—at least in so far that multi-step-ahead forecasts are of interest. To see this, consider the general nonlinear model with additive innovations:

$$y_{t+1}=f(y_t)+{\epsilon }_{t+1},{\epsilon }_{t+1}\sim (0,{\sigma }_{\epsilon }^2).$$

Multi-step-ahead values for y depend on ${\epsilon }_{t+1},{\epsilon }_{t+2},\dots ,{\epsilon }_{t+h};$

$$\begin{array}{rl}& y_{t+2}=f(f(y_t)+{\epsilon }_{t+1})+{\epsilon }_{t+2},\\ & y_{t+3}=f(f(f(y_t)+{\epsilon }_{t+1})+{\epsilon }_{t+2})+{\epsilon }_{t+3},\\ & y_{t+h}=f(f(y_{t+h-1})+{\epsilon }_{t+h-1})+{\epsilon }_{t+h}.\end{array}\text{\hspace{1em}(8.26)}$$

If $f$ is a nonlinear function, then in general we have $E\left[y_{t+h}|Z_t\right]\ne E\left[f^h(y_t)\right]$ even if $E\left[{\epsilon }_{t+j}|Z_t\right]=0$ . This means that the full density of ε matters even under MSE loss where the focus is on computing the expectation of $y_{t+h}$ . Brown and Mariano (1989) discuss the behavior of predictors in nonlinear dynamic models.

There are three strategies for dealing with this issue. First, if the density for ε is known or can be estimated, we can use Monte Carlo simulation to draw future values $\{{\epsilon }_{t+1},\dots ,{\epsilon }_{t+h}\}$ and compute $y_{t+h}$ iteratively. Letting $b=1,\dots ,B$ , we can compute

$$\begin{array}{c}{y}_{t+1}^b=f(y_t)+{\epsilon }_{t+1}^b,\\ \\ ⋮\\ y_{t+h}^b=f(f(y_{t+h-1}^b)+{\epsilon }_{t+h-1}^b)+{\epsilon }_{t+h}^b.\end{array}\text{\hspace{1em}(8.27)}$$

Finally, the forecast can be based on the simulated density for $y_{t+h}$ . For example, under MSE loss, the forecast would be the conditional mean

$$f_{t+h|t}=\frac{1}{B}\sum _{b=1}^B{\gamma }_{t+h}^b,$$

whereas under lin-lin loss, the forecast would be a quantile of the simulated distribution for $y_{t+h}$

The second approach is to use a nonparametric bootstrap rather than assuming some parametric distribution for ε. The bootstrap repeatedly draws $({\hat{\epsilon }}_{t+1},{\hat{\epsilon }}_{t+2},\dots ,{\hat{\epsilon }}_{t+h})$ with replacement from the empirical c.d.f. of the fitted model for $b=1,\dots ,B:$

$$\begin{array}{rl}& y_{t+1}^b=f(y_t)+{\hat{\epsilon }}_{t+1}^b,\\ & \\ & ⋮\\ & \\ & y_{t+h}^b=f(f(y_{t+h-1}^b)+{\hat{\epsilon }}_{t+h-1}^b)+{\hat{\epsilon }}_{t+h}^b.\end{array}\text{\hspace{1em}(8.28)}$$

As for the Monte Carlo simulation approach, these draws can be used to compute expected loss. Several different bootstrap methods can be used here, including the block bootstrap, the stationary bootstrap, or even semiparametric bootstraps that allow for dynamics in the second moment of the residuals.

The third and final approach is to use the direct forecast approach and model $y_{t+h}$ directly as a function of ${\gamma }_t$

$$y_{t+h}=f_h(y_t)+{\epsilon }_{t+h}.\text{\hspace{1em}(8.29)}$$

Under this approach, we use the nonlinear model to relate $y_{t+h}$ directly to ${\gamma }_t$ and so there is no need to iterate on the forecasts to obtain multi-step-ahead predictions. Conversely, each forecast horizon requires its own separate forecasting model, $f_h,$ and there is no attempt to make sure that multi-step-ahead forecasts are consistent across different horizons. This is important since the functional form of nonlinear models typically is not preserved at multi-step-ahead horizons as can easily be verified from the iteration in (8.26).

Once again, there will be a trade-off between the efficiency gains that can be obtained by using a model-consistent approach that iterates on a model fitted to the highest frequency at which data are available to obtain a multi-step-ahead forecast versus using the direct approach which is likely to be less sensitive to misspecification.

The Markov switching model can be used to illustrate the potential gains from using nonlinear as opposed to linear forecasts as we next show using an example from Hamilton (1989).

Example 8.5.1 (Forecasting with linear versus nonlinear models). Following Hamilton (1989), consider a simple version of the Markov switching model:

$$y_t=s_t+{\epsilon }_t,\text{\hspace{1em}(8.30)}$$

where ${\epsilon }_t\sim iid(0,{\sigma }^2)$ , and $s_t$ is a two-state binary variable that follows the first-order Markov process2

$$\begin{array}{rl}& \mathrm{Pr}(s_t=0|s_{t-1}=0)=p_0,\\ & \\ & \mathrm{Pr}(s_t=1|s_{t-1}=1)=p_1.\end{array}\text{\hspace{1em}(8.31)}$$

Even though this is a nonlinear process, it can be represented through an $AR(1)$ model with an unusual error distribution:

$$\begin{array}{l}{s}_t=(1-p_0)+\lambda s_{t-1}-v_t,\\ \\ \lambda =p_0+p_1-1,\end{array}\text{\hspace{1em}(8.32)}$$

where the innovation term $v_t$ is conditionally binomially distributed. Conditional on $s_{t-1}=0,v_t$ follows the distribution

$$v_t=\{\begin{array}{ll}1-p_0& withprobability{p}_0,\\ -p_0& withprobability1-p_0,\end{array}$$

while, conditional on $s_{t-1}=1$

$$v_t=\{\begin{array}{ll}-(1-p_1)& withprobability{p}_1,\\ p_1& withprobability1-p_1.\end{array}$$

These properties mean that, conditional on $s_{t-1},v_t$ is a martingale difference sequence and so has zero conditional mean in both states:

$$\begin{array}{rl}& E\left[v_t|s_{t-1}=0\right]=p_0(1-p_0)-p_0(1-p_0)=0,\\ & E\left[v_t|s_{t-1}=1\right]=-p_1(1-p_1)+p_1(1-p_1)=0.\end{array}\text{\hspace{1em}(8.33)}$$

However, $v_t$ is clearly not independent of ${\mathbf{f}}_{{\mathbf{S}}_{t-1}}$ . For example, its conditional variance is

$$E\left[v_t^2|s_{t-1}=0\right]=p_0(1-p_0{)}^2+p_0^2(1-p_0)=p_0(1-p_0),$$

$$E\left[v_t^2|s_{t-1}=1\right]=p_1(1-p_1{)}^2+p_1^2(1-p_1)=p_1(1-p_1).$$

Using (8.30) and (8.32), notice that

$$\begin{array}{rl}& y_t-\lambda y_{t-1}=s_t-\lambda s_{t-1}+{\epsilon }_t-\lambda {\epsilon }_{t-1}\\ & \\ & =(1-p_0)-v_t+{\epsilon }_t-\lambda {\epsilon }_{t-1}.\end{array}$$

As shown by Hamilton, the error term on the right-hand side of this equation follows an MA(1) process:

$$-v_t+{\epsilon }_t-\lambda {\epsilon }_{t-1}=u_t-\theta u_{t-1},$$

where $u_t\sim \mathrm{WN}(0,{\sigma }_u^2)$ and θ and ${\sigma }_u^2$ are parameters determined by the equations

$$(1+{\theta }^2){\sigma }_u^2=(1+{\lambda }^2){\sigma }_{\epsilon }^2+{\sigma }_v^2,$$

$$\theta {\sigma }_u^2=\lambda {\sigma }_{\epsilon }^2,$$

and

$${\sigma }_v^2=(1-\pi )p_0(1-p_0)+\pi p_1(1-p_1)$$

is the unconditional variance of $v_t$ . Here $\pi =(1-p_0)/(2-p_0-p_1)$ is the steadystate (unconditional) probability of state 1. This suggests using the ARMA(1,1) linear model,

$$y_{t+1}=\lambda y_t+(1-p_0)+u_{t+1}-\theta u_t,$$

to predict $y_{t+1}$ or, more generally, $y_{t+h}$ . However, such a forecasting scheme is not optimal: although $u_t$ is uncorrelated with $u_{t-1},u_{t-2},\dots ,it$ is not independently distributed of earlier values. The optimal forecast is in fact

$$E\left[y_{t+h}|y_t,y_{t-1},...\right]=\pi +{\lambda }^h(\mathrm{Pr}(S_t=1|y_t,y_{t-1},...)-\pi ).\text{\hspace{1em}(8.34)}$$

The state probability $\mathrm{Pr}(S_t=1|y_t,y_{t-1},...)$ is a highly nonlinear function of ${\gamma }_t$ which makes the optimal forecast a nonlinear function of current and past values of $y.$ See Hamilton (1989) for further details.