11.1 KERNEL ESTIMATION OF FORECASTING MODELS

11.1 KERNEL ESTIMATION OF FORECASTING MODELS

Local linear regression methods can be used to produce forecasts under MSE loss. These methods are essentially weighted least squares estimators that assign large weights to previous data for which the predictor variables are “close” to the values observed at the point where the current forecast is made. Conversely, less weight gets assigned to observations with very different values for the predictor variables. The idea is that if the true model really is nonlinear, then observations that are not close to the current predictor variables are less useful in estimating the conditional mean at the current levels of the predictor variables. Let $z_t$ be the predictors for $y_{t+1}$ , excluding a constant. We can directly estimate the conditional mean, and hence produce a forecast at time T , from

$$f_{T+1|T}(z_T)=\frac{{\sum }_{t=1}^{T-1}{y}_{t+1}K(z_t-z_T)}{{\sum }_{t=1}^{T-1}K(z_t-z_T)},\text{\hspace{1em}(11.1)}$$

where $K(z_t-z_T)$ is a multivariate kernel. For example, the Gaussian kernel sets

$$K(z_t-z_T)=|B{|}^{-1/2}\exp \{-0.5(z_t-z_T{)}^{\prime }{B}^{-1}(z_t-z_T)\},\text{\hspace{1em}(11.2)}$$

where B is a bandwidth matrix chosen by the forecaster. In practice, often a product kernel is chosen, for which we have $K(z)=K_1(z_1)\times \cdot \cdot \cdot \times K_k(z_k)$ , where $K_i(\cdot )$ is a

univariate Gaussian kernel. Choosing B to be diagonal in (11.2) results in a product kernel with $K_i(\cdot )$ being the univariate Gaussian kernel.

The method can be extended to locally weighted linear regression to obtain estimates of $\beta =({\beta }_0^{\prime },{\beta }_1^{\prime })$ by solving

$$({\hat{\beta }}_0,{\hat{\beta }}_1)=\arg \underset{{\beta }_0,{\beta }_1}{\min }\sum _{t=0}^{T-1}(y_{t+1}-{\beta }_0-{\beta }_1^{\prime }(z_t-z_T){)}^{2}K(z_t-z_T),\text{\hspace{1em}(11.3)}$$

so that the forecast at time T is the estimated value ${\hat{\mathbf{\beta }}}_0$ . This follows because $(z_t-$ $z_T)=0$ at $z_T$ . For calculation purposes let $\begin{array}{r}y=(y_1,\dots ,y_T{)}^{\prime },\tilde{z}=([1(z_0-z_T)]\end{array}$ $[1(z_1-z_T)],...,[1(z_{T-1}-z_T)])$ and let K be a diagonal matrix whose $(i,i)$ element is $K(z_i-z_T)$ . Then the estimator for the locally weighted regression coefficients is $\hat{\beta }=({\tilde{z}}^{\prime }K\tilde{z}{)}^{-1}{\tilde{z}}^{\prime }Ky$ and the forecast is the first element of $\hat{\mathbf{\beta }},\mathrm{i}.\mathrm{e}.,{\hat{\mathbf{\beta }}}_0{.}^2$

Kernel regression methods were first proposed by Nadaraya (1965) and Watson (1964). Stone (1977) introduced local linear regression. Pagan and Ullah (1999) provide an excellent introduction and overview. The methods are appropriate under a wide range of assumptions on the data, allowing for heterogeneity and dependence in the data. See Hansen (2008b) for a general treatment.

For local linear regression methods to work well, we clearly require that there is a reasonable amount of data with predictors near the values we wish to condition on, i.e., close to $z_T$ . As always with kernel-based methods, both the kernel and bandwidth must be chosen. For both the conditional mean estimator (11.1) and the local linear regression estimator (11.3), a larger bandwidth choice means averaging over more data and hence getting closer to either the unconditional mean for (11.1) or the OLS regression estimator for (11.3). Smaller bandwidths average over data with predictor variables very close $\mathrm{to}{z}_T$ , the current predictor variables. A trade-off therefore arises between a larger bias when averaging over more of the data (if the model is truly nonlinear) versus a larger variance when averaging over less of the data. The optimal bandwidth lies somewhere in the middle, depending on what is optimized. Minimizing MSE results in an optimal bandwidth that is proportional to ${\bar{T}}^{1/5};$ see Pagan and Ullah (1999) for a discussion. A natural approach in the context of forecasting is to choose the bandwidth to minimize pseudo out-of-sample MSE, typically calculated using some proportion of the latter sample observations. This method is the forecast equivalent of cross validation, although its properties have not, as far as we know, been systematically evaluated.