11.2 ESTIMATION OF SIEVE MODELS

11.2 ESTIMATION OF SIEVE MODELS

Sieve estimation, due to Grenander (1981), seeks to approximate an unknown function by combinations of functions of the data. Applied to nonparametric forecasting, the idea is to construct a forecasting model of the form

$$f(z_T)={\beta }_0^{\prime }{z}_T+\sum _{j=1}^{J_T}{\beta }_{j}g(z_T,{\gamma }_j),\text{\hspace{1em}(11.4)}$$

where $J_T$ is the number of scalar nonlinear terms, each denoted $g(z_T,{\gamma }_j)$ , and ${\beta }_j$ are weights on those terms. Variations in choice of the $g$ functions capture different approaches to the nonlinear approximation. Note that we could set ${\beta }_0=0.$ , although often it is better in practice to let this coefficient be estimated. The inclusion of a linear lead term ${\beta }_0^{\prime }{z}_T$ reflects that linear models often perform well empirically and many of the choices for the $g$ functions do not approximate linear models well unless $J_T$ is very large. The image of a sieve is useful here—think of a larger number of terms as a more finely meshed sieve; the number of functions the sieve will “catch” is larger for large values of $J_T$ because it is able to better approximate a wider set of possible functions. A sieve with fewer terms will catch fewer functional forms.

Once the functions in (11.4) are determined, estimation involves solving an extremum problem based on the loss function. Specifically, the free parameters $\{{\beta }_0,{\beta }_j,{\gamma }_j\}$ , and perhaps the fineness of the mesh, ${\mathbf{J}}_T,$ , are chosen by minimizing the average sample loss,

$${\bar{L}}_T=(T-1{)}^{-1}\sum _{t=0}^{T-1}L(f(z_t),y_{t+1}).\text{\hspace{1em}(11.5)}$$

Most often the loss criterion is MSE, and so sieve estimators will approximate the conditional mean of $y_{t+h}$ given $z_t$ . Nonlinear regression can be used in this case to construct estimates of the forecasting model.

Theoretical results suggest that for the right choice of $g$ functions, $E[(f^{\ast }(z_t)-$ $f(z_t){)}^2]$ can become arbitrarily small as $J_T$ gets large for any member of a given set of families of models. This suggests that as $J_T\to \infty$ , the forecasting model becomes a better and better approximation to any model in some set of nonlinear models, $\mathcal{M}.$ . This feature of being a good approximator for a very large set of models is often used to justify the use of nonparametric methods. However, any empirical application must use a finite number of terms in the approximation, thus introducing both approximation and estimation errors.

The number of terms in (11.4), $J_T$ , is subscripted by the sample size, $T_{:}$ because the approximating properties are established asymptotically by letting the number of terms increase with the sample size. From a practical perspective, $J_T$ needs to be small relative to the sample size to ensure that estimation of the parameters $({\beta }_j,{\gamma }_j)$ is not so imprecise that it overwhelms any gains from obtaining a better approximation to the optimal forecast model. If $J_T$ is chosen to be too small, the model can be a poor approximation to the true conditional mean even when the model has attractive properties for larger values of $J_T$

Complications might arise when estimating ${\gamma }_j$ which, depending on the selected functional forms, can enter the specification nonlinearly and hence requires using search methods. Notice that for a given value of ${\gamma }_j$ , the model is linear in ${\beta }_j$ and hence these parameters can be estimated by OLS or by any of the shrinkage methods examined in chapter 6.

Sieve methods, of which there are many, differ in their specification of the functional form for $g$ . These functions have the property that as $J_T$ gets large the models are dense in some set M. Not all choices of functions have useful properties in this regard. The desire for good approximating properties leads directly to the use of basis functions as specifications for $g$ . Basis functions can be mutually orthogonal or correlated. We next provide a partial list of functions that can be employed.

11.2.1 Polynomials

When the form of the forecasting model, $g_{:}$ , is unknown, a seemingly reasonable approach is to take a Taylor-series approximation to the function. For a univariate predictor, $z_t$ , this suggests using the forecasting model,

$$y_{t+1}=\sum _{i=0}^m{\beta }_i{z}_t^i+u_{t+1}.\text{\hspace{1em}(11.6)}$$

Although this estimator can have useful approximation properties— particularly if the true function is very smooth—in most cases it is really a local approximation. When the parameters ${\beta }_i$ are estimated over a range of data, the estimated weights are not the local parameters at any particular point but instead an average of them. In practice, the method can therefore prove to be a poor predictor. To remedy that the parameter estimates are a data-weighted average obtained over many points, estimation can focus on a single point by extending the local linear regression in (11.3) to include polynomial terms. This approach is the most common form of application in empirical work.