Nonparametric Forecasting Methods

The methods examined in the preceding chapters have all been either fully parametric—in which case the forecasting model is specified up to a finitedimensional vector of unknown parameters, β—or close to fully parametric, involving a search over a fixed set of parametric models (as in chapter 6). In chapter 4 we noted the possibility that any given parametric forecasting model $f(z,\beta )$ might not include the true model. Indeed, to know the true model one must know all or a substantial part of the joint density of Z—a requirement that typically does not hold in practical forecasting situations.

The loss function for a given problem at hand indicates which features of the joint distribution of Z we are interested in. For MSE loss, this is the conditional mean of Y given Z. For lin-lin loss, it is a conditional quantile. This chapter examines methods that have been used to approximate such objects nonparametrically or semi-non-parametrically with minimal information on the underlying model. These methods use the data to estimate the forecasting model in a way that can approximate as many of the potentially true models as possible. There are still restrictions on the set of functions that can be handled, but nonparametric approaches aim to find methods that work for a wide class of functions. By widening the set of models that can be estimated, there is a greater chance that the estimated forecasting model is close to the optimal model. Naturally there is a price to pay for the wider search adopted by nonparametric forecasting models. First, a practical consideration is the dimension of the variables available for constructing forecasts. Since many methods become difficult to implement with a large set of predictors, they are primarily used when the list of predictors is relatively short. Related to this, the wider search over functional forms means that there is much greater risk of overfitting the models insample, and hence of constructing models that deliver poor forecasting performance when used in practice.

For any nonparametric method there is a very large set of variations and modifications to the benchmark methods. It is not possible to examine all of these here, so instead we present the simplest forms of the methods supplemented with discussions of their advantages and disadvantages. We focus on three approaches. The first approach, used less commonly in forecasting situations, is kernel estimation. Kernel estimation can be employed in two ways. First, we can estimate the conditional density of Y given Z, then use this density to construct the forecast, just as is done in estimation of density forecasts. Alternatively, we can use kernel methods to directly estimate the feature of the conditional distribution we are interested in, e.g., the conditional mean. The second approach, sieve estimation, is far more commonly used in forecasting studies. Sieve estimators include neural nets as well as other regression methods that use basis functions for approximation. A third approach is to use methods developed in the statistical (machine) learning literature such as boosted regression trees. These can accommodate forecasts even in the presence of large-dimensional predictors.

Nonparametric approaches to forecasting require large data samples and so can be expected to yield better results for such situations. Electricity demand and electricity prices is one such area where a large number of published papers use nonparametric forecasting methods. Electricity loads predicted by temperature variations exhibit a clear nonlinearity as loads are expected to be high for both very cold and very hot weather. Because of the strong nonlinearity in this relationship, nonparametric procedures provide better forecasts than linear models which are not considered reasonable. However, there appears to be little empirical evidence to suggest that one approach is dominant. There is less evidence of nonlinearity in electricity price forecasting. Despite a very large number of applications of various nonparametric procedures there is not much evidence to suggest that one nonparametric approach dominates other approaches or even improves upon linear models; see Aggarwal, Saini, and Kumar (2009) for a review.

Section 11.1 reviews kernel models and section 11.2 goes over the estimation of sieve models. Boosted regression trees are covered in section 11.3, while section 11.4 concludes.