13.4 APPROACHES TO DENSITY FORECASTING

13.4 APPROACHES TO DENSITY FORECASTING

Fully parametric approaches to density estimation assume that the conditional density ${\rho }_Y(y|z,\theta )$ is known apart from a finite-dimensional vector of parameters, θ . Classical approaches use the data, z, to obtain parameter estimates, $\hat{\theta }(z)$ , and estimate the predictive density, $p_Y(y|z,\hat{\theta }(z))$ , conditional on these plug-in estimates.

To estimate $\theta ,$ we require some form of loss function on the estimates: something is maximized or minimized to obtain the estimates, $\hat{\theta }(z)$ . The most popular loss function relating how “close” a candidate density, $p(y|z,\theta )$ , is to the true density, ${\varphi }_0(y|z,{\theta }^0)$ , is the Kullback–Leibler (KL) distance. The KL distance between the true distribution, $\begin{array}{r}\frac{\hslash }{P0}:\end{array}$ , and the parametric distribution, $\mathbf{p},$ is

$$\mathrm{KL}(p_0,p)=E_{p_0}\left[\ln \left(\frac{p_0}{p}\right)\right]=E_{p_0}[\ln (p_0(y|z,{\theta }^0))]-E_{p_0}[\ln (p(y|z,\theta ))].\text{\hspace{1em}(13.25)}$$

Since the first term in (13.25) does not depend on $\theta ,$ , minimizing the KL distance over $\theta$ is the same as maximizing $E_{{\mathbf{p}}_0}[\ln (\mathbf{p}(\mathbf{y}|\mathbf{z},\mathbf{\theta }))]$ , the expected log of the likelihood, over $\theta .$ . This is known as the expected log score. Hence, using the MLE or quasi-maximum likelihood estimation (QMLE), ${\hat{\theta }}_{\mathrm{MLE}}(z)$ , as a plug-in estimator to construct the estimated predictive density is the same as minimizing the KL distance between the densities. MLE and QMLE methods are well known and well understood which helps explain the popularity of this approach.

For outcomes that are continuously distributed, a wide range of models could be employed. An obvious first approach is to assume normality; for example $p(y|z,\theta )$ could be a conditional normal model with mean ${\beta }^{\prime }z$ and variance ${\sigma }^2,$ , and so maximum likelihood methods would give the usual OLS plug-in estimator, $p(y|z)\sim$ $\begin{array}{r}\mathrm{N}({\hat{\beta }}_{\mathrm{MLE}}^{\prime }z,{\hat{\sigma }}_{\mathrm{MLE}}^2)\end{array}$ . For many problems this distribution is considered too restrictive, so many other distributions have been considered. We next cover some of the most popular of these parametric models.

In situations where the density of ${\eta }_{t+1}$ is unknown, we can use quasi-maximum likelihood estimation to obtain the parameters of the density model. This works by first estimating the parameters of the conditional mean and conditional variance equations, assuming a normal density. Provided that the conditional mean and variance equations are correctly specified, this will yield consistent estimates even if the density for $\eta$ is misspecified. Density forecasts can then be generated based on the standardized residuals, ${\hat{\eta }}_{t+1}=(y_{t+1}-{\mu }_{t+1|t}(\hat{\theta }))/{\sigma }_{t+1|t}(\hat{\theta })$ . For example, a bootstrap can be used to draw from the empirical distribution function.

Given a set of estimates $\{{\bar{\hat{\mu }}}_{t+1|t},{\hat{\sigma }}_{t+1|t}{\}}_{t=0}^{T-1}$ , we can construct standardized residuals,

$${\hat{\eta }}_{t+1}=\frac{y_{t+1}-{\hat{\mu }}_{t+1|t}}{{\hat{\sigma }}_{t+1|t}},\text{\hspace{1em}(13.26)}$$

and use these to estimate the distribution of ${\eta }_{t+1}$

The simplest approach is to use the standardized residuals $\{{\hat{\eta }}_{t+1}\},t=0,\dots ,$ $T-1$ as raw data to estimate a parametric density, ${\mathbf{p}}_{\mathbf{\eta }}$ . In fact, since the standardized residuals are constructed to have zero mean and unit variance, if we assume a normal distribution for $\eta ,p_{\eta }$ would be fully known without requiring data on the residuals. More commonly, a less restrictive density could be chosen—for example the tdistribution with an estimated number of degrees of freedom.

Alternatively, any nonparametric approach to density estimation can be used on the raw data $\{{\hat{\eta }}_{t+1}\},t=0,\dots ,T-1$ , to obtain an estimate for ${\hat{p}}_{\eta }$ . A typical approach is to use a kernel estimator for the density. Estimates for the density ${\mathbf{p}}_{\eta }$ at η would then be given by

$${\hat{p}}_{\eta }(\eta )=\frac{1}{Tb}\sum _{t=0}^{T-1}K\left(\frac{{\hat{\eta }}_{t+1}-\eta }{b}\right),\text{\hspace{1em}(13.27)}$$

for some bandwidth b and kernel K (·). A standard choice of kernel is either the Gaussian kernel $K(x)=(2\pi {)}^{-1}\exp (-0.5x^2)$ or the Epanechnikov kernel $K(x)=$ $0.75(1-x)1(|x|<1)$ , where $1(a)=1$ if a is true and is 0 otherwise. For either choice—or any choice in which the kernel integrates to 1—the mean of the estimated density equals the mean of $\{{\hat{\eta }}_{t+1}\},t=0,\dots ,T-1$ . The choice of kernel is generally not considered too important for the properties of the density estimate. The choice of the bandwidth parameter b is considered more critical. Larger values for b cause the estimate, ${\hat{\mathbf{p}}}_{\eta }(\eta )$ , to put larger weights on values of ${\hat{\eta }}_{t+1}$ further away from η and hence provide a smoother distribution. This, however, comes at the cost of adding a bias to the estimate. Optimal choices of bandwidth trade off these effects. A standard choice of the bandwidth for the Gaussian kernel is $1.06{\hat{\sigma }}_{\eta }{T}^{1/5}$ , where ${\hat{\sigma }}_{\eta }$ is the standard deviation of $\{{\hat{\eta }}_{t+h}{\}}_{t=0}^{T-1}$ , which should be 1 given the centering and standardization in (13.26). It should be recalled, however, that such choices are “optimal” not in regards to obtaining the best density forecast, but in terms of some criterion defined over the density of ${\hat{\eta }}_{t+1}$ itself. This is only one component of the density forecast, so it would not be surprising to find that optimal choices of the bandwidth from the kernel literature do not result in better density forecasts on some relevant criterion for the density forecast. A thorough exposition of density estimation is available in Pagan and Ullah (1999, chapter 2).