13.4.2 Nonparametric and Semiparametric Density Estimation

13.4.2 Nonparametric and Semiparametric Density Estimation

Conditional distributions can be estimated fully nonparametrically by kernel density methods. From Bayes’ rule, the conditional distribution of y given z is

$$p(y|z)=\frac{p(y,z)}{p(z)}.\text{\hspace{1em}(13.38)}$$

The standard approach to estimating conditional densities is thus to combine two density estimates—one for $p(y,z)$ and one for $p(z)$ . For a random $(k\times 1)$ vector $Z$

Figure 13.7: Time-series plots of one-step-ahead conditional volatility estimates for daily US stock market returns implied by a two-state Markov switching model.

and observations $\{z_t\},t=1,...,T$ , the joint density at any point, $z,$ can be estimated from a weighted average around that point:

$${\hat{p}}_Z(z)=\frac{1}{T{b}^k}\sum _{t=1}^{T}K\left(\frac{z_t-z}{b}\right),\text{\hspace{1em}(13.39)}$$

where the multivariate kernel K is nonnegative, integrates to 1, and often is assumed to be symmetric. To allow for different bandwidths, a product kernel might be used:

$${\hat{p}}_Z(z)=\frac{1}{T{b}_1{b}_2\cdot \cdot \cdot b_k}\sum _{t=1}^{T}K\left(\frac{z_{1t}-z_1}{b_1}\right)K\left(\frac{z_{2t}-z_2}{b_2}\right)\times \cdot \cdot \cdot \times K\left(\frac{z_{kt}-z_k}{b_k}\right).\text{\hspace{1em}(13.40)}$$

When the scales of the different variables in Z are different, it can be useful to choose individual bandwidths. The product kernel in (13.40) also ensures internal consistency in the use of kernels for the conditioning variables since the same kernels appear in the numerator and denominator of the approximation to (13.38).

Some methods mix purely parametric and purely nonparametric approaches. Gallant and Nychka (1987) proposed what they call a semi-non-parametric (SNP) approach to constructing conditional densities. The motivation for their method is broader than constructing forecast densities—it is a general approach to robust maximum likelihood estimation—but is a useful procedure for this problem as well. The idea is to construct a model for the mean and variance, just as in section 13.2, and then apply a series approximation to the density of the standardized data. Note that the mean, variance, and distribution are estimated simultaneously in a single maximum likelihood setting.

The SNP method approximates the density ${\varphi }_Y$ by means of a series approach. Gallant and Nychka (1987) consider many types of series, but we restrict the exposition here to the most popular version which uses Hermite polynomials. The density can then be written as

$$p_{{\eta }_{t+1}}^{\mathrm{SNP}}=\frac{c^{-1}}{{\sigma }_{t+1|t}}{\left(\sum _{i=0}^m{\omega }_i{\eta }_{t+1}^i\right)}^2\varphi ({\eta }_{t+1}),\text{\hspace{1em}(13.41)}$$

where $\varphi$ is the standard normal distribution and ${\eta }_{t+h}={\sigma }_{t+1|t}^{-1}(y_{t+1}-{\mu }_{t+1|t})$ For example, if we approximate the mean by $\beta z_t,$ and assume a constant variance, ${\sigma }^2$ , we get ${\eta }_{t+1}={\sigma }^{-1}(y_{t+1}-{\beta }^{\prime }{z}_t)$ . The normalizing constant $c=$ $\begin{array}{r}\int {\left({\sum }_{i=0}^m{\omega }_i{\eta }_{t+1}^i\right)}^2\varphi ({\eta }_{t+1})d{\eta }_{t+h}\end{array}$ ensures that $\begin{array}{r}\int d{\mathbf{p}}_{{\eta }_{t+1}}^{\mathrm{SNP}}d{\eta }_{t+1}=1\end{array}$ . Using squared values of the weights on the standard normal terms $\varphi ({\eta }_{t+1})$ ensures that the density is nonnegative everywhere. Parameter estimation for this type of model involves a constrained maximum likelihood problem,

$$(\hat{\beta },{\hat{\sigma }}^2,{\hat{\omega }}_1,\dots ,{\hat{\omega }}_m)=\arg \max T^{-1}\sum _{t=0}^{T-1}\log \left[\frac{c^{-1}}{\hat{\sigma }}(\sum _{i=0}^m{\omega }_i{\eta }_{t+1}^i{)}^2\varphi ({\eta }_{t+h})\right].\text{\hspace{1em}(13.42)}$$

As stated in (13.42), the weights ${\omega }_i$ are identifiable only up to a scalar and so it is standard to use the normalization ${\omega }_0=1$ . If we set $m=0$ , it is clear that the method nests standard quasi-maximum likelihood estimation. Also note that c is a function of the weights. For example, if $m=1$ , we have

$$\begin{array}{l}c=\int {\left(\sum _{i=0}^m{\omega }_i{\eta }^i\right)}^2\varphi (\eta )d\eta \\ \\ \\ =\int (1+{\omega }_1\eta {)}^2\varphi (\eta )d\eta \\ \\ \\ =\int (1+{\omega }_1^2{\eta }^2+2{\omega }_1\eta )\varphi (\eta )d\eta \\ \\ =(1+{\omega }_1^2).\end{array}$$

The remaining problem is to select the order of the polynomial, m. Standard model selection procedures such as AIC or BIC can be employed. Coppejans and Gallant (2002) also suggest a cross-validation method.

Figure 13.8 plots SNP densities fitted to daily US stock market returns on the S&P500 index over the period 1990–2015. The model allows for GARCH dynamics in the conditional volatility and assumes a constant mean. We show results for $m=1,5$ , 10 for a day with high volatility (left panels) and a day with low volatility (right panels). The densities for $m=5,$ , 10 are notably flatter than the Gaussian density $(m=0)$ which is superimposed on the graphs.

Extensions of the SNP method allow the weights ${\omega }_i$ to depend on a set of conditioning variables; see Gallant and Tauchen (1989) for an exposition and application. As stated above, all of the variation in y due to the conditioning z-variables arises from centering on the conditional mean. However, extensions also allow the variance as well as the density of the centered and standardized outcome to depend on conditioning information.

m = 5, low volatility (5th percentile)

m = 10, high volatility (95th percentile)

m = 10, low volatility (5th percentile)

Figure 13.8: Semi-non-parametric (SNP) density plots for m = 1, 5, 10 at high and low volatility levels.

The SNP approach has been used in a number of empirical applications. Using models for expected stock returns and conditional volatility, Harrison and Zhang (1999) use the SNP approach to model the risk-return trade-off on stocks at long horizons, while Gallant, Rossi, and Tauchen (1992) use the approach to model the relation between stock price movements and trading volume.