11.2.3.1 White’s QuickNet

11.2.3.1 White’s QuickNet

White (2006) suggests a convenient QuickNet algorithm that provides an easy and convenient way to estimate ANN models. The algorithm was proposed by White (2006) as a way to deal with nonlinearities in ANNs which can greatly complicate estimation of these models. Rather than estimating the values of ${\gamma }_j$ the approach simply draws these coefficients randomly from a reasonable distribution, thereby limiting the estimation part of the problem to running linear regressions.

The algorithm makes use of the leave-one-out cross-validation MSE, CVMSE(q), defined $\mathsf{as}$

$$\mathrm{CVMSE}(q)=\frac{1}{T-1}\sum _{t=1}^{T-1}{\hat{\epsilon }}_{qt+1(-t-1)}^2,\text{\hspace{1em}(11.9)}$$

where

$${\hat{\epsilon }}_{qt+1(-t-1)}=y_{t+1}-{\hat{\beta }}_{0q(-t-1)}^{\prime }{z}_t-\sum _{j=1}^{q}g({\hat{\gamma }}_j^{\prime }{z}_t){\hat{\beta }}_{qj(-t-1)},\text{\hspace{1em}(11.10)}$$

and the estimates ${\hat{\beta }}_{0q(-t-1)}$ and ${\hat{\beta }}_{qj(-t-1)}$ are fitted to the data that omit observation $t+1$ . Here $q$ refers to the number of nonlinear terms and ${\hat{\mathbf{\beta }}}_{0q}^{\prime },{\hat{\gamma }}_j^{\prime },{\hat{\mathbf{\beta }}}_{qj}$ are parameter estimates defined below. More general cross-validation techniques, including methods that leave out blocks of the data, could also be considered; see White (2006). As described by White (2006), the algorithm involves the following steps, none of which involves any nonlinear optimization.

1. Compute initial OLS estimates ${\hat{\beta }}_{00}$ by regressing $y_{t+1}$ on an intercept and $z_t$ Also compute regression residuals, ${\hat{\epsilon }}_{0t+1}=y_{t+1}-{\hat{\beta }}_{00}^{\prime }{z}_t$ and the cross-validation MSE value, CVMSE(0), defined in (11.9) under the assumption that ${\hat{\gamma }}_j=0$ , for $j=1,\dots ,q$ . This defines the linear benchmark. Set $q=1$

2. Randomly generate m sets of coefficients, ${\gamma }_j=\{{\gamma }_{0j},{\gamma }_{1j}{\}}_{j=1}^m$ . For each ${\gamma }_j,$ regress the residuals from iteration $q-1,{\hat{\epsilon }}_{q-1,t+1}$ , on a constant and $g({\gamma }_j^{\prime }{z}_t)$ Pick ${\hat{\gamma }}_q$ as that value of ${\gamma }_j,j=1,\dots ,m$ , that maximizes the $R^2$ of this OLS regression.

3. Compute OLS regression coefficients, ${\hat{\mathbf{\beta }}}_{0q},{\hat{\mathbf{\beta }}}_q=({\hat{\mathbf{\beta }}}_{q1},\dots ,{\hat{\mathbf{\beta }}}_{qq}{)}^{\prime }$ , and residuals,

$${\hat{\epsilon }}_{qt+1}=y_{t+1}-{\hat{\beta }}_{0q}^{\prime }{z}_t-\sum _{j=1}^{q}g({\hat{\gamma }}_j^{\prime }{z}_t){\hat{\beta }}_{qj},$$

by regressing $y_{t+1}$ on $z_t$ and $g({\hat{\gamma }}_i^{\prime }{z}_t),j=1,\dots ,q$ . Also compute the $\mathrm{CVMSE}(q)$ . Increase $q$ to $q+1$ , or stop in the case $q>\bar{q}$ . Otherwise, go back to step 2.

4. Choose $\hat{q}$ by cross validation, i.e., by finding the value of $q$ that minimizes

$$q=\underset{q\in [1,...,\bar{q}]}{\arg }\underset{\mathbf{CVMSE}}{\min }(q).$$

Using this value, $\hat{q}:$ , let ${\hat{\theta }}_{\hat{q}}=({\hat{\beta }}_{0\hat{q}}^{\prime },{\hat{\beta }}_{\hat{q}}^{\prime },{\hat{\gamma }}_1^{\prime },\dots ,{\hat{\gamma }}_{\hat{q}}^{\prime }{)}^{\prime }.$

White suggests that a nonlinear optimization step can be added after the fourth step. He also provides specific advice on how to avoid multicollinearity in the regressions by properly scaling the distribution from which the random coefficients ${\gamma }_j$ are drawn in step 2. Specifically, to avoid multicollinearity between the terms in $g({\gamma }_0+{\gamma }_1^{\prime }{z}_t)$ , in the univariate case, ${\gamma }_1$ should be selected to be of the same order of magnitude as the standard deviation of the data, sd $\left(z_t\right)$ , or possibly larger, while ${\gamma }_0$ should be of roughly the same order of magnitude as sd $({\gamma }_1{z}_t)$ . White notes that using $m=500\mathrm{or}m=1000$ values to estimate $\gamma$ in step 2 tends to work well.

Racine (2001) finds that neural networks do not improve on linear forecasts of monthly stock returns by means of a variety of macroeconomic and financial predictor variables. When used to guide the positions of an investment rule that switches in and out of stocks and T-bills depending on the sign of the predicted excess return on stocks over T-bills, Racine finds that recursive neural net forecasts result in lower wealth and higher risks than forecasts from a linear model.