17.4.2 Recentered Test Statistic

17.4.2 Recentered Test Statistic

Clark and West (2007) argue that encompassing tests should account for the covariance of ${\hat{e}}_{1t+1}$ and $({\hat{e}}_{2t+1}-{\hat{e}}_{1t+1})$ in cases where the (large) model used to construct $f_{2t+1|t}$ nests the (smaller) model used to construct $f_{1t+1|t}$ . They note that the difference in the squared forecast errors is

$$\begin{array}{r}{\hat{e}}_{1t+1}^2-{\hat{e}}_{2t+h}^2=-2{\hat{e}}_{1t+1}({\hat{e}}_{2t+h}-{\hat{e}}_{1t+1})-({\hat{e}}_{2t+1}-{\hat{e}}_{1t+1}{)}^2.\end{array}\text{\hspace{1em}(17.29)}$$

The term entering the numerator for $\hat{\beta }$ in (17.7) is the first term in (17.29). When the restricted (small) model is true, we expect the variance of the forecasts generated by the large model to be bigger since this model needs to estimate a greater number of parameters. When the small model encompasses the large model, we expect the differences in average MSE values to be negative, so the left-hand side in (17.29) is expected to be negative on average even though both models are “true,” one being more profligately parameterized. When the forecasts are equal on average (when weighted), the first term on the right-hand side of (17.29) is expected to be 0. The last term on the right-hand side in (17.29) is clearly positive on average. Clark and West (2007) argue that we should correct the difference in MSE values by the second term in (17.29) which accounts for the negative bias, suggesting a test of the sample average of ${\hat{e}}_{1t+1}^2-{\hat{e}}_{2t+h}^2+({\hat{e}}_{2t+1}-{\hat{e}}_{1t+1}{)}^2=-2{\hat{e}}_{1t+1}({\hat{e}}_{2t+1}-{\hat{e}}_{1t+1})$ . This motivates a test of the covariance of ${\hat{e}}_{1t+h}$ and $({\hat{e}}_{2t+1}-{\hat{e}}_{1t+1})$ to be 0 as a test of equal MSE in situations with nested forecasting models.

Clark and West (2007) propose the following adjusted MSE test statistic:

$$\begin{array}{l}\mathrm{MS}{\mathrm{E}}^{\mathrm{adj}}=T_P^{-1}\sum _{t=T_R}^{T-1}{\hat{e}}_{1t+1}^2-\left[T_P^{-1}\sum _{t=T_R}^{T-1}{\hat{e}}_{2t+1}^2-T_P^{-1}\sum _{t=T_R}^{T-1}({\hat{f}}_{1t+1|t}-{\hat{f}}_{2t+1|t}{)}^2\right]\\ =-2T_P^{-1}\sum _{t=T_R}^{T-1}{\hat{e}}_{1t+1}\left({\hat{f}}_{1t+1|t}-{\hat{f}}_{2t+1|t}\right)\\ =2T_P^{-1}\sum _{t=T_R}^{T-1}{\hat{e}}_{1t+1}\left({\hat{e}}_{1t+1|t}-{\hat{e}}_{2t+1|t}\right).\end{array}\text{\hspace{1em}(7.30)}$$

The term $\begin{array}{r}{T}_P^{-1}{\sum }_{t=T_R}^{T-1}({\hat{f}}_{1t+1|t}-{\hat{f}}_{2t+1|t}{)}^2\end{array}$ is a measure of the extent to which parameter estimation error affects the second (large) model more than the nested first (small) model. Since this term gets subtracted from the MSE performance of the second model, the adjusted MSE test will of course make the larger model look better than the conventional MSE test which does not include such an adjustment term.

Specifically, define

$${\tilde{d}}_{t+1}=(y_{t+1}-{\hat{f}}_{1t+1|t}{)}^2-\left[(y_{t+1}-{\hat{f}}_{2t+1|t}{)}^2-({\hat{f}}_{1t+1|t}-{\hat{f}}_{2t+1|t}{)}^2\right].\text{\hspace{1em}(17.31)}$$

We can then run an OLS regression of the adjusted MSE difference, ${\tilde{d}}_{t+1}$ , on a constant,

$${\tilde{d}}_{t+1}=\mu +{\epsilon }_{t+1},\text{\hspace{1em}(17.32)}$$

and use a one-sided t-test for $\mu .$ . Clark and West propose to compare this adjusted MSE test against standard normal critical values of the normal distribution, i.e., 1.282 for a test with a size of 10%, and 1.645 for a 5% test, although these result in a size slightly below the nominal size in sufficiently large samples.

The adjusted test statistic is closely related to the earlier tests for encompassing. In fact, the t-statistic for the adjusted test for equal predictive accuracy based on (17.31) and (17.32) is the same as the ENC-T statistic and Clark and West (2007) establish conditions under which the distribution of the ENC-T test is asymptotically normal.9

Again this test addresses whether one model is better than another model, when evaluated at the pseudo-true probability limits. By construction, the test favors the larger model over the smaller one, relative to the unadjusted MSE test. Even if it is concluded from the test that the larger model is best, it does not follow that, in any given finite sample, it is best to use this model to generate out-of-sample forecasts. This is because the adjustment term does not actually reduce the forecast errors of the larger model—it only serves the purpose of sharpening inference about the relative performance of the two models in an environment where parameter estimation error is not a factor.