正在学习

17.2 TESTS OF EQUIVALENT EXPECTED LOSS: THE DIEBOLD–MARIANO TEST

Rather than focusing on the null that one of the forecasts dominates the other, an alternative is to examine whether the forecasts perform equally well, i.e., test whether the losses are equivalent across the two methods. Such tests for equivalent loss can be quite useful. For example, we might not bother to use a new and complicated forecast method if it only attains the same expected loss as a simpler, existing method. This section covers tests that assume estimation errors arising from the sampling scheme used to generate the forecasts can largely be ignored. The next section explicitly focuses on tests that account for such effects.

17.2.1 Tests of Loss Equivalence under MSE Loss

Using ideas in Morgan (1939), Granger and Newbold (1973) suggest an approach that is based on correlations of variables constructed from the forecast errors. Consider two forecast errors $e_{t + h} = (e_{1 t + h}, e_{2 t + h})$ , with zero mean (so they are unbiased) and a variance–covariance matrix whose elements are labeled $σ_{ij}$ . Regardless of the correlation between the two forecast errors, we have

$E [(e_{1 t + 1} + e_{2 t + 1}) (e_{1 t + 1} - e_{2 t + 1})] = σ_{11}^{2} - σ_{22}^{2}, (17.12)$

i.e., the correlation between these two constructed variables is equal to the difference between the variances. Under MSE loss, this is also the difference between the two loss functions. Thus a test based on the correlation between the sum of the forecast errors and the difference between them amounts to a test of the null hypothesis $H_{0} : σ_{11}^{2} = σ_{22}^{2}$ versus $H_{1} : σ_{11}^{2} \neq = σ_{22}^{2}$ , i.e., that the expected losses are identical versus one model producing lower losses than the other.6 Alternatively, a simple regression can be used to test this hypothesis:

$e_{1 t + 1} + e_{2 t + 1} = β (e_{1 t + 1} - e_{2 t + 1}) + ε_{t + 1} . (17.13)$

This regression has no constant since the forecast errors are assumed to have zero mean. The t-statistic for the slope coefficient can be used to test that $β$ equals 0 and this will follow an approximate normal distribution under the null. As pointed out by Harvey, Leybourne, and Newbold (1998), unless the regressor and regressand are independent (e.g., the forecasts are bivariate normal), then robust standard errors such as those in White (1982) need to be used since the variances of the residuals and the regressor are related. Since the forecasts are often generated from estimated models, the issues in West (1996) also arise here.

练习题

What is the primary purpose of tests for equivalent loss in forecasting?

A. To determine which forecast method dominates the other

B. To examine whether the forecasts perform equally well

C. To compare the accuracy of different forecasting models

D. To evaluate the complexity of forecasting methods

What assumption is made by the tests covered in this section regarding estimation errors?

A. Estimation errors are significant and must be accounted for

B. Estimation errors can be largely ignored

C. Estimation errors follow a normal distribution

D. Estimation errors are independent of the forecast errors

What is the null hypothesis for the loss equivalence test under MSE loss?

H_{0} : σ_{11}^{2} \neq = σ_{22}^{2}

H_{0} : σ_{11}^{2} = σ_{22}^{2}

H_{0} : σ_{11}^{2} > σ_{22}^{2}

H_{0} : σ_{11}^{2} < σ_{22}^{2}

Which of the following are correct regarding the regression-based test for loss equivalence?

A. The regression has a constant term

B. The t-statistic for the slope coefficient is used to test

β = 0

C. The regression follows an approximate normal distribution under the null

D. Robust standard errors are always used in the regression

What are the conditions under which robust standard errors should be used in the regression-based test for loss equivalence?

A. When the forecasts are bivariate normal

B. When the regressor and regressand are independent

C. When the variances of the residuals and the regressor are related

D. When the forecasts are generated from estimated models

The Diebold–Mariano test focuses on testing whether one forecast dominates the other.

Under MSE loss, the difference between the variances of the forecast errors is equal to the difference between the two loss functions.

The null hypothesis for the loss equivalence test is $H_{0} : σ_{11}^{2} = σ_{22}^{2}$ versus the alternative $H_{1} : σ_{11}^{2} \neq = σ_{22}^{2}$ . The test statistic used is the t-statistic for the slope coefficient in the regression $e_{1 t + 1} + e_{2 t + 1} = β (e_{1 t + 1} - e_{2 t + 1}) + ε_{t + 1}$ , which follows an approximate ___ distribution under the null.

When the regressor and regressand in the regression-based test for loss equivalence are not independent, ___ standard errors such as those in White (1982) need to be used.

Explain the significance of the Diebold–Mariano test in the context of forecasting.

What is the role of robust standard errors in the regression-based test for loss equivalence, and when are they typically used?

Which of the following statements are true regarding the assumptions and implications of the Diebold–Mariano test?

A. The test assumes that estimation errors can be largely ignored.

B. The test is designed to determine if one forecast method dominates the other.

C. The test compares the expected losses of two forecasting methods.

D. The test requires the forecast errors to have a non-zero mean.

Which of the following are key considerations when conducting the regression-based test for loss equivalence?

A. The regression includes a constant term.

B. The t-statistic for the slope coefficient is used.

C. The test assumes the forecast errors are independent.

D. Robust standard errors may be necessary.

When testing for loss equivalence under MSE loss using the Diebold–Mariano test, which of the following correctly describes the null hypothesis? Assume two forecast errors $e_{t + h} = (e_{1 t + h}, e_{2 t + h})$ with zero mean and variance-covariance matrix elements $σ_{ij}$ .

H_{0} : σ_{11}^{2} = σ_{22}^{2}

H_{0} : σ_{11}^{2} \neq = σ_{22}^{2}

H_{0} : β = 1

e_{1 t + 1} + e_{2 t + 1} = β (e_{1 t + 1} - e_{2 t + 1}) + ε_{t + 1}

H_{0} : β = 0

y_{t + 1} = β_{1} f_{1 t + 1∣ t} + β_{2} f_{2 t + 1∣ t} + ε_{t + 1}

Which of the following statements are correct regarding the Diebold–Mariano test for loss equivalence under MSE loss? Select all that apply.

A. The test assumes forecast errors are unbiased.

B. The test uses a regression of the form

e_{1 t + 1} + e_{2 t + 1} = β (e_{1 t + 1} - e_{2 t + 1}) + ε_{t + 1}

C. The test statistic for

β

follows an approximate normal distribution under the null.

D. The test requires robust standard errors if the regressor and regressand are independent.

E. The test is valid even if forecasts are generated from nested models.

The Diebold–Mariano test for loss equivalence under MSE loss can be used to determine whether one forecast method encompasses another.

Under MSE loss, the Diebold–Mariano test for loss equivalence is based on the correlation between the sum of the forecast errors ( $e_{1 t + 1} + e_{2 t + 1}$ ) and their difference ( $e_{1 t + 1} - e_{2 t + 1}$ ), which equals $___$ .

登录后解锁笔记、知识点解析、AI 问答

立即登录

17.2 TESTS OF EQUIVALENT EXPECTED LOSS: THE DIEBOLD–MARIANO TEST