正在学习

10.2.5 Bayesian Methods

De Mol, Giannone, and Reichlin (2008) compare the performance of principal components methods to that of Bayesian linear regression models with Gaussian or double exponential priors. Bayesian regressions with a Gaussian prior result in nonzero coefficients on all variables under consideration, although the coefficients are shrunk compared with their OLS equivalents. In fact, assuming i.i.d. regression coefficients, Bayesian prediction with a Gaussian prior amounts to Ridge regression. To see this, consider the linear regression model with Gaussian prior $β \sim N (β_{0}, Σ_{0})$ and normally distributed i.i.d. residuals, $y_{t + 1} = β^{'} x_{t} + u_{t + 1}$ , u ∼ ind $N (0, σ_{u}^{2})$

If we restrict the prior parameters to be independent, $Σ_{0} = σ_{β}^{2} I,$ , as noted by De Mol, Giannone, and Reichlin (2008), the Bayesian estimation problem reduces to Ridge regression,

$\hat{β}^{Bayes} = ar g β min {∣∣ y - X β ∣ ∣^{2} + (σ_{u}^{2} / σ_{β}^{2}) ∣∣ β ∣ ∣^{2}}, (10.21)$

where $∣∣.∣∣$ denotes the Euclidean norm and X and y stack the sample observations of $y_{τ + 1}$ and $x_{τ}$

Assuming shrinkage towards $0 (β_{0} = 0)$ , and a known variance, $σ_{u}^{2}$ , the Bayesian estimator and forecasts take the form

$\hat{β}^{Bayes} = (X^{'} X + σ_{u}^{2} Σ_{0}^{- 1})^{- 1} X^{'} y, \hat{f}_{t + 1∣ t} = x_{t}^{'} \hat{β}^{Bayes} . (10.22)$

Unlike the case with Gaussian priors, double exponential priors tend to put either large or zero weights on the individual regressors and so give rise to sparse prediction models. There is no closed-form solution to the coefficients under double exponential priors, but optimization reduces to the penalized least absolute deviation problem and so can be solved using the Lasso algorithm.

Using the data of Stock and Watson (2005) to forecast growth in industrial production and inflation, De Mol, Giannone, and Reichlin (2008) find empirically that, despite their very different implications for model selection, Bayesian linear regression forecasts generated under either Gaussian or double exponential priors tend to be strongly correlated and produce very similar mean squared forecast errors. In turn, the two sets of forecasts are similar to those generated by principal components regression. The reason why two seemingly very different approaches— one that favors sparse models that typically select just a few predictors versus another that puts nonzero weights on most predictors—yield similar forecasts and predictive accuracy is of course that the underlying X-variables are strongly collinear. In this situation, many variables will be close substitutes and it does not matter a great deal which specific variables—or how many—get selected.

练习题

Which of the following statements about Bayesian regressions with a Gaussian prior is correct?

A. They result in zero coefficients on all variables.

B. They result in nonzero coefficients on all variables, but the coefficients are shrunk compared with their OLS equivalents.

C. They are equivalent to Lasso regression.

D. They are only suitable for sparse prediction models.

What does Bayesian prediction with a Gaussian prior amount to under the assumption of i.i.d. regression coefficients?

A. Principal components regression

B. Lasso regression

C. Ridge regression

D. Double exponential regression

Which of the following is the form of the Bayesian estimator with shrinkage towards $0$ and a known variance $σ_{u}^{2}$ ?

\hat{β}^{Bayes} = (X^{'} X)^{- 1} X^{'} y

\hat{β}^{Bayes} = (X^{'} X + σ_{u}^{2} Σ_{0}^{- 1})^{- 1} X^{'} y

\hat{β}^{Bayes} = (X^{'} X + σ_{β}^{2} I)^{- 1} X^{'} y

\hat{β}^{Bayes} = (X^{'} X + σ_{u}^{2} I)^{- 1} X^{'} y

Which of the following are characteristics of double exponential priors in Bayesian regression?

A. They result in nonzero coefficients on all variables.

B. They tend to put either large or zero weights on the individual regressors.

C. They give rise to sparse prediction models.

D. They have a closed-form solution for the coefficients.

E. The optimization problem can be solved using the Lasso algorithm.

Bayesian linear regression forecasts generated under either Gaussian or double exponential priors produce very different mean squared forecast errors.

Principal components regression forecasts are similar to Bayesian linear regression forecasts under Gaussian priors because the underlying X-variables are strongly collinear.

Assuming i.i.d. regression coefficients, Bayesian prediction with a Gaussian prior amounts to ___.

Unlike Gaussian priors, double exponential priors tend to put either large or zero weights on the individual regressors and so give rise to ___ prediction models.

Explain why Bayesian linear regression forecasts generated under either Gaussian or double exponential priors can be similar.

What is the form of the Bayesian estimator and forecasts with shrinkage towards $0$ and a known variance $σ_{u}^{2}$ ?

Which of the following statements are true about the comparison of principal components methods and Bayesian linear regression models?

A. Bayesian linear regression models with Gaussian priors always outperform principal components methods.

B. De Mol, Giannone, and Reichlin (2008) compare the performance of principal components methods to that of Bayesian linear regression models with Gaussian or double exponential priors.

C. Principal components methods are more suitable for sparse prediction models.

D. Bayesian linear regression models with double exponential priors can give rise to sparse prediction models.

E. Principal components methods and Bayesian linear regression models with Gaussian priors have similar forecasts when the underlying X-variables are strongly collinear.

Which of the following knowledge points are related to the properties of Bayesian regression with different priors and their forecasting performance? (Combine current and prior knowledge)

A. Gaussian latent variable interpretation of principal components

B. Bayesian regressions with a Gaussian prior result in nonzero coefficients on all variables

C. Double exponential priors and sparse prediction models

D. Equivalence of principal components and factor augmented forecasts

E. Bayesian prediction with a Gaussian prior amounts to Ridge regression

F. Empirical findings on Bayesian linear regression forecasts

Which of the following statements correctly describes the relationship between Bayesian regression with a Gaussian prior and Ridge regression?

A. Bayesian regression with a Gaussian prior is equivalent to OLS regression.

B. Bayesian regression with a Gaussian prior is equivalent to Lasso regression.

C. Assuming i.i.d. regression coefficients, Bayesian prediction with a Gaussian prior amounts to Ridge regression.

D. Bayesian regression with a Gaussian prior has no relationship with Ridge regression.

Which of the following are true about Bayesian linear regression forecasts generated under Gaussian and double exponential priors?

A. They tend to be strongly correlated.

B. They produce very different mean squared forecast errors.

C. They are similar to those generated by principal components regression.

D. They favor sparse models that typically select just a few predictors.

E. They put nonzero weights on most predictors.

Bayesian regression with a double exponential prior can be solved using the Lasso algorithm because the optimization reduces to the penalized least absolute deviation problem.

Assuming shrinkage towards $0 (β_{0} = 0)$ and a known variance, $σ_{u}^{2}$ , the Bayesian estimator takes the form $\hat{β}^{Bayes} = (X^{'} X + σ_{u}^{2} Σ_{0}^{- 1})^{- 1} X^{'} y$ , and the forecasts are given by ___.

登录后解锁笔记、知识点解析、AI 问答

立即登录

10.2.5 Bayesian Methods