正在学习

14.6 BAYESIAN MODEL AVERAGING

Bayesian model averaging (BMA) typically examines combinations of the form

$p^{c} (y) = i = 1 \sum m ω_{i} : p (y ∣ M_{i}), (14.29)$

where the BMA weights $ω_{i}$ are posteriors for the individual models, $M_{i}$ . The combined forecast is a weighted average of the individual models’ forecasts with weights proportional to the posterior for the models. BMA is, as the name suggests, a model averaging procedure rather than a predictive density combination procedure per se. This means that it assumes the availability of both the data underlying the construction of each of the densities, $p_{i} = p (y ∣ M_{i})$ , along with knowledge of how the data are employed to obtain a predictive density. BMA methods also apply more generally than indicated here—rather than y being the object of interest as it is in this book, we might instead be interested in a parameter which would then replace $y$ in the derivations below. More broadly, the primary motivation for BMA is as a way of dealing with model uncertainty. Of course, this also explains its interest to forecasters.

Denote the m models by $M_{1}, \dots, M_{m}$ . Let $p (M_{i})$ be the prior probability that model i is the true model, while Z is the data. Then the posterior probability for model i can be written

$p (M_{i} ∣ Z) = \frac{p ( Z ∣ M _{i} ) p ( M _{i} )}{\sum _{j = 1}^{m} p ( Z ∣ M _{j} ) p ( M _{j} )} . (14.30)$

The combined model average is then

$p^{c} (y) = i = 1 \sum m p (y ∣ M_{i}) p (M_{i} ∣ Z) . (14.31)$

The marginal likelihood of model i in (14.30) is given by

$P (Z ∣ M_{i}) = \int P (Z ∣ θ_{i}, M_{i}) P (θ_{i} ∣ M_{i}) d θ_{i}, (14.32)$

where $p (θ_{i} ∣ M_{i})$ is the prior density of model $i^{'} s$ parameters, and $ℏ (Z ∣ θ_{i}, M)$ is the likelihood of the data given the parameters and model i.

The difficulty in constructing BMA estimates lies in the complexity of obtaining the objects required to construct the weighted average given above; see Hoeting et al. (1999) for a detailed discussion. Initially a list of models, $M_{1}, \dots, M_{m},$ considered in the combination must be constructed. After choosing the set of candidate models, three issues must be confronted in implementing BMA methods. First, computation of $p (M_{i} ∣ Z)$ in (14.30) requires computation of $p (Z ∣ M_{i})$ in (14.32). Although this is a typical Bayesian estimation problem with plenty of algorithms available for different models, it is time consuming and must be undertaken separately for each model. Second, if m is large, repeated computation of these objects becomes difficult or infeasible unless there are closed-form solutions for the marginal likelihood. Finally, researchers must provide prior probabilities for all of the models, $p (M_{1}), \dots, p (M_{m})$ and (assuming the models are known) priors for the model parameters, $P (θ_{1} ∣ M_{1}), \dots, P (θ_{m} ∣ M_{m})$ . For large values of m such choices are not obvious when many of the models are related to other models.

For linear regression models with normal priors, closed-form solutions are available to solve (14.32). Applying BMA methods to such forecasting models is therefore straightforward. This explains the popularity of the approach. For example, consider the set of forecasting models

$y = X_{i}^{'} β_{i} + W γ + u_{i}, (14.33)$

where the regressors in W are always included, $X_{i}$ represents $k_{i}$ additional regressors included in model $i,$ constructed to be orthogonal to $W$ by using the residuals from a regression of the individual X-variables on $W .$ Assuming that the predictors are strictly exogenous, with $u_{i} \sim N (0, σ_{i}^{2})$ and placing an uninformative prior on $(γ, σ^{2})$ and a Zellner (1986b) g -prior on $β_{i}$ , the posterior for $y_{1}, \dots, y_{t}$ given (X, W) has a closed-form solution—in the above notation this is $p (Z ∣ M_{i})$ for each model $M_{i}$ . Following Magnus, Powell, and Prüfer (2010), apart from assigning equal prior probabilities to each model, the resulting weights are given by

$p (M_{i} ∣ Z) = \frac{( \frac{1}{1 + g _{i}} ) ^{k_{i} /2} ( \frac{1}{1 + g _{i}} SSR _{- i}^{R} + \frac{g _{i}}{1 + g _{i}} SSR _{i}^{U} ) ^{- d /2}}{\sum _{i = 1}^{m} ( \frac{1}{1 + g _{i}} ) ^{k_{i} /2} ( \frac{1}{1 + g _{i}} SSR _{- i}^{R} + \frac{g _{i}}{1 + g _{i}} SSR _{i}^{U} ) ^{- d /2}}, (14.34)$

where, for each model $M_{i}, SSR_{i}^{U}$ is the sum of squared residuals from regression (14.33) and $SSR_{- i}^{R}$ is the sum of squared residuals from the same regression that omits the $X_{i}$ -variables, and $d$ is the number of degrees of freedom from the regression omitting the $X_{i}$ -variables. To implement (14.34), we need to specify a value for $g_{i}$ Fernandez, Ley, and Steel (2001) and Magnus, Powell, and Prüfer (2010) suggest setting $g_{i} = max (T, k_{i}^{2})^{- 1}$

Raftery, Madigan, and Hoeting (1997) suggest a different set of priors that do not result in such closed-form solutions and instead provide an $M C^{3}$ algorithm to estimate the BMA weights. Koop and Potter (2007) consider a set of priors, which differ from the g -prior approach by using an informative prior for the variance. They also use principal components methods to first orthogonalize the $X_{i}$ -variables since orthogonal components are required in their algorithm.

In cases where the models’ marginal likelihoods in (14.32) are difficult to compute, one can use a simple approximation. Recall from chapter 6 that the Bayesian Information Criterion (BIC) is given by

$BIC_{i} = - 2 LL_{i} / T + k_{i} ln (T) / T,$

where $LL_{i}$ is the log-likelihood of model $M_{i}, k_{i}$ is the number of parameters estimated for this model, and $T$ is the sample size. The BIC provides an asymptotic approximation to the marginal likelihood, ln $(P (Z ∣ M_{i})) \approx BIC_{i}$ . Hence, the BMA

weights can be approximated by

$ω_{i} = P (M_{i} ∣ Z) \approx \frac{exp ( - 0.5 BIC _{i} )}{\sum _{i = 1}^{m} exp ( - 0.5 BIC _{i} )} . (14.35)$

One approach to lessening the computational workload, especially if m is large, is to consider only a subset of the models in the model averaging. If this approach is taken, a procedure for choosing which models to include and which models to ignore must be adopted. One method is to remove models from consideration if they appear not to be very good. Raftery, Madigan, and Hoeting (1997) suggest removing models for which $p (M_{i} ∣ Z)$ is much smaller than the posterior probability of the best model. For example, a large model with a relatively low posterior probability would be excluded from the set of models under consideration. The final model average is not a weighted sum over all m original models, but over only a subset of them. An alternative method for including a subset of models is to use a simulation approach to approximating the average in (14.31). This approach uses Monte Carlo methods that require only a subset of the models to actually be evaluated.

Finally, consider how the prior probabilities are set. Any information that can guide this choice should of course be used, but often there are no particular reasons to ex ante prefer some models over others. The linear regression model requires setting priors over the slope parameters. Since these parameters appear in many of the submodels, internal consistency requires the same coefficients to have the same priors across different models. Typically a conjugate prior is chosen. For the normal regression model it is common to assume that $p (β ∣ σ^{2}) \sim N (β_{0}, Ω)$ on the full set of parameters; for each model the prior is then a subset of these parameters. For example, the Zellner g -prior satisfies this internal consistency constraint.

Turning to the model priors, an obvious approach is to set the priors $p (M_{i})$ equal to $1/ m,$ giving each model the same weight. A drawback of this is that many of the models under consideration might be very closely related, and hence as a group end up having a much higher weight than other different and potentially useful models that are not part of a similar cluster. This suggests paying close attention to the design of the list of models $M_{1}, \dots, M_{m}$ considered in the combination, although we are not aware of papers that directly address this important issue.

练习题

Which formula represents the Bayesian Model Averaging (BMA) combination?

p^{c} (y) = \prod_{i = 1}^{m} ω_{i} p (y ∣ M_{i})

p^{c} (y) = \sum_{i = 1}^{m} ω_{i} p (y ∣ M_{i})

p^{c} (y) = \frac{1}{m} \sum_{i = 1}^{m} p (y ∣ M_{i})

p^{c} (y) = max_{i} p (y ∣ M_{i})

What does BMA primarily aim to address?

A. Data normalization

B. Model uncertainty

C. Parameter estimation

D. Computational efficiency

Which of the following are assumptions of BMA? (Select all that apply)

A. Availability of data underlying each density

p_{i} = p (y ∣ M_{i})

B. Knowledge of how data are employed to obtain predictive densities

C. All models must have the same number of parameters

D. Models must be nested within each other

In BMA, the posterior probability for model $i$ is given by $p (M_{i} ∣ Z) = \frac{p ( Z ∣ M _{i} ) p ( M _{i} )}{\sum _{j = 1}^{m} p ( Z ∣ M _{j} ) p ( M _{j} )}$ .

BMA methods are only applicable when the object of interest is $y$ .

The combined model average in BMA is given by $p^{c} (y) = \sum_{i = 1}^{m} p (y ∣ M_{i}) p (M_{i} ∣ Z)$ . The term $p (M_{i} ∣ Z)$ represents the ___.

The marginal likelihood of model $i$ in BMA is given by $P (Z ∣ M_{i}) = \int P (Z ∣ θ_{i}, M_{i}) P (θ_{i} ∣ M_{i}) d θ_{i}$ . The term $P (θ_{i} ∣ M_{i})$ represents the ___.

Explain why constructing BMA estimates can be difficult.

What makes BMA straightforward for linear regression models with normal priors?

Which of the following are difficulties in constructing BMA estimates? (Select all that apply)

A. Computing

p (Z ∣ M_{i})

for each model

B. Specifying prior probabilities for all models

C. Ensuring all models have the same likelihood function

D. Repeated computation for large

m

Which of the following are true about the posterior weights formula in BMA? (Select all that apply)

A. It involves the sum of squared residuals from regression

B. It requires specifying a value for

g_{i}

C. It assumes equal prior probabilities for all models

D. It is only applicable for linear regression models

What is the suggested value for $g_{i}$ in the posterior weights formula?

g_{i} = 1

g_{i} = max (T, k_{i}^{2})^{- 1}

g_{i} = \frac{1}{m}

g_{i} = \sum_{i = 1}^{m} p (M_{i} ∣ Z)

Which of the following are combined in BMA? (Select all that apply)

A. Point forecasts

B. Density forecasts

C. Prior probabilities

D. Posterior probabilities

Which of the following are true about the marginal likelihood formula in BMA? (Select all that apply)

A. It involves integrating over the parameter space

B. It requires knowledge of the prior density of parameters

C. It is the same as the likelihood of the data

D. It is used to compute posterior probabilities

Which of the following correctly describes the relationship between Bayesian Model Averaging (BMA) and linear combinations of density forecasts?

A. BMA is a special case of linear combinations where weights are posteriors from marginal likelihoods, while linear combinations use arbitrary weights.

B. BMA and linear combinations are identical methods with different names.

C. Linear combinations require prior probabilities for models, while BMA does not.

D. BMA can only be applied to point forecasts, while linear combinations are for density forecasts.

Which of the following are valid conditions for ensuring a combined density forecast is mathematically proper? Select all that apply.

A. The weights

ω_{i}

must sum to 1.

B. The weights

ω_{i}

must be nonnegative.

C. The individual densities

p_{i} (y)

must be identical.

D. The combined density must integrate to 1 over its support.

E. The weights

ω_{i}

must be derived from Bayesian posterior probabilities.

In Bayesian Model Averaging, the marginal likelihood $P (Z ∣ M_{i})$ is computed by integrating the likelihood of the data given parameters and model over the prior density of the model's parameters. True or false?

To implement the posterior weights formula in BMA, a value for $g_{i}$ must be specified. A commonly suggested value is $g_{i} = max (T, k_{i}^{2})^{- 1}$ , where $T$ is the sample size and $k_{i}$ is the number of additional regressors in model $M_{i}$ . This value helps balance model complexity and goodness of fit by scaling with _ and _.

登录后解锁笔记、知识点解析、AI 问答

立即登录

14.6 BAYESIAN MODEL AVERAGING