A Defense Of Model Averaging

Abstract

Model-averaged partial regression coefficients have been criticized for averaging over a set of models with coefficients that have different meanings from model to model. This criticism arises because statisticians since Fisher believe that the meaning of a coefficient in a regression model arises from probabilistic conditioning (P (Y |X)) and that coefficients are mere descriptors of conditional association (or \"differences in conditional means\"). Because this association parameter is conditional on a specific set of covariates, the parameter for a predictor varies from model to model. The coefficients in many applied regression models, however, take their meaning from causal conditioning (P (Y |do(X))) and these coefficients estimate causal effect parameters (or simply, causal effects or Average Treatment Effects). Causal effect parameters are also differences in conditional expectations, but the event conditioned on is not the set of covariates in a regression model but a hypothetical intervention. Because an effect parameter for a predictor takes its meaning from causal and not probabilistic conditioning, it is the same from model to model, and an averaged coefficient has a straightforward interpretation as an estimate of a causal effect. But, because the effect parameter is the same from model to model, the estimates of the parameter will generally be biased. By contrast, with probabilistic conditioning, the coefficients are consistent estimates of their parameter in every model. Confounding and omitted variable bias, which are central to explanatory modeling, are meaningless in regression modeling as mere description.