Bayesian Inference about Relative Importance of Variables

• Activity Number: 74
• Type: Topic Contributed
• Date/Time: Monday, August 9, 2004 : 8:30 AM to 10:20 AM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Ehsan S. Soofi*+ and Joseph J. Retzer
• Companies: University of Wisconsin, Milwaukee and Maritz Research
• Keywords: Bayes factor ; entropy ; mutual information ; uncertainty reduction ; variable selection

Abstract

Comparison of the relative importance of explanatory variables appear in many disciplines. Various measures of relative importance have been proposed for regression, ANOVA, logit, and survival analysis. Attempts have been made to define a framework for and requirements of relative importance measures. However, little attention has been paid to characterizing the more general, underlying notion of "importance." It is natural to measure the importance of an explanatory variable by the extent to which its use reduces uncertainty about predicting the outcome of the dependent variable, namely, its information content. Information importance measures are applicable to GLM with qualitative variables and quantitative variables. For normally distributed variables and for variables that can be transformed to normality, the information measures are functions of simple and multiple correlations. These information measures are equivalent to the sequential Bayes factors for the normal regression. For the normal regression, inference about the relative importance of explanatory variables and about the differences between their relative importance measures are performed.