Estimates of Variance Components in Random Effects Meta-Analysis: Sensitivity to Violations of Normality and Variance Homogeneity

• Activity Number: 112
• Type: Contributed
• Date/Time: Monday, August 12, 2002 : 12:00 PM to 1:50 PM
• Sponsor: Social Statistics Section*
• Author(s): Jeffrey Kromrey*+ and Kristine Hogarty+
• Affiliation(s): University of South Florida and University of South Florida
• Keywords: meta-analysis ; variance components ; random effects models

Abstract

Models for meta-analysis may be roughly divided into fixed and random effects models, with the latter incorporating an estimate of the population variance of effect sizes (tau-squared). Because sample estimates of this variance have been derived under assumptions of normality and variance homogeneity, this research used Monte Carlo methods to investigate the accuracy and precision of point and interval estimates of tau-squared when such assumptions are violated. Factors investigated included characteristics of both the populations from which samples were drawn for primary studies (distribution shape, variance heterogeneity, true value of tau-squared), and the corpus of studies comprising each simulated meta-analysis (sample size and number of studies). Of the three-point estimates, the observed variance estimator showed poor performance across most conditions. The Q-based and ML estimators provided similar performance in terms of bias and standard errors, but notable differences in confidence band coverage and width with the latter estimator providing narrower but less accurate intervals. Results were interpreted in terms of the sensitivity of random-effects meta-analytic models.