Adjusting for Publication Bias in the Presence of Heterogeneity

• Activity Number: 141
• Type: Topic Contributed
• Date/Time: Monday, August 12, 2002 : 2:00 PM to 3:50 PM
• Sponsor: Section on Health Policy Statistics*
• Author(s): Norma Terrin*+ and Christopher Schmid and Joseph Lau and Ingram Olkin
• Affiliation(s): Tufts New England Medical Center and New England Medical Center and Tufts New England Medical Center and Stanford University
• Keywords: publication bias ; heterogeneity ; selection model ; trim and fill ; funnel plot

Abstract

We present a simulation study that compares two statistical methods of adjusting for publication bias in meta-analyses: "trim and fill" and selection modeling. Trim and fill, which is based on a popular graphical tool called the funnel plot, is intuitively appealing and comprehensible by non-statisticians. Selection modeling, which uses likelihood methods, is more complex. Parameters for the simulations were selected following a descriptive analysis of a database of actual meta-analyses. The results indicate that when studies are heterogeneous (estimate different effects), trim and fill may inappropriately adjust for publication bias where none exists. We found trim and fill may spuriously adjust for non-existent bias if: 1.) variability among studies causes some precisely estimated studies to have effects far from the global mean or: 2.) an inverse relationship between treatment effect and sample size is introduced by the studies' a priori power calculations. The selection model performed better than trim and fill, although its frequency of convergence varied, depending on the simulation parameters.