Addressing the Type-I Error Inflation Problem in Cumulative Metaanalysis

• Activity Number: 398
• Type: Contributed
• Date/Time: Wednesday, August 10, 2005 : 10:30 AM to 12:20 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Mingxiu Hu*+ and K. Gordon Lan and Joseph Cappelleri
• Companies: Pfizer, Inc. and Sanofi-Aventis and Pfizer, Inc.
• Keywords: Type I Error ; Cumulative Meta-Analysis ; Law of Iterated Logarithm ; Random Effects Model ; Sequential Tests

Abstract

Cumulative metaanalysis (CMA) typically involves performing an updated metaanalysis every time a new trial is added to a series of similar trials. The estimation of between-study variation poses a major challenge because the CMA testing process generally starts with a small number of studies, which partially explains why the overall type-I error in CMA is so hard to control through conventional random-effect models (CRM) and group sequential methods. As shown in Whitehead (1997), the type-I error rates of both the CRM and the sequential method proposed there can go over the desired level quickly, even with only four inspections. In our simulation studies, the type-I error rates of the CRM method often are 10 times larger than the prespecified level. We proposed a new method based on the law of iterated logarithm (LIL), which controls the type-I error rate at the desired level for a range of scenarios. The number of inspections could affect type-I error rates greatly when it is small, but it has no material impact on the LIL method when it passes 25 inspections---which indicates the proper function of the LIL in controlling type-I error for large numbers of inspections.