Impact of Randomization and Concurrent Controls in a Pilot Study Designed to Provide Preliminary Evidence to Support Subsequent Investigation

• Activity Number: 412
• Type: Contributed
• Date/Time: Tuesday, August 2, 2011 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Consulting
• Author(s): Rickey E. Carter*+ and Qian Shi
• Companies: Mayo Clinic and Mayo Clinic
• Keywords: Pilot study ; Study design

Abstract

A well-designed pilot study helps advance science by providing essential preliminary data to support or motivate further study. Often, preliminary evidence of efficacy is sought after to provide rationale for the continuation of the investigations of a candidate intervention. When an intervention's efficacy is quantified as a Bernoulli random variable, for example, tumor response in oncology studies, probability mass functions can be enumerated to determine the probability that the observed result from a pilot study supports further evaluation of the intervention. In this paper, an 'efficacy signal' is computed using one- and two-sample pilot study designs. Efficacy signal is defined as the probability of observing a more favorable response proportion relative to a historical control (one sample setting) or the probability of having the new intervention's response proportion numerically superior to a concurrent control (two sample setting). In this sense, the 'efficacy signal' can be viewed as an innovative power function for pilot studies. Recommendations for pilot study designs will be drawn from case studies emanating from our Center for Translational Science Activities (CTSA).