The Use of Internal Pilots to Derive Powerful and Cost-efficient Designs for Studies with Multilevel Data

• Activity Number: 199
• Type: Contributed
• Date/Time: Tuesday, August 10, 2004 : 8:30 AM to 10:20 AM
• Sponsor: Social Statistics Section
• Author(s): Mirjam Moerbeek*+
• Companies: Utrecht University
• Keywords: optimal sample sizes ; sample size adjustment ; power ; multilevel model ; intraclass correlation ; cost function

Abstract

Many data in social science research have a multilevel structure with persons nested within groups. To design a cost-efficient study to test the effect of a predictor on the outcome at a desired power level, the optimal number of groups and group size need to be calculated. Optimal sample size formulae depend on the intraclass correlation coefficient (ICC) and variance of the predictor. An initial estimate of these may be obtained from similar studies in the past or from subject-matter knowledge. Incorrect estimates, however, may result in an under- or over-powered study. This presentation illustrates the use of internal pilots for studies with costs associated with sampling and measuring groups and persons within groups. At the design stage the sample sizes and associated costs are calculated based on an initial estimate of the ICC and variance of the predictor. At an interim time point in the study these two parameters are estimated and the sample sizes and costs are adjusted. All data, including those of the pilot study, are used in the final analysis. Designs with different interim time points are compared on basis of their Type I error rates, power levels, and median costs.