Calculating Effect Sizes Using Survey Data

• Activity Number: 279
• Type: Contributed
• Date/Time: Tuesday, August 8, 2006 : 10:30 AM to 12:20 PM
• Sponsor: Section on Survey Research Methods
• Author(s): Jun Liu*+
• Companies: Research Triangle Institute
• Keywords: effect size ; survey data analysis ; clustering ; unequal weighting ; variance component

Abstract

Effect Sizes are widely used in comparative studies to measure the impact of an intervention over the control. Although the definition of the Effect Size is well known and generally accepted, its estimation is less so. Furthermore, when data is collected from complex surveys, where clustering and stratification are common, it is not clear how the Effect Size should be calculated. We present a formula for calculating Effect Sizes that takes into consideration of the survey design features and is design consistent. We use simulations to examine its properties against other alternatives.