Evaluation of Sample Designs Using Results from the Programme for the International Assessment of Adult Competencies

• Activity Number: 304
• Type: Contributed
• Date/Time: Tuesday, August 5, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Survey Research Methods Section
• Author(s): Valerie Hsu*+ and Thomas Krenzke and Leyla Mohadjer and Sixia Chen
• Companies: Westat and Westat and Westat and Westat
• Keywords: sample design ; intra-class correlation ; design effect ; clustering ; stratification

Abstract

This presentation describes our evaluation of sampling results in the Programme for the International Assessment of Adult Competencies (PIAAC) Cycle 1 Round 1 to help guide the development of efficient sample designs for the second cycle. We used intra-class correlation coefficients estimated at each sampling stage to evaluate how to reduce the clustering effect. We also reviewed the survey outcome (proficiency scores) to identify effective stratification and sorting variables. For the next cycle, initial sample sizes may need to be increased to take into account the effect of the sample design components (cluster sizes, stratification, variation in weights, multiple imputation) on the resulting design effects (DEFFs) observed in Cycle 1 (or an expected DEFF due to design improvements since Cycle 1) so that the quality of the resulting estimates is comparable across countries. In addition, sample designs may need to be modified to reduce the overall expected DEFF.