Incorporating a First-Stage Finite Population Correction (FPC) in Variance Estimation for a Two-Stage Design in the National Assessment of Educational Progress (NAEP)

• Activity Number: 36
• Type: Contributed
• Date/Time: Sunday, July 31, 2011 : 2:00 PM to 3:50 PM
• Sponsor: Section on Survey Research Methods
• Author(s): Jennifer Kali*+ and John Burke and Lloyd Hicks and Lou Rizzo and Keith Rust
• Companies: Westat Inc. and Westat Inc. and Westat Inc. and Westat Inc. and Westat Inc.
• Keywords: Jackknife ; school sample ; replicate weights

Abstract

NAEP utilizes a two-stage design to select samples of school students in each state. Schools are selected with varying probabilities, via a stratified systematic sample. Students are selected with equal probability within schools. Jackknife replication is used for variance estimation, and the current procedure assumes that schools were selected with replacement. In many states the sampling fraction of schools is large, so that this approach overestimates the sampling error. We evaluated an approximate method for incorporating conservative fpc factors directly into the replicate weights. The evaluation of this approach involved three components: 1) implementation under tight deadlines; 2) the impact of the change, using historic data; 3) an evaluation of the biases and variances of the past and proposed approaches, using a population generated to simulate the NAEP grade 8 reading populati