Adjustments to Student Weights to Account for Student Nonresponse in the National Assessment of Educational Progress

• Activity Number: 226 - Recent Developments in Statistical Methodology for the National Assessment of Educational Progress
• Type: Topic Contributed
• Date/Time: Monday, July 31, 2017 : 2:00 PM to 3:50 PM
• Sponsor: Social Statistics Section
• Author(s): John Riddles* and Gonzalo Rivero and John Burke and Keith Rust
• Companies: Westat and Westat and Westat and Westat
• Keywords: Survey weights ; non-ignorable non response ; model assessment ; conditional inference trees ; decision trees

Abstract

The National Assessment of Educational Progress (NAEP) conducts regular assessments in mathematics and reading among samples of students in grades 4, 8, and 12. Since the resulting missing data are not missing at random, nonresponse adjustments are made via a process of weighting class adjustments. Using 2015 NAEP data we investigated whether improvements could be made to the process, analyzing the relationship of student and school characteristics to both student achievement and nonresponse. The aim was to establish an effective procedure for creating weighting classes that could be used for future assessment cycles. We determined the appropriate variables and their relevant grouping and cut points through the use of conditional inference trees, creating the trees using recursive partitioning with inference-based splits and unbiased variable selection. We describe the approaches used and illustrate the procedures by showing some of our findings.