Survey Questionnaire Innovations for the National Assessment of Educational Progress: Noncognitive Modules and Questionnaire Indices (303645)

The core goals for education systems in the 21st century have shifted from teaching clearly defined knowledge and skills to promoting lifelong learners who are able and eager to face challenges of a truly global society. National and international large-scale assessments (LSA) have started to broaden their focus from achievement results as their only key outcome to also measure skills, strategies, attitudes, and behaviors that are distinct from content knowledge and academic skills, often called “noncognitive factors”. Measuring noncognitive factors in LSAs faces the challenges of how robust measurement approaches can be implemented while keeping student burden low, maximizing sub-group comparability of recorded responses, and mitigate growing concerns of students, parents, teachers, or school administrators about sensitivity and privacy around questions asked in school-based surveys. The National Assessment of Educational Progress (NAEP) administers survey questionnaires to students, teachers, and school administrators to put the assessment results into context and collect additional data relevant to educators and policymakers. This presentation will summarize current efforts for NAEP to develop a series of new noncognitive modules and highlight innovations in two key areas: (1) expanding the constructs measured in NAEP to include additional factors such as grit, desire for learning, and self-efficacy; and (2) creating Item Response Theory (IRT)-based questionnaire indices for enhanced reporting. We will give a preview of new constructs and questions in the 2017 NAEP survey questionnaires and will present lessons learned from a large nationally representative pilot study in 2016, drawing on response frequency data, factor analysis results, and response time data. The talk will conclude with a discussion of potential future directions and challenges in NAEP and LSAs more generally.