474 – New Advances in Modeling Survey Data
Evaluation of Multi-Class Classification Models for Census Mindsets
Mary H. Mulry
US Census Bureau
The U.S. Census Bureau fielded the 2020 Census Barriers, Attitudes and Motivators Study (CBAMS) sample survey in 2018 to collect data on attitudes and knowledge about the U.S. Census. Data from over 17,000 respondents was used to cluster individuals into one of six psychographic profiles referred to as Census “mindsets.� In social marketing campaigns, mindsets are constructed to reflect an individual’s knowledge, attitudes and opinions toward a topic. The mindsets are then used in developing messages with a call to action. In our case, the requested action is responding to the 2020 Census. We recently readministered the 2020 CBAMS to a sample selected from an online panel in anticipation of the 2020 Census, but a new classifier is needed to assign new respondents to existing mindsets. We compare several methods for fitting multi-class classification models to assign the mindsets and examine methods for model evaluation and selection. We include the evaluation method for multi-class classification models developed by Hand and Till (2001), which is a generalization of the area under the curve (AUC) for receiver operating curves (ROC) for binary classification models.