Comparison of Simple and Complex Predictive Models Applied to the National Surveys on Drug Use and Health

• Activity Number: 178 - Novel Applications and Extensions of Dimension Reduction Methods
• Type: Contributed
• Date/Time: Monday, July 29, 2019 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Georgiy Bobashev* and Emily Hadley
• Companies: Research Triangle Institute and RTI International
• Keywords: Model complexity; Effective degrees of freedom; neural networks; random forest; regression

Abstract

Repeated cross-sectional national surveys provide rich resources to calibrate and validate predictive models. We examined how well demographics, substance use, mental and other health indicators from multiple years of National Surveys on Drug Use and Health (NSDUH) predict individual propensity for multiple visits (< 3 per year) to Emergency Departments (ED). We considered a range of years from 2007 to 2015. Models were developed on some years’ data, validated and tested on the others. We compared performance of the backwards, stepwise logistic regressions, LASSO, classification trees, combination of trees and regressions, neural networks and random forest models. We assessed the complexity of the models by calculating Effective Degrees of Freedom (EDF). We explain similarity of performance of complex models with simpler regression models by similarity in EDF.