Sensitivity Testing: Issues and Solutions

• Activity Number: 85 - Machine Learning in Biomedical Data
• Type: Contributed
• Date/Time: Sunday, July 28, 2019 : 4:00 PM to 5:50 PM
• Sponsor: ENAR
• Author(s): David H. Collins* and Kimberly Kaufeld and Michael S. Hamada and Richard Warr
• Companies: Los Alamos National Laboratory and Los Alamos National Laboratory and Los Alamos National Laboratory and Brigham Young University
• Keywords: Sensitivity test; Binary regression; Logistic; Probit; Heteroscedasticity

Abstract

Sensitivity testing using binary regression is used in a wide range of applications including bioassay, explosives testing, market research, political science, and predictive analytics. This works well under appropriate assumptions, but there are many cases where various issues invalidate test results or reduce the accuracy and precision of numerical estimates. Engineers, and even statisticians, may be unaware of the need for more sophisticated analysis to get better results from the tests. This talk discusses several common issues that affect estimates derived from sensitivity tests, and presents diagnostics and corrective measures. Issues include heteroscedasticity, complete or quasi-complete separation, and poor model fit. We also emphasize the value of using continuous data, if available, rather than dichotomizing to a binary outcome.