Confidence Intervals for the Performance of a Sequence of Diagnostic Tests

• Activity Number: 85 - Machine Learning in Biomedical Data
• Type: Contributed
• Date/Time: Sunday, July 28, 2019 : 4:00 PM to 5:50 PM
• Sponsor: ENAR
• Author(s): Beau Nunnally and Christine Schubert Kabban*
• Companies: Air Force Institute of Technology and Air Force Institute of Technology
• Keywords: Test Sequence; Believe the Positive; Believe the Negative; Bayes Cost; Youden's Index; Confidence Interval

Abstract

Previous work has described the usefulness of a sequence of tests for diagnosis, including the order and number of tests in a sequence as well as methods to examine sequence thresholds. However, inference for the performance of the sequence itself has not yet been accomplished. In this work, we present methods to compute confidence intervals (CIs) for the performance of a sequence of two diagnostic tests. We consider two test sequences: Believe the Positive and Believe the Negative. Both Delta and Generalized methods are derived for CI estimation of sequence-specific, optimal point criterion based upon minimizing diagnostic misclassifications. Simulation results examine the effects of test accuracy, correlation, and biomarker characteristics on CI coverage, and associated Youden Index values are provided. Sample size estimates to maintain CI coverage ranged about 100-250 for the Generalized method in all simulation scenarios and for the Delta method in specific scenarios. Only the Generalized method achieved coverage in all scenarios. An application using NHANES data demonstrates the usefulness of these CIs for computation and comparison of test sequence performance.