Estimating accuracy of a diagnostic test with verification bias when the number of observed false negatives is small: a Monte Carlo approach

• Activity Number: 534
• Type: Contributed
• Date/Time: Wednesday, August 4, 2010 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Computing
• Author(s): Li Deng*+
• Companies: New England College of Optometry
• Keywords: Sensitivity ; Specificity ; Verification bias ; Two-stage sampling ; Small sample size

Abstract

The verification bias in a validation study for a new diagnostic procedure presents a statistical challenging. Adjusted estimates for sensitivity and specificity and their asymptotic variances have been developed previously. Simulation studies showed that the ROC curve constructed using the asymptotic variance estimation could produce misleading results when the number of verified non-diseased subjects is too small (n<5). In this paper, we propose a new approach to estimate the variability of adjusted sensitivity and specificity in attempt to address the small sample issue. This model allows a more complete count of the sampling errors. Our simulation results show considerable improvement in the coverage probability for sensitivity with the new confidence intervals when compared to the asymptotic ones especially in the case when the number of non-diseased subjects is small.