Accessing Diagnostic Accuracy with Ordinal Symptom Statuses Under the Absence of a Gold Standard

• Activity Number: 113
• Type: Topic Contributed
• Date/Time: Monday, August 1, 2011 : 8:30 AM to 10:20 AM
• Sponsor: Section on Health Policy Statistics
• Author(s): Zheyu Wang*+ and Xiao-Hua "Andrew" Zhou
• Companies: University of Washington and University of Washington
• Keywords: Diagnostic tests ; EM algorithm ; Volume under the ROC surface (VUS) ; Repeated ordinal data ; Random effects models ; Traditional Chinese Medicine (TCM)

Abstract

Two common problems in assessing the accuracy of diagnostic tests or performance of doctors in detecting a symptom are the unknown true symptom status and the ordinal-scale of the symptom status. This is especially true for the studies on traditional Chinese medicine. In this paper, we proposed a nonparametric maximum likelihood method for estimating the accuracy of different doctors in detecting a symptom with an ordered multiple-class and without a gold standard. In addition, we extended the concept of the area under the ROC curve (AUC) to a hyper-dimensional overall accuracy measure and provided alternative graphs for displaying a visual result. The simulation studies showed that the proposed method has good performance in terms of bias and mean squared error. Finally, we applied our method to our motivating example on assessing the diagnostic abilities of five Chinese medicine doctors in detecting symptoms related to Chills disease. In addition, we discussed further on how to incorporate a dependence structure into the model under existence of a patient-level random effect. An ad-hoc test of the model fitting and a likelihood ratio test on the random effect were also provided.