Keywords: expected utility, risk threshold, patient preference, operating point, benefit-risk trade-off, decision analysis
A diagnostic test to rule-in or rule-out disease either now or in the future is useful for determining or at least informing on the appropriate clinical action among available options. For example, in pregnant women with symptoms of pre-term labor, a negative fetal fibronectin (fFN) test result is used to indicate that spontaneous pre-term delivery is unlikely, that is, can be ruled-out, which can be the basis for deciding that hospital admission, corticosteroid administration, or tocolytic therapy is unnecessary.
To evaluate rule-in and rule-out claims for diagnostic tests, traditional measures of classification accuracy (specificity, sensitivity, AUC) or predictive accuracy (NPV, PPV) can be inadequate because they do not consider the clinical consequences of the test results. Unfortunately, randomized trials for evaluating the clinical consequences of the test in treatment decision making can be prohibitive to conduct because of cost, long duration, large sample size, etc. To evaluate the clinical utility of a test for ruling in or ruling out disease based on a cross-sectional or other clinical performance study, we consider descriptive, graphical, and decision-theoretic summaries, specifically, diagnostic yield, mean risk stratification, predictiveness curve, weighted accuracy, relative net benefit, and relative utility. The decision-theoretic summaries consider the relative importance k of a false positive to a false negative test error in the context in which the test is used.