Randomized clinical trials are the gold standard for evaluating the benefits and risks of interventions. Diagnostic studies with appropriate reference standards are the quintessential model for evaluating diagnostic classification accuracy. However these studies often fail to provide the necessary evidence to inform practical medical decision-making. The important implications of these deficiencies are largely absent from discourse in medical research communities.
Typical analyses of clinical trials involve intervention comparisons for each efficacy and safety outcome. Outcome-specific effects are tabulated and potentially systematically or unsystematically combined in benefit:risk analyses with the belief that such analyses inform the totality of effects on patients. However such approaches do not incorporate associations between outcomes of interest, suffer from competing risk challenges, and since efficacy and safety analyses are conducted on different analysis populations, the population to which these benefit:risk analyses apply, is unclear.
This deficit can be remedied with more thoughtful benefit:risk evaluation with a pragmatic focus in future clinical trials. Critical components of this vision include: (i) using outcomes to analyze patients rather than patients to analyze outcomes, (ii) incorporating patient values, and (iii) evaluating personalized effects. Crucial to this approach entails improved understanding of how to analyze one patient before analyzing many. Newly developed approaches to the design and analyses of trials such as partial credit and the desirability of outcome ranking (DOOR), are being implemented to more optimally inform patient treatment.
Standard evaluation of diagnostic studies consists of estimating sensitivity, specificity, positive/negative predictive values and likelihood ratios, and overall accuracy. Although useful, these statistics have limited utility for guiding clinical decision-making and do not convey the impact of clinical application. Evaluation of diagnostic utility depends on prevalence and the relative importance of potential errors (false positive vs. false negative). Benefit-Risk Evaluation of Diagnostics: A Framework (BED-FRAME) is a strategy for pragmatic evaluation of diagnostics designed to supplement traditional approaches. BED-FRAME introduces weighted accuracy and diagnostic yield, addressing 2 key issues: (1) that diagnostic yield depends on prevalence, and (2) that different diagnostic errors carry different clinical consequences, providing a tool for communicating the expected clinical impact of diagnostic application and the expected tradeoffs of diagnostic alternatives.
Bio: Dr. Scott Evans, Harvard University is the Director of the Statistical and Data Management Center (SDMC) for the Antibacterial Resistance Leadership Group (ARLG). His interests include the design, monitoring, analyses, and reporting of and education in clinical trials. He is the author of more than 100 peer-reviewed publications and three textbooks on clinical trials including Fundamentals for New Clinical Trialists. Dr. Evans is a member of the Board of Directors for the American Statistical Association and the Society for Clinical Trials, and a former Director for Mu Sigma Rho (the National Honorary Society for Statistics). He is a member of an FDA Advisory Committee, the Steering Committee of the Clinical Trials Transformation Initiative (CTTI), and numerous DMCs. He is the Chair of the Trial of Year Committee for the Society for Clinical Trials and Chair of Q-SPIs DMC Benefit:risk Working Group. He is the Editor-in-Chief of CHANCE and Statistical Communications in Infectious Diseases (SCID).
Dr. Evans is a Visiting Professor at the Department of Medical Statistics at Osaka University in Japan. He is the Past-President of the Boston Chapter of the American Statistical Association (ASA), the Past-Chair of the Development Committee of ASA, the Past-Chair of the Teaching Statistics in the Health Sciences section of ASA, and the Past-Chair of the Statistics in Sports section of ASA.
He has taught numerous short courses on topics in clinical trials as professional meetings such as the JSM, SCT, BASS, the Deming Conference on Applied Statistics, the FDA-Industry Statistics Workshop, the DIA/FDA Statistics Forum, the Graybill Conference, and at the FDA (CDER and CDRH). He’s delivered keynote addresses at the Midwest Biopharmaceutical Statistics Workshop and the Multidisciplinary Symposium on Non-Inferiority Trials in Brazil, and invited addresses to the Japanese Biometric Society, Japanese National Institute of Public Health, and the Statistical Society of Australia. He’s received the prestigious Mosteller Statistician of the Year Award, the Robert Zackin Distinguished Collaborative Statistician Award, and is a Fellow of the ASA.