Online Program

Estimating an ROC Curve: Models, Assumptions, and Interpretation

*Alicia Y Toledano, Biostatistics Consulting, LLC 

Keywords: Receiver operating characteristic curve, models, optimal operating point, minimally important change

Receiver operating characteristic (ROC) curves are used in diverse medical and epidemiological studies, including evaluating diagnostic performance of radiological imaging technologies, evaluating properties of biomarkers for risk factors and diseases, and determining minimally important change (MIC) in scores on questionnaires and other patient reported outcomes (PROs). We will present ROC methodology beginning with the underlying signal detection theory, to provide a unified framework for constructing and interpreting ROC curves based on ordered categorical data such as Likert scales, 0-100% rating scales, visual analog scales, and continuous data such as concentrations of biomarkers in patient samples. The empirical ROC plot and its relationship with logistic regression will be discussed; this plot is not a smooth curve. We will then discuss the binormal model for estimating a smooth ROC curve, focusing on model assumptions. This model applies in a variety of situations, and does not assume that the observed data actually follows a normal distribution. We will touch briefly on a proper binormal model that ensures concavity of the ROC curve, and what this model assumes about how human receivers use the information in the signal. Finally, we will discuss identification of the optimal operating point on an ROC curve, focusing on trade-offs in consequences of false positive and false negative results, correspondence with the value of the signal variable, and implications of using empirical or smooth ROC curves for this purpose. These methods are useful in, e.g., setting thresholds for positive biomarker-based medical tests and determining the MIC in multi-item questionnaires evaluating response to therapy. Examples using real-world study data will be shown throughout.