We in the clinical trials community have an urgent need to develop and test models to identify site errors and cases of fraud within trials. Statisticians have already provided a wide array of statistical summaries and models that may be useful in this central statistical monitoring, but it is unclear which ones will meet the needs of a typical trial. Since site errors are heterogeneous in nature arising from distinct causes, it is likely that different types of errors will have different important predictors. One needs to approach such questions with scientific focus and operationalized definitions, thereby developing a PICOT statement for the study of central statistical monitoring. Our population (P) is the sites within a multi-center trial, rather than the patients, which has important implications for power in these analyses. The Exposures [rather than intervention, (I) and control (C), as for any epidemiology study] are the factors that we believe to be causes or indicators of site errors. The errors themselves are our outcomes (O) and these need to be clearly defined, both in terms of type and importance, as not all errors are equally bad. Rather than concerning ourselves with any possible errors, significant improvement in trial quality will be made by focusing our efforts on the prediction of those errors that bias trial results or expose participants to unnecessary risk. The timing (T) of our central statistical monitoring needs to be occur while the trial is still being conducted, or ideally before it even begins, as prevention is our ultimate goal.

There has been a lot of work published on predicting fraud through statistical means (Evans, Fraud and Misconduct in Medical Research 1996; Buyse, Stat Med 1999; Taylor, Drug Inform J 2002; Al-Marzouki, BMJ 2005; Carlisle, Anaesthesia 2012; Venet, Clin Trials 2012; Kirkwood, Clin Trials 2013). Although cases of fraud are thought to be rare, this behavior is considered to be particularly egregious and central statistical monitoring appears to be the optimal means of identifying this. Based on a set of fabricated data discovered during a clinical trial, we developed an easy to use fraud risk score (Pogue, Clin Trials 2013; Pogue, Clin Trials 2104). The development and validation of this risk scoring system will be described.

It is the prediction of unintentional errors (foibles) that is the next challenge we face. Previous attempts have tended to lump all error types together, likely reducing the power to detect important predictors. The focus of predicting procedural errors has been concentrated on the detection of outlier data at a site. In a retrospective analysis we attempted to define a distance measurement on baseline characteristics of the patients enrolled at each site with a multi-center trial and use this to identify sites who had made inclusion/exclusion protocol violations (Pogue, SCT presentation 2014). We anticipated that the sites who made these errors would be outliers and have larger distance measurements compared to sites without errors. Paradoxically, we found the opposite was true and that this robust effect can be explained through other site-level variables. We conclude that as useful as these central statistical monitoring statistics are, there is a need to relate them back to known site characteristics in order to identify errors in real time, or ideally prevent them from occurring in the first place. Future directions will be discussed.