Abstract:
|
The Fragility Index (FI) has been introduced as tool to summarize the strength of evidence of a trial’s result. The FI is defined in trials with two equally sized arms, with a dichotomous or time-to-event outcome, and is calculated as the minimum number of conversions from non-event to event in the treatment group needed to shift the p-value from Fisher’s Exact Test over the 0.05 threshold. As the index lacks a well-defined probability motivation, its interpretation is challenging for consumers. We demonstrate with a Bayesian analysis that when the probability model holds, the FI inappropriately penalizes small trials for using fewer events than large trials to achieve the same significance level. Therefore, it can only be useful as a measure of robustness to violation of model assumptions. We illustrate shortcomings of the FI to do that and discuss the FI’s emphasis on significance in context of current debate over statistical significance. We argue that the FI does not promote statistical thinking and can mislead consumers, and that sensitivity analyses – including Monte Carlo and Bayesian approaches - are superior for quantifying and communicating robustness of results.
|