JSM 2014 Home
Online Program Home
My Program

Abstract Details

Activity Number: 643
Type: Contributed
Date/Time: Thursday, August 7, 2014 : 10:30 AM to 12:20 PM
Sponsor: International Chinese Statistical Association
Abstract #311682
Title: Quantifying the Probability That a Follow-Up Experiment Will Falsify a Scientific Claim
Author(s): Douglas Hayden*+ and Brian Healy and Mark Kon
Companies: Massachusetts General Hospital and Massachusetts General Hospital and Boston University
Keywords: Uncertainty ; Scientific Claims ; Falsification Probability ; Follow-Up Study
Abstract:

In medical research, p-values and confidence intervals are used for inference regarding scientific claims, but these quantities do not provide direct information of the probability that a claim will be falsified in a follow-up study. Recent studies have demonstrated that inferences based on published data often fail to replicate, but the falsification probability has received only limited attention. We derive a formal definition of the falsification probability, Pf. Since the expected value of Pf is the decision theoretic risk that a follow-up study will falsify an initial study, we use it to develop the operating characteristics of claims derived from hypothesis testing and confidence intervals. In the simple point null hypothesis setting when the null is rejected, Pf can be estimated using the power for a follow-up study of the same size. Maximum likelihood plug-in estimators of Pf can be computed using standard software. We recommend reporting the estimated Pf as a measure of the potential uncertainty when reporting a scientific claim.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2014 program




2014 JSM Online Program Home

For information, contact jsm@amstat.org or phone (888) 231-3473.

If you have questions about the Professional Development program, please contact the Education Department.

The views expressed here are those of the individual authors and not necessarily those of the JSM sponsors, their officers, or their staff.

ASA Meetings Department  •  732 North Washington Street, Alexandria, VA 22314  •  (703) 684-1221  •  meetings@amstat.org
Copyright © American Statistical Association.