Evaluation of agricultural or pharmaceutical compound safety requires animal studies in multiple species under detailed international guidelines. Statistical false positives arise from the large number of variables examined. Ad hoc methods may not control false positives, but standard frequentist multiplicity adjustment methods do. We show by example they can bring "statistical significance" close to a comprehensive assessment of "biological significance," including flagging real effects.
Consistency imposed by regulatory guidelines make historical information relatively reliable. The Bayesian paradigm is a powerful procedure generator to gain sensitivity. Westfall and Soper (JASA 2001) apply it to carcinogenicity studies. For each tumor type, the total count over all doses is not used by Peto's test; comparison with the expected count from historical control data can be used to focus power on tumor types most likely to show a true treatment effect. Using Monte Carlo on control populations, we compare operating characteristics of competing methods and show the Bayesian paradigm yields greater power, even compared to methods that do not control false positives.
|