Pitfalls of Simplistic Normalization in Basic Science Experiments

• Activity Number: 187
• Type: Contributed
• Date/Time: Monday, August 2, 2010 : 10:30 AM to 12:20 PM
• Sponsor: ENAR
• Author(s): Tatsuki Koyama*+ and Zhiguo Zhao and Jeffrey Blume
• Companies: Vanderbilt University School of Medicine and Vanderbilt University School of Medicine and Vanderbilt University School of Medicine
• Keywords: fold change ; normalization ; type I error

Abstract

Basic science experiments often use a separate control group for each treatment group. Typically, the treatment group outcomes are scaled by the average of the corresponding control group outcomes. Despite its overwhelming popularity, this "fold change" method has serious statistical problems resulting in reduced validity and efficiency. When the implicit variability of the control group outcomes is ignored, a large type I error inflation can result. Likewise, this scaling induces correlation and can substantially inflate the type I error when this correlation is ignored (as it often is). We present simulations showing that this inflation results in type I error rates as high as 50% in everyday settings. We propose some computational and analytical approaches for dealing with this problem, and we present some practical recommendations for experimental designs with small sample sizes.