Dispersion Estimation and Its Effect on Test Performance in RNA-Seq Data Analysis

• Activity Number: 194
• Type: Contributed
• Date/Time: Monday, August 5, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Biometrics Section
• Author(s): William Landau*+ and Peng Liu
• Companies: Iowa State University and Iowa State University
• Keywords: RNA Sequencing ; RNA-Seq ; genomics ; differential expression ; dispersion

Abstract

A central goal of RNA sequencing (RNA-seq) experiments is to detect differentially expressed genes. In the ubiquitous negative binomial model, each gene is given a dispersion parameter, which measures the overdispersion of the gene counts relative to a Poisson distribution. Underestimation of the dispersions leads to false discovery, while overestimation lowers the rate of true detection. No single method of estimating gene-wise dispersions is standard, but several methods exist. This article describes a simulation study that compares these dispersion estimation methods in terms of point estimation quality and effect on the performance of tests for differential expression. Overall, the mean squared error-best methods are the ones that set the dispersions to a fitted trend. The dispersions estimated independently for each gene have the strongest linear relationships with the true dispersions. The methods that maximize the performance of tests for differential expression are the ones that use a moderate degree of dispersion shrinkage, regardless of whether this shrinkage is toward a common value, trend, or prior distribution.