The Role of Power Analysis in Large-Scale Statistical Inference

• Activity Number: 456
• Type: Contributed
• Date/Time: Wednesday, August 6, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Biometrics Section
• Author(s): Ruobin Gong*+ and Edoardo M. Airoldi
• Companies: Harvard and Harvard
• Keywords: statistical power ; large scale inference ; functional relationship detection ; gene set enrichment analysis

Abstract

With the acquisition of data becoming easier than ever, scientists are often faced with the problem of learning from large, high-dimensional datasets without much a-priori sense of direction. They seek for help from automated or semi-automated pattern recognition algorithms, which aim to provide a one-stop solution to this problem. Without substantial care to its design and modeling assumptions however, such algorithms may suffer low statistical power and do not make the most use of data. We propose a direct assessment framework on the efficacy of pattern recognition and large-scale inference algorithms under both the Frequentist and Bayesian approach of statical hypothesis testing and power analysis. We present two simulation experiments incorporating permutation-based methods for comparing competing sets of algorithms, which are designed for functional relationship detection and large-scale gene expression evaluation separately, analyze the effect of auxiliary variables on the power of each algorithm, and conclude that it is beneficial to utilize known information when constructing the testing metric and decision rules.