High-Dimensional Linear Model Stability and Robustness in Large Database Applications (303072)

Keywords: linear models, high dimensional data analysis, mis-specification, robustness

The development of large databases as a statistical research setting is an area of great research interest. Linear models in the setting of large genomic databases and related high dimensional analysis require careful application when the number of variables (p) exceeds the number of subjects (n). The resulting predictive models, even with sparsity restrictions imposed, are often non-robust. These issues are examined here using detailed numerical studies. In particular nonlinearity, model mis-specification and issues related to Simpson’s paradox affect the stability of related statistical methods and search algorithms.