Exploratory and Inferential Methods for Massive Data

• Activity Number: 619
• Type: Invited
• Date/Time: Thursday, August 8, 2013 : 8:30 AM to 10:20 AM
• Sponsor: Statistical and Applied Mathematical Sciences Institute
• Author(s): Naomi S. Altman*+ and Wei Luo and Garvesh Raskutti
• Companies: Pennsylvania State University and Pennsylvania State University and SAMSI
• Keywords: singular value decomposition ; robust principal components ; dimension reduction ; feature selection ; maximum likelihood ; multiple testing

Abstract

The availability of massive datasets enhance our ability to detect patterns and relationships. However, the volume of data make it more difficult to maintain quality checks, visualize patterns and fit complex models. Dimension reduction and feature selection methodologies are key to reducing the complexity and volume of data while suppressing noise and retaining informative dimensions and features. Standard methods are sensitive to outliers, assume that the data are elliptically distributed and are often computationally slow. In this talk we discuss new developments for dimension reduction and feature selection that are less sensitive to outliers and have less restrictive distributional assumptions. These methods provide maximum likelihood estimates in a general setting. We also discuss improvements to the computational algorithms that make implementation of these methods feasible for larger datasets.