Improved Statistical Inference via Nonlinear Data Transformation

• Activity Number: 225
• Type: Topic Contributed
• Date/Time: Monday, July 30, 2012 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Ann Lee*+
• Companies: Carnegie Mellon University
• Keywords: high-dimensional inference ; data mining ; kernel machine learning ; spectral graph theory

Abstract

Many estimation problems in astrophysics are highly complex, with high-dimensional, non-standard data objects (e.g., images, spectra, entire distributions, etc.) that are not amenable to formal statistical analysis. To utilize such data and make accurate inferences, it is crucial to transform the data into a simpler, reduced form. Spectral kernel methods are non-linear data transformation methods that efficiently reveal the underlying geometry of observable data. Here we focus on one particular technique: diffusion maps, or more generally, spectral connectivity analysis (SCA). We describe its novel use in high-dimensional regression and density estimation via adaptive bases, with applications in astronomy such as photometric redshift prediction and estimation of the evolution of galaxy morphology.

(Part of this work is joint with Peter Freeman, Rafael Izbicki, Jeffrey Newman, Joseph Richards, and Chad Schafer)