Sparse and Low-Rank Models in Statistics: Theory and Applications

• Activity Number: 314
• Type: Invited
• Date/Time: Tuesday, July 31, 2012 : 10:30 AM to 12:20 PM
• Sponsor: IMS
• Author(s): Emmanuel Jean Candes*+
• Companies: Stanford University
• Keywords: convex programming ; high-dimensional statistics ; matrix completion ; sparse regression ; robust principal component analysis

Abstract

In statistics, it is now common to have fewer observations than variables. In biomedical imaging applications, for example, we have a small number of measurements and yet wish to reconstruct a high-resolution image. In data analysis, we may be interested in knowing users' preferences for a collection of items, but only observe their preferences for just a few (as in the Netflix prize).

We will survey recent results in the field of compressed sensing and sparse regression showing that it is possible to estimate accurately sparse high-dimensional vectors from a limited set of observations, emphasizing the role played by convex optimization. This phenomenon is not an isolated success story, as it is now well understood that one can recover other types of objects from what appear to be incomplete data. Important examples include low-rank matrices. We explain how these ideas have a bearing on robust estimation and especially on that of principal components or low-dimensional structure. All along, we shall discuss applications in computer vision, single molecule imaging, X-ray diffraction imaging, and other areas of science.