Medallion Lectures: A Journey to Ultrahigh Dimensional Space

• Activity Number: 149
• Type: Invited
• Date/Time: Monday, August 1, 2011 : 10:30 AM to 12:20 PM
• Sponsor: IMS
• Author(s): Jianqing Fan*+
• Companies: Princeton University
• Keywords: Variable selection ; high-dimensional data ; penalized method ; ISIS

Abstract

Ultrahigh-dimensionality characterizes many contemporary statistical problems from genomics and engineering to finance and economics. To visit such a vast dimension, large-scale and moderate-scale vehicles are needed for such a venture. We outline a unified framework to ultrahigh dimensional variable selection problems: Iterative applications of vast-scale screening followed by moderate-scale variable selection, called ISIS. The framework is widely applicable to many statistical contexts: from multiple regression, generalized linear models, survival analysis to machine learning and compress sensing. The fundamental building blocks are marginal variable screening and penalized likelihood methods. How high dimensionality can such methods handle? How large can false positive and negative be with marginal screening methods? What is the role of penalty functions? This talk will provide some fundamental insights into these problems. The focus will be on the sure screening property, false selection size, the model selection consistency and oracle properties. The advantages of using folded-concave over convex penalty will be clearly demonstrated. The methods will be illustrated.