Probability-Enhanced Sufficient Dimension Reduction for Binary Classification

• Activity Number: 367
• Type: Contributed
• Date/Time: Tuesday, August 6, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Seung Jun Shin*+ and Yichao Wu and Hao Helen Zhang and Yufeng Liu
• Companies: North Carolina State University and NC State University and North Carolina State University and The University of North Carolina
• Keywords: binary classification ; conditional class probability ; Fisher consistency ; sufficient dimension reduction ; weighted support vector machines

Abstract

Sufficient dimension reduction (SDR) arises to reduce the data dimensionality without loss of information. Although many successful SDR methods have been developed, most existing methods suffer in binary classification. For example, sliced inverse regression can estimate at most one direction with binary response. We develop a probability-enhanced SDR for binary classification. The key idea is to slice data based on the conditional class probability rather than the binary response. We first show that the central subspace based on the conditional class probability is the same as that based on the binary response. This result justifies the proposed slicing scheme from a theoretical perspective and assures no information loss. In practice, the true conditional class probability is generally not available, and its estimation can be challenging for high-dimensional data. We show that to implement the new slicing scheme, one does not need exact probability values, but their relative order. Motivated by this, our new SDR procedure bypasses probability estimation, and estimates the order of probability values directly via Fisher consistency of the weighted support vector machine.