High-Dimensional Discrimination with Trace Regularization

• Activity Number: 506 - Advances in Multivariate Analysis for High-Dimensional, Complex Data Problems
• Type: Topic Contributed
• Date/Time: Wednesday, August 1, 2018 : 10:30 AM to 12:20 PM
• Sponsor: Korean International Statistical Society
• Author(s): Jeongyoun Ahn* and Yongho Jeon and Hee Cheol Chung
• Companies: University of Georgia and Yonsei University and University of Georgia
• Keywords: Discriminant subspace; Linear Discriminant Analysis; Orthogonal Iterations; Indefinite Matrix; Sparse Estimation

Abstract

Fisher's original idea on linear discriminant analysis (LDA) is to find a low-dimensional subspace that yields the largest between-class scatter and the smallest within-class scatter. We note that a solution to this optimization is not provided by the classical LDA as commonly discussed in the literature. In this work we deal with Fisher's original optimization problem and develop a discrimination method for high dimensional multi-class discrimination problem. The proposed DTR (Discrimination with Trace Regularization) identifies a subspace in which the between-class scatter is maximized in terms of its trace while the trace of the within-class scatter is controlled at a specified level. Theoretical investigation on the trace regularization reveals some unexpected aspects of the scatter controls, as well as interactive relationships among the traces of scatters. An interesting relationship with existing methods such as maximal data piling and ridge-corrected LDA is also discussed. Empirical examples suggest that the proposed method works competitively with or better than existing approaches in a wide range of problems, in terms of variable selectivity and classification accuracy.