Using Kullback-Leibler Divergence to Estimate the Regularized Discriminant Function

• Activity Number: 354
• Type: Contributed
• Date/Time: Tuesday, July 31, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Computing
• Author(s): John Beeson*+
• Keywords: RDA ; Kullback Leibler ; discriminant ; covariance ; learning ; regularized

Abstract

Regularized discriminant analysis, originally proposed by Friedman in 1989, is a well researched topic. It allows for more accurate classification in the presence of high dimensional, low sample size data. However, it has shortcomings in the manner to which it pools the covariance structures between classes if there exist classes that have drastically different structures. We overcome this limitation, by introducing a pre-processing step that allows for only the most structurally similar covariance matrices to be pooled. The advantage to this approach when contrasted to other methods in the literature, is it involves optimization versus cross-validation. Simulation results show that this method works well when compared to previous methods.