This is the program for the 2010 Joint Statistical Meetings in Vancouver, British Columbia.

Abstract Details

Activity Number: 531
Type: Topic Contributed
Date/Time: Wednesday, August 4, 2010 : 10:30 AM to 12:20 PM
Sponsor: Section on Statistical Learning and Data Mining
Abstract - #308264
Title: Measuring Classifier Performance: A Penalty/Profit Perspective
Author(s): Kati Lentz and Chamont Wang*+ and Robert Stine
Companies: The College of New Jersey and University of Pennsylvania
Address: 2000 Pennington Road, Ewing, NJ, 08628-4700,
Keywords: Support Vector Machines ; Stochastic Gradient Boosting ; Neural Networks ; Regression ; Predictive Modeling ; Classifier Performance
Abstract:

In binary classification, the comparison of model performances is usually based on false positive, false negative, sensitivity, specificity and a host of other measures. In a series of papers, Professor David Hand (2009, 2008A, 2008B, 2007, 2006) discussed the difficulties of using these measures in binary prediction. Hand (2009) further proposed an H-measure for model comparison.

Wang and Zhuravlev (2009) and others, in contrast, used a profit/cost structure that avoids the difficulties raised in the Hand papers. In this study, we further break the outcome probabilities in four or more different categories to suit a specific application and then assign profit to each scenario. Furthermore, we use a variety of penalty functions to explore the consequences of the decisions. Our framework may be suitable for a variety of other applications that use classifiers in the modeling process.


The address information is for the authors that have a + after their name.
Authors who are presenting talks have a * after their name.

Back to the full JSM 2010 program




2010 JSM Online Program Home

For information, contact jsm@amstat.org or phone (888) 231-3473.

If you have questions about the Continuing Education program, please contact the Education Department.