Estimating a Misclassification Probability in a Binomial Setting

• Activity Number: 243
• Type: Contributed
• Date/Time: Monday, August 2, 2010 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistics and the Environment
• Author(s): Patrick Lennon Kendall Zimmerman*+ and Greg C. Liknes
• Companies: University of Minnesota, Twin Cities and U.S. Forest Service
• Keywords: misclassification ; remote sensing ; classification

Abstract

When an interpreter observes and classifies items of two classes, but has a positive probability of misclassifying items, this misclassification can bias proportion estimates. For the case when the true class of items is unknown, we present a maximum-likelihood estimator of misclassification probability based on agreement between two interpreters. The estimator requires the misclassification probabilities of both interpreters to be equal, but simulation suggests that, for unequal probabilities, the estimator behaves as if it were estimating their average. Also, the estimator requires the correlation of the correctness of the interpreters' classifications to be known, which is unrealistic. We discuss a strategy for estimating an interval within which a misclassification probability lies, and argue that the estimator can produce a useful lower bound in practice.