Classification with Known Class Probabilities

• Activity Number: 301
• Type: Contributed
• Date/Time: Tuesday, August 5, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Joshua Magarick*+
• Companies: University of Pennsylvania
• Keywords: Sequential Prediction ; Machine Learning ; Classification

Abstract

We investigate the problem of classification when the marginal distribution over classes is known but varies substantially from sample to sample. This will occur when the data are not independent. In particular, this situation arises when the classes are generated from a Markov process with a known stationary distribution and unknown other parameters. In real data, this phenomenon has been observed in studies of sleep, where the amount of time spent in different sleep stages is well known overall but variable across individuals and studies. We show how to use this additional information in several classifiers, such as logistic regression to mitigate the effect of inter-sample class proportion variability using both simulation studies and real data.