The Role of Active Learning in Sequential Decisionmaking

• Activity Number: 155
• Type: Topic Contributed
• Date/Time: Monday, July 30, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Daniel J Lizotte*+
• Companies: University of Waterloo
• Keywords: Dynamic treatment regimes ; Machine learning ; Active learning

Abstract

Existing methods for learning dynamic treatment regimes require that future decision-makers will have complete access to the covariates that are relevant for decision making. Real-world data, for example from clinical trials and electronic health records, often includes a large set of relevant and irrelevant features. Besides causing overfitting problems, failing to eliminate irrelevant features can result in decision support systems that are costly to implement: for each new patient, all features must be measured in order to make a decision, which in a clinical setting can be cost-prohibitive. We discuss potential methods for learning dynamic treatment regimes that recommend high-quality decisions without requiring that every feature of every future patient be collected. These methods are drawn from previous research on 'active learning' and 'budgeted learning.' We formalize the cost and benefit associated with feature measurement and construct initial strategies for actively measuring measuring covariates.