Online Program Home
My Program

Abstract Details

Activity Number: 331 - Statistical and Practical Issues for Reproducible Molecular Prediction in Biomedical Studies
Type: Topic Contributed
Date/Time: Tuesday, July 31, 2018 : 10:30 AM to 12:20 PM
Sponsor: ENAR
Abstract #327284
Title: DREAM Challenges: a Community Framework for Benchmarking Reproducible Biomedical Models
Author(s): Justin Guinney*
Companies: Sage Bionetworks
Keywords: data challenges; prognostic; biomarker

The DREAM Challenges hosts crowd-sourced data competitions with an emphasis on model reproducibility. DREAM has developed cloud infrastructure to support the execution of models utilizing data that participants have no direct access. In recent Challenges, participants packaged algorithm(s) as a Docker container, allowing organizers to run models on participants' behalf for objective benchmarking. Models were rigorously assessed for predictive accuracy, and tested for statistical improvements over established benchmarks. A recent example is the Digital Mammography (DM) Challenge, were DREAM utilized 640k DM images contributed by Kaiser Permanente to benchmark deep-learning methods for cancer detection. Similarly, in the Multiple Myeloma (MM) Challenge, DREAM utilized proprietary and hidden data for the objective evaluation of prognostic models. The MM challenge asked participants to develop predictive models - utilizing clinical, genomic, and gene-expression data - to identify high-risk patients with newly diagnosed MM. The challenge resulted in several models that significantly outperformed existing public benchmarks for molecular-based prognostic models.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program