|
Abstract:
|
The DREAM Challenges hosts crowd-sourced data competitions with an emphasis on model reproducibility. DREAM has developed cloud infrastructure to support the execution of models utilizing data that participants have no direct access. In recent Challenges, participants packaged algorithm(s) as a Docker container, allowing organizers to run models on participants' behalf for objective benchmarking. Models were rigorously assessed for predictive accuracy, and tested for statistical improvements over established benchmarks. A recent example is the Digital Mammography (DM) Challenge, were DREAM utilized 640k DM images contributed by Kaiser Permanente to benchmark deep-learning methods for cancer detection. Similarly, in the Multiple Myeloma (MM) Challenge, DREAM utilized proprietary and hidden data for the objective evaluation of prognostic models. The MM challenge asked participants to develop predictive models - utilizing clinical, genomic, and gene-expression data - to identify high-risk patients with newly diagnosed MM. The challenge resulted in several models that significantly outperformed existing public benchmarks for molecular-based prognostic models.
|
