Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 306 - SPEED: SPAAC SESSION II
Type: Topic-Contributed
Date/Time: Wednesday, August 11, 2021 : 3:30 PM to 5:20 PM
Sponsor: Biometrics Section
Abstract #317740
Title: Utilizing Stability Criteria in Choosing Feature Selection Methods Yields Reproducible Results in Microbiome Data
Author(s): Lingjing Jiang*
Companies: Johnson & Johnson
Keywords: classification; prediction; reproducible; feature selection; microbiome; stability
Abstract:

Feature selection is indispensable in microbiome data analysis, but it can be particularly challenging as microbiome data sets are high-dimensional, underdetermined, sparse and compositional. Great efforts have been made on developing new methods for feature selection that handle the above data characteristics, but almost all methods were evaluated based on performance of model predictions. However, little attention has been paid to address a fundamental question: how appropriate are those evaluation criteria? Most feature selection methods often control the model fit, but the ability to identify meaningful subsets of features cannot be evaluated simply based on the prediction accuracy. This crucial need of identifying relevant and reproducible features motivated the reproducibility evaluation criterion such as Stability. We compare the performance of popular model prediction metrics (MSE or AUC) with proposed reproducibility criterion Stability in evaluating four used feature selection methods in both simulations and experimental microbiome applications. We conclude that Stability is a preferred feature selection criterion over model prediction metrics.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2021 program