Keywords: microbiome, statistical learning, diversity, networks
Diversity is a marker of ecosystem health in ecology, microbiology and immunology, with implications for disease diagnosis and infection resistance. However, accurately comparing diversity across environmental gradients is challenging, especially when number of different taxonomic groups in the community is large. Furthermore, existing approaches to estimating diversity do not perform well when the taxonomic groups in the community interact via an ecological network. To address this, we propose DivNet, a method for estimating within- and between-community diversity in ecosystems where taxa interact via an ecological network. We show that accounting for network structure permits more accurate estimates of diversity, even in settings with a large number of taxa and a small number of samples. We illustrate the performance of the method and show how modeling interactions controls Type 1 error rate in hypothesis tests for changes in diversity.