Much of the Census Bureau's budget is dedicated to personnel who collect data across the country. This survey collection effort is managed by 6 regional offices, who then divide into smaller sub-regional teams to manage Census Bureau field staff. The American Community Survey (ACS) is one such survey collected by field staff on an ongoing basis, orchestrated in monthly collection periods. In this paper, we use machine learning to better understand the effectiveness of ACS data collection. In particular, we seek to understand how ACS response rates are associated with demographic characteristics, and we investigate ways in which this information can be used to evaluate and supplement our data collection efforts. We start by merging ACS paradata with tract-level demographic data. Then, we use machine learning models to predict response rates by training on 2018 data and evaluating on 2019 data. We conclude by offering some insights on how this information can be used to evaluate and support ACS data collection.