Assessing Differences of Estimates Across Partial Overlapped Surveys Under Limited Design Information with Modified T-Tests

• Activity Number: 250 - Weighting and Variance Estimation in Complex Samples
• Type: Contributed
• Date/Time: Tuesday, August 9, 2022 : 8:30 AM to 10:20 AM
• Sponsor: Survey Research Methods Section
• Author(s): Rong Wei* and Van Parsons and Katherine Irimata and Yulei He
• Companies: CDC and CDC and National Center for Health Statistics and National Center for Health Statistics
• Keywords: Two sample test; Partial overlapped sample; Design-based; On-line survey; US population health

Abstract

Population-based surveys may be designed with an intended overlap of sample subjects over different time points, but in practice, such surveys may have only partial overlap due to non-response and survey design changes over time. Public release of such data may only provide limited design-level information. Considering two such surveys, statistical testing for differences between estimated means is often based on a t-test using the survey design structures to properly define a variance estimator for the difference subject to a non-deterministic overlap of sample. For limited design information, two design-based t-tests are discussed. The methods identified some concerns related to data assumptions, both for the population and for sampling distributions. This study will present some details in aspects of variance estimators and make comparisons. Results from real survey data and simulated data will be presented.