Track Me, but Not Really: Tracking Undercoverage in Metered Data Collection

• Activity Number: 89 - Combining Survey and Digital Trace Data
• Type: Invited
• Date/Time: Monday, August 8, 2022 : 8:30 AM to 10:20 AM
• Sponsor: Social Statistics Section
• Author(s): Oriol J. Bosch* and Jouni Kuha and Patrick Sturgis
• Companies: The London School of Economics and Political Science and The London School of Economics and Political Science and The London School of Economics and Political Science
• Keywords: Passive data; Data quality; Undercoverage; Measurement error; Digital trace data; Survey research

Abstract

Metered data has been proposed as a useful way of measuring online behaviours, since it allows observation of web browsing unobtrusively and without relying on fallible self-reports. Metered data is generally collected from a sample of respondents who willingly install or configure, into their devices, a tracking technology that collects the traces left when people go online (e.g. URLs visited). To track the complete online behaviour of an individual, individuals must be tracked on all devices, browser and/or networks used to go online. Tracking only a subset of these can negatively affect data quality, producing potentially large biases in population estimates. Although little is known about this type of undercoverage, past research indicates that a high proportion of individuals participating in metered studies might be undercovered. To assess the impact of this type of undercoverage on the quality of metered data estimates, we combine metered survey and paradata in Spain, Portugal and Italy. Using simulations we show: 1) the prevalence and characteristics of undercoverage and 2) the extent and mechanisms in which undercoverage biases both univariate and bivariate estimates.