Measuring Facebook Use: The Accuracy of Self-Reported Data Versus Digital Trace Data

• Activity Number: 89 - Combining Survey and Digital Trace Data
• Type: Invited
• Date/Time: Monday, August 8, 2022 : 8:30 AM to 10:20 AM
• Sponsor: Social Statistics Section
• Author(s): Paulina Pankowska* and Florian Keusch and Ruben Bach and Alexandru Cernat
• Companies: Vrije Universiteit Amsterdam and University of Mannheim and University of Mannheim and University of Manchester
• Keywords: Measurement error; Social media use; Digital trace data; Survey data; Data quality

Abstract

Given the important role social media plays in today’s society, its use and effects on various outcomes such as mental health or voting behavior have been extensively studied by researchers across different disciplines. The majority of studies to date use self-reported measures of social media use, the validity of which is largely unassessed. While the increased availability of digital trace data has prompted some researchers to validate/benchmark their survey-based measures using log-based ones, these studies rely on the assumption that the log-data is error-free. Thus, any inconsistencies between the measures are attributed to the presence of (measurement) error in the survey data. However, log-data is also often subject to non-negligible error. In our paper, we assess the quality of measures of Facebook use coming from a survey, a tracking app, and Facebook data files. Having data for the same sample from three different sources allows us to model error in each of them using LCA. This enables us to compare the incidence and nature of the error in each of the sources and establish whether measures based on log-data are truly superior in quality to those based on surveys.