Online Program Home
My Program

Abstract Details

Activity Number: 495 - The Potential for Web-Scraping in the Production of Official Statistics: An Opportunity for Statistics to Lead?
Type: Invited
Date/Time: Wednesday, August 1, 2018 : 10:30 AM to 12:20 PM
Sponsor: Government Statistics Section
Abstract #326661 Presentation
Title: Modernizing Government Statistics While Preserving Principles
Author(s): Robert Sivinski and Rochelle (Shelly) Wilkie Martinez*
Companies: Office of Management and Budget and Office of Management and Budget
Keywords: ethics; policy; big data; web-scraping

To meet the needs of the many stakeholders and policy-makers who depend on high quality, reliable federal statistical data, statistical agencies must take advantage of new technologies and data sources. Modernizing federal statistics by integrating new data sources such as satellite, sensor, social media, and other non-designed sources has the potential to increase the scope, coverage, and accuracy of statistical products while reducing costs and burden on respondents. Of course, as with any innovation there are risks and pitfalls associated with the use of non-designed data to generate federal statistics. In addition to methodological, technical, and skills challenges there are ethical and policy concerns around bias, transparency, informed consent, and data quality. This paper examines those challenges and possible solutions using current examples from federal agencies and through the structure of the four fundamental responsibilities of federal statistical agencies and recognized statistical units, as described in OMB's Statistical Policy Directive No. 1.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program