Using Paradata to Inform the Design of the 2017 Economic Census Electronic Reporting System (303575)

Keywords: Economic Census, Paradata

The U.S. Census Bureau is currently re-engineering the electronic data collection system that will be used to conduct the 2017 Economic Census, which collects detailed data from more than 4 million business establishments. This process has involved multiple rounds of qualitative research---respondent debriefings and usability interviews---to learn more about how respondents use the current reporting system and inform development of requirements for the new system. As these qualitative methods rely on small purposive samples, there is a risk that the findings may overlook some vital functionality. This risk is compounded by the fact that the design of the current system is not optimal, and, as a result, some respondents are not aware of the full suite of functions available and so are unable to report on their perceived utility during qualitative interviews.

To augment our qualitative research findings, we undertook an evaluation of paradata captured by the current electronic systems during data collection. While paradata are generally defined as process data associated with the survey process, and includes indicators such as response rates and contact outcomes, we focus on process data generated by self-administered electronic survey instruments. Examples include buttons clicked, pages viewed, and time spent on individual pages as well as over entire sessions. We analyzed these paradata to identify the relative usage of various functions and other types of relevant reporting behavior to supplement our findings from other stages of research. We plan to use our findings to refine the requirements for the new system. This presentation will discuss the methods we used to analyze electronic survey paradata and challenges we had to overcome. We integrate our findings with results from qualitative research and discuss revelations related to respondent behavior and the business survey response process that aid recommendations for instrument design.