Online Program

Return to main conference page

All Times ET

Program is Subject to Change

Thursday, June 17
Thu, Jun 17, 1:30 PM - 3:30 PM
Creative Approaches to Questionnaire Development and Pretesting

Adapting Qualitative Research Techniques to Pretest Emerging Economic Topics (308010)

*Amy Anderson Riemer, U.S. Census Bureau 

Keywords: pretesting, cognitive interviews, qualitative research

The U.S. and worldwide economy is changing and evolving at an ever-increasing pace. In order to fully understand these changes and to better inform decision-making, economists and policy makers need timely and relevant statistics based on business data related to these emerging topics. As a major producer of official statistics about the economy, the U.S. Census Bureau is tasked with identifying effective and reliable ways to measure these topics in a way that manages respondent burden and produces high quality statistics. This paper will focus on how researchers at the U.S. Census Bureau have adapted their typical cognitive pretesting techniques to account for the challenges encountered with collecting data from businesses on a variety of emerging economic topics (e.g., artificial intelligence, robotics, telemedicine, factoryless goods, or management practices and performance indicators). In many instances, we are asking business respondents to provide information on subjects that are new and potentially ill defined. Gathering data on these topics may also require different sources of information or burden than establishment survey questions requesting traditional financial data. For example, developing metrics across the economy that capture robotics use and implementation can be challenging when businesses in different industries can have varied definitions for what is considered a robot. In order to effectively evaluate respondents’ understanding of these developing concepts, and the level of effort or burden that may be required to report related data, researchers have incorporated other qualitative research techniques, such as exploratory Early Stage Scoping interviews, extensive screening, and creating closer partnerships with subject matter specialists to supplement typical pretesting techniques. This paper will discuss our experiences with developing and testing survey questions about these new economic topics, along with sharing lessons learned.