Assessing Measurement Error in an Energy Use Survey of Manufacturing Businesses

• Activity Number: 426
• Type: Contributed
• Date/Time: Tuesday, August 11, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Survey Research Methods Section
• Author(s): Kenneth Pick* and Tom Lorenz and Mary Susan Bucci and Leif E. Crider
• Companies: U.S. Census Bureau and U.S. Energy Information Administration and U.S. Census Bureau and U.S. Census Bureau
• Keywords: establishment surveys ; cognitive testing ; questionnaire development ; qualitative methodology

Abstract

The Manufacturing Energy Consumption Survey (MECS) is a national sample survey that collects energy-related building characteristics, energy consumption, and energy expenditures of manufacturing businesses in the United States every four years. The U.S. Energy Information Administration (EIA) sponsors the MECS and the U.S. Census Bureau collects the data. For the 2014 MECS, EIA proposed implementing a modified estimation model of energy use in the United States. In order to provide the data necessary to make the modifications to the estimation model, EIA developed seven questions specific to the flow of energy through asphalt plants and seven questions specific to the flow of energy through petrochemical plants. Cognitive testing with petrochemical plants and asphalt plants revealed not only potential measurement errors related to these seven questions, but also the existing paper questionnaire. This paper highlights the findings from cognitive testing and discusses their implications not only to the 2014 MECS, but also to questionnaire design best practices for business surveys. Specifically, we focus on lessons learned on using the correct terminology, appropriate questionnaire formatting, and being cognizant of question order effects. Additionally, the paper outlines the steps EIA and the U.S. Census Bureau are taking to reduce measurement error in the 2014 MECS and future data collection cycles.