Effect of Form Design on the Reporting of Complex Concepts in a Survey of Businesses: Evidence from the Business R&D and Innovation Survey

• Activity Number: 221
• Type: Topic Contributed
• Date/Time: Monday, August 1, 2011 : 2:00 PM to 3:50 PM
• Sponsor: Section on Government Statistics
• Author(s): Brandon Shackelford*+ and Susan Helig
• Companies: Twin Ravins Consulting and U.S. Census Bureau
• Keywords: establishment survey ; confirmation bias ; redesign ; measurement error ; form design

Abstract

This paper examines the impact of form design changes made to the Business R&D and Innovation Survey (BRDIS) for the 2009 survey cycle that were aimed at reducing measurement error in certain key survey items. Although a controlled experiment was not designed to test the impact of these changes, we attempt to isolate effects of changes to the form by exploiting characteristics of the BRDIS sample design. Changes in reporting patterns between the 2008 and 2009 survey cycles are significant and the direction of the changes are contrary to what would be expected during a period of economic contraction. We also find that the form design changes appear to have more of an impact on companies receiving the survey for the first time compared to those with established reporting patterns from prior year surveys.