Effects of Employment Classifications of Conceptual Variability of Response Category Options—Implications for Data Quality
John Bosley
Bureau of Labor Statistics
Scott Fricker
Bureau of Labor Statistics
Dan Gillman
Bureau of Labor Statistics
Close-ended survey response options need to be exhaustive, mutually exclusive, and well understood by respondents, but sometimes differential conceptual complexity in the response categories can make response choice difficult and reduce data quality. The present study demonstrated this effect by examining respondent classification decisions in a class-of-worker (COW) question that asks respondents to select one of four employment categories: government, private company, non-profit organization, or self employed. Study participants (n=90) read a series of narrative vignettes describing fictional employment situations, and then classified each worker using two different groupings of the COW classification. Vignettes contained cues indicative of the job's membership in just one of the first three categories, but no clear indications about self-employment. One-half of the sample made choices from among the entire set of four response options, while in the other half-sample respondents were presented with only the "self-employed" option as a "yes-or-no" choice. The vignettes were presented a second time to both groups, and everyone classified the same jobs into just three employment categories government, private or non-profit. The data show that the stability and accuracy of respondents' answers were highly dependent on the set of response options provided. We focus particular attention on the self employment class of work, which appears conceptually distinct from the other three classes, and for which we observed the highest number of classification errors. We also examine the impact of including conceptually variable response categories in the set of response options of a single close-ended question. The results of this study are discussed in the context of cognitive theories of concept formation and categorization, along with broader implications for questionnaire designers.
