Adapting and Improving Methods to Manage Cognitive Pretesting of Multilingual Survey Instruments
Barbara Lazirko
U.S. Census Bureau
Yuling Pan
U.S. Census Bureau
M. Mandy Sha
RTI International
Current methods to manage cognitive pretesting of multilingual survey materials and instruments are based on well documented lessons learned. Yet, they did not follow a systematic approach because translation pretesting was still at its early development stage. This paper examines the application of the methods in the current literature to a large-scale translation pretesting project, and recommends improvements and areas for future research. The U.S. Census Bureau pretested the Chinese and Korean translation of the American Community Survey (ACS) Language Assistance Guide (LAG), which included a variety of housing and population questions, such as mortgage, health, education, and ancestry. A total of 258 respondents who were 129 Chinese and 129 Korean speakers with limited English language proficiency participated in cognitive interviews. The large number of respondents was selected from completed screener records after contacting more than 1,000 candidates in three states. By following a systematic process and guided by sociolinguistic research framework, we were successful in adapting the methodology in the current literature, particularly with using cognitive interviews to identify translation problems at the pragmatic level that cannot be revealed in traditional translation review process. We also found implementing the following steps to be advantageous: conducting a systematic translation review process prior to cognitive testing, tailoring respondent recruitment methods to recruitment criteria, and implementing an interview lifecycle process to control and monitor the progress of interviewing.
