Assessing Cognitive Functioning in Multiple Modes: The CogUSA Study (303641)

Keywords: cognition, cognitive testing, mode effects, web, telephone

The CogUSA study collected longitudinal cognitive test data from a random sample of 1,514 community-dwelling adults born in 1956 or earlier (i.e., age 51+) and their significant others, regardless of age, beginning in 2007. Over the course of 7 waves, cognitive tests were administered using telephone, face-to-face, and web modes. An important aim of waves 4 and 5 of the CogUSA study was to adapt cognitive tests to the web, both to learn about the effectiveness of conducting cognitive tests on the web and how measures of cognitive ability from web tests compare to measures from interviewer-administered tests over the telephone. A number of tests including number series, verbal analogies (both adapted from the Woodcock-Johnson III), and a Red-Green Task-switching test (adapted from the Brief test of Adult Cognition by Telephone) were administered to respondents in both telephone and web modes in a randomized order in waves 4 and 5 of the study, allowing for the identification of mode and practice effects. The median time between waves 4 and 5 was 18 days.

All the tests showed a mode effect, with higher mean respondent cognitive test scores on the web versions than on the telephone. Educational attainment and higher performance on similar cognitive tests in earlier face-to-face administration were predictive of smaller mode effects. While the observed mode effects may be in part unique to the specific implementation of these particular cognitive tests, they are also consistent with prior research suggesting basic differences between interviewer- and self-administered modes. The current findings suggest that researchers should be cautious in combining results from cognitive tests administered on the web and by telephone, and that calibration across modes by the application of a scalar or constant may be insufficient to achieve comparability.