Cross-Cultural Web Probing and How it Can Enhance Equivalence in Cross-Cultural Studies (303140)

Keywords: cross-cultural surveys, cross-national surveys, equivalence, testing, probing, mixed-method

Cross-national surveys are burgeoning, and so are cross-cultural surveys in general, whether conducted within a country or across borders. It is therefore more pressing than ever to have a sound methodology at one’s disposal that allows a researcher to produce and establish equivalent data. On the one hand, equivalence needs to be assessed after data collection and prior to any substantive analysis to guard against methodological artefacts. Statistical data analysis methods are a key to this step. On the other hand, equivalence needs to be assured in the process of developing a survey. Assuring an equivalent measurement instrument, the focus of this presentation, greatly hinges on the chosen questionnaire development approach and on suitable translation and translation assessment methods. Cross-cultural cognitive interviewing, used to enhance comparability in an intercultural study, is a relatively recent addition to the tool box of a cross-cultural survey methodologist. Nevertheless, it has already become an indispensable part of questionnaire development; with some constraints, it can equally be implemented once the actual translation and translation assessment phase has started. Even more recent is cross-cultural web probing, that is, the simultaneous implementation of cognitive probes in web surveys in several countries. Cross-cultural web probing has been developed by Braun et al. to collect qualitative data from a large number of intercultural respondents in a standardized and efficient way. It therefore tackles the issues of (potentially) low same size, (potential) lack of standardization and (potential) lack of trained cognitive interviewers that may be associated with standard face-to-face cognitive interviewing. This presentation focuses on the use of web probing in the cross-national context. It will present (1) the origins of cross-cultural web probing; (2) the actual web implementation, including do’s and don’ts; (3) a selection of substantive results; (4) a discussion on best use in a multi-method questionnaire design process; and (5) further uses of the method.