79 – Survey Design
Text Analysis: Further Work on Computer-Assisted Techniques
Bernard Dugoni, PhD
NORC at the University of Chicago
Dugoni and Brown (2014) presented initial work exploring a new three stage approach to developing code frames for open ended questions in survey research. This approach combines current approaches to software assisted content analysis with expert rater review and statistical, psychometric analysis to generate a code frame for the coding of verbatim (text) responses to open-ended survey items. This study applies the same approach to the analysis of longer verbal protocols by re-analyzing verbal protocols collected by Dugoni and Ilgen (1981). This set of archival data was constructed using structured interview techniques and classified into categories using a technique known as Q-Sort to develop the classification scheme prior to the development of the computer assisted techniques used by Dugoni and Brown. The use of the three-stage approach was found to be effective with the larger data sets yielding comparable results to earlier results with short responses to single items. The present study also supports the usefulness of this approach with archival data. Suggestions for modifications of the technique for archival re-analysis and structured protocols are discussed.