403 – Analytics, Attrition, and Words
Code-frame Development for Verbatim Responses Using a 3-stage Approach
Bernard L. Dugoni
NORC at the University of Chicago
Kevin Brown
NORC at the University of Chicago
In a study of the priorities of educators in the midwest, a three step process was used to assess open-ended responses, combining the use of text-analysis software with researcher review techniques for a tailored solution that is computer assisted but retains the human element. The first stage used the software "QDAMiner" developed by Provalis Research to conduct content analysis based on word frequency. In the second stage, related words were combined into phrases by using "keyword-in-context" analysis, which assesses whether the same words or phrases share an single concept or mean different things to different respondents. At this stage, responses were combined by the program into a matrix of co-occurrence, which allows the analysis of cases where two or more of the keywords are used in a given response. The software tools used at the initial stages do not capture nuance or ambiuity, so it is important that post-processing be done by the researcher. The third stage involved an analysis of internal consistency and review by multiple raters to achieve useful qualitative categories which were combined for further analysis with quantitative variables also collected in the project.