St. James Ballroom
Textual Analysis of Course Evaluations (303868)
*Caitlin Mary Cunningham, Le Moyne College
Theresa L. White, Le Moyne College
Keywords: Course Evaluations, Textual Analysis, Randomization Test, Education
Le Moyne College has approved a new short answer course evaluation form, which asks students to comment on what aspects of the course the instructor should Stop, Start, and Continue (the SSC form). To assess the reliability and validity of the SSC method formally, students were recruited to view one of two videos on basic statistics that featured instructors with different styles. Each student viewed their video twice. This poster presents the results of a textual analysis on the form responses, including word frequencies, sentiment analysis, and comment lengths. A randomization test was performed to assess the level of "uniqueness" in terms of word choice between the two videos and between the two sessions. The analyses performed confirmed that students used different language to assess the two videos, indicating validity, and used similar terms between the two sessions, indicating reliability of the form. The sentiment analysis showed that students tended to write more negatively in the Start and Stop sections, and positively in the Continue section. This analysis shows that short answer evaluation forms can act as a viable alternative to the typical multiple choice forms.