Abstract #300949

This is the preliminary program for the 2003 Joint Statistical Meetings in San Francisco, California. Currently included in this program is the "technical" program, schedule of invited, topic contributed, regular contributed and poster sessions; Continuing Education courses (August 2-5, 2003); and Committee and Business Meetings. This on-line program will be updated frequently to reflect the most current revisions.

To View the Program:
You may choose to view all activities of the program or just parts of it at any one time. All activities are arranged by date and time.

The views expressed here are those of the individual authors
and not necessarily those of the ASA or its board, officers, or staff.

Back to main JSM 2003 Program page
JSM 2003 Abstract #300949
Activity Number: 474
Type: Contributed
Date/Time: Thursday, August 7, 2003 : 10:30 AM to 12:20 PM
Sponsor: Section on Statistical Graphics
Abstract - #300949
Title: Visualizing Disagreements in Experts Ratings for Evaluations and Performance Assessments
Author(s): Chris W. T. Chiu*+
Companies: Illinois Student Assistance Commission
Address: 52 Gable Wing Circle, Newtown, PA, 18940-3320,
Keywords: graphics and visualization ; computation and statistic ; repeated measure ; testing and psychometrics ; sparse matrix and hash function ; Law School Admission Test (LSAT)
Abstract:

Analysts, researchers, scientists, and policy makers often encounter large amounts of data. This paper extends a visualization technique, SEER, for graphically analyzing and exploring massive amounts of data collected in large-scale surveys and high-stake testing programs involving expert judgments. Despite rigorous trainings provided to expert raters when scoring open-ended questions, the raters may still disagree on their ratings. Depicting disagreements meaningfully and rapidly remains a challenge because the data manipulation process can be labor intensive and thus makes the graphical displays difficult to create. Also, considerable efforts are required to show score discrepancy, score distribution, and other diagnostic information all at once. To this end, the current paper modified the hash function so that it can display pairs of ratings simultaneously while showing the score distributions. The visualization technique (a) draws on sparse matrix methods and (b) is illustrated by a dataset from the Law School Admission Test (LSAT). Results indicated that the graphical technique can aid statisticians to diagnose score discrepancy too trivial for nongraphical techniques.

  • The address information is for the authors that have a + after their name.
  • Authors who are presenting talks have a * after their name.

Back to the full JSM 2003 program

JSM 2003 For information, contact meetings@amstat.org or phone (703) 684-1221. If you have questions about the Continuing Education program, please contact the Education Department.
Revised March 2003