JSM 2005 Online Program

Abstract #303929

This is the preliminary program for the 2005 Joint Statistical Meetings in Minneapolis, Minnesota. Currently included in this program is the "technical" program, schedule of invited, topic contributed, regular contributed and poster sessions; Continuing Education courses (August 7-10, 2005); and Committee and Business Meetings. This on-line program will be updated frequently to reflect the most current revisions.

To View the Program:
You may choose to view all activities of the program or just parts of it at any one time. All activities are arranged by date and time.

The views expressed here are those of the individual authors
and not necessarily those of the ASA or its board, officers, or staff.

The Program has labeled the meeting rooms with "letters" preceding the name of the room, designating in which facility the room is located:

Minneapolis Convention Center = “MCC” Hilton Minneapolis Hotel = “H” Hyatt Regency Minneapolis = “HY”

Back to main JSM 2005 Program page

Legend:

= Applied Session,

= Theme Session,

= Presenter

Activity Number:	326
Type:	Contributed
Date/Time:	Tuesday, August 9, 2005 : 2:00 PM to 3:50 PM
Sponsor:	Section on Statistical Graphics
Abstract - #303929
Title:	A Visualization System to Monitor Rater Reliability in Large-scale Computer- and Paper-based Systems
Author(s):	Chris Chiu*+
Companies:	Law School Admission Council
Address:	662 Penn Street, Newtown, PA, 18940, United States
Keywords:	Visualization ; Computational Statistics ; Rater Reliability ; Quality Control ; Computer-Based Systems ; Paper-Based Systems
Abstract:	It is known that data collection design plays a critical role in the estimation of inter-rater reliability coefficients and their standard errors. With today's technological advances in computing and internet development, data collection designs can become highly dynamic, making it a challenge to diagnose factors that may influence the quality of inter-rater reliability estimates in measurement instruments. To overcome these challenges, we propose an integrated system to visualize the linkage among raters, the workload of raters, the decision consistency of raters, the distribution of ratings, and the underlying data collection design (e.g., crossed, nested, and MBIB). The proposed visualization system is developed from published research (Chiu 2001, Chiu and Fecso 2004) with an aim to visually monitor factors that influence the precision and accuracy of reliability estimates, identify raters in need of training and calibration, equalize workloads for raters, and ensure sufficient linkages among raters, tasks, and persons for scaling.

The address information is for the authors that have a + after their name.
Authors who are presenting talks have a * after their name.

Back to the full JSM 2005 program