JSM 2013 Home
Online Program Home
My Program

Abstract Details

Activity Number: 33
Type: Contributed
Date/Time: Sunday, August 4, 2013 : 2:00 PM to 3:50 PM
Sponsor: Section on Statistical Computing
Abstract - #309491
Title: Automating Data Exploration Through Interestingness and Insights
Author(s): Jing Shyr*+ and Damir Spisic
Companies: IBM and IBM
Keywords: data exploration ; automation ; interestingness ; insights

Data exploration includes generating an overview of variables contained in a dataset and an understanding of the most important relationships among the existing variables. Data needs to be prepared for modeling while the models themselves are used for gaining insights into the main aspects of the discovered relationships. It involves the use of many statistical techniques that require the judicious application of a proficient analyst. The task becomes much more difficult for large datasets with many variables. We present a robust automation framework that runs a set of exploratory statistical analyses on a given dataset. The statistics range from univariate analyses establishing the necessary metadata information to multivariate analyses discovering the relationships between the target variables and potential predictors. All the results are sorted by a new "interestingness" index comprised of suitable statistics. This allows the most relevant relationships to be selected. The most important aspects of the discovered relationships are further analyzed and presented as insights and expressed using a non-technical language allowing non-experts to gain a better understanding of their data.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2013 program

2013 JSM Online Program Home

For information, contact jsm@amstat.org or phone (888) 231-3473.

If you have questions about the Continuing Education program, please contact the Education Department.

The views expressed here are those of the individual authors and not necessarily those of the JSM sponsors, their officers, or their staff.

ASA Meetings Department  •  732 North Washington Street, Alexandria, VA 22314  •  (703) 684-1221  •  meetings@amstat.org
Copyright © American Statistical Association.