Abstract Details
Activity Number:
|
33
|
Type:
|
Contributed
|
Date/Time:
|
Sunday, August 4, 2013 : 2:00 PM to 3:50 PM
|
Sponsor:
|
Section on Statistical Computing
|
Abstract - #309491 |
Title:
|
Automating Data Exploration Through Interestingness and Insights
|
Author(s):
|
Jing Shyr*+ and Damir Spisic
|
Companies:
|
IBM and IBM
|
Keywords:
|
data exploration ;
automation ;
interestingness ;
insights
|
Abstract:
|
Data exploration includes generating an overview of variables contained in a dataset and an understanding of the most important relationships among the existing variables. Data needs to be prepared for modeling while the models themselves are used for gaining insights into the main aspects of the discovered relationships. It involves the use of many statistical techniques that require the judicious application of a proficient analyst. The task becomes much more difficult for large datasets with many variables. We present a robust automation framework that runs a set of exploratory statistical analyses on a given dataset. The statistics range from univariate analyses establishing the necessary metadata information to multivariate analyses discovering the relationships between the target variables and potential predictors. All the results are sorted by a new "interestingness" index comprised of suitable statistics. This allows the most relevant relationships to be selected. The most important aspects of the discovered relationships are further analyzed and presented as insights and expressed using a non-technical language allowing non-experts to gain a better understanding of their data.
|
Authors who are presenting talks have a * after their name.
Back to the full JSM 2013 program
|
2013 JSM Online Program Home
For information, contact jsm@amstat.org or phone (888) 231-3473.
If you have questions about the Continuing Education program, please contact the Education Department.
The views expressed here are those of the individual authors and not necessarily those of the JSM sponsors, their officers, or their staff.
Copyright © American Statistical Association.