JSM 2011 Online Program

The views expressed here are those of the individual authors and not necessarily those of the JSM sponsors, their officers, or their staff.

Abstract Details

Activity Number: 15
Type: Topic Contributed
Date/Time: Sunday, July 31, 2011 : 2:00 PM to 3:50 PM
Sponsor: Biometrics Section
Abstract - #301234
Title: A Local Information Measure and Its Application to Screening for High Correlations in Large Data Sets
Author(s): Kumar Sricharan*+ and Alfred Hero
Companies: University of Michigan and University of Michigan
Address: , , ,
Keywords: structure discovery ; large dimensional probelms ; correlation screening ; non-parametric estimation
Abstract:

Correlation screening is frequently the only practical way to discover dependencies in very high dimensional data. In correlation screening a a high threshold is applied to the matrix of sample correlation coefficients of the multivariate data. The variables having coefficients that exceed the threshold are called discoveries and there exists a abrupt phase transition in the number of discoveries. This critical threshold that defines this phase transition depends on a measure $J$, which is a novel type of information divergence, that is a function of the joint density of pairs of variables. In this paper, we propose a simple uniform kernel estimator for $J$, establish asymptotic consistency and provide a central limit theorem for the estimate. The results of the analysis are used to optimize the estimator and to obtain confidence intervals. We illustrate our results on experimental data sets including network traffic and gene expression.


The address information is for the authors that have a + after their name.
Authors who are presenting talks have a * after their name.

Back to the full JSM 2011 program




2011 JSM Online Program Home

For information, contact jsm@amstat.org or phone (888) 231-3473.

If you have questions about the Continuing Education program, please contact the Education Department.