JSM 2013 Home
Online Program Home
My Program

Abstract Details

Activity Number: 309
Type: Contributed
Date/Time: Tuesday, August 6, 2013 : 8:30 AM to 10:20 AM
Sponsor: Section on Statistical Computing
Abstract - #309138
Title: Simulation Based Nearest Neighbor Entropy Estimation for (Adaptive) MCMC Evaluation
Author(s): Didier Chauveau*+ and Pierre Vandekerkhove
Companies: CNRS and University Marne la Vallée - CNRS
Keywords: Entropy estimation ; Adaptive MCMC algorithms ; Nonparametric statistics ; Parallel computation ; Nearest Neighbor estimates ; Kullback divergence

Many recent (including adaptive) MCMC methods are associated in practice to unknown rates of convergence. We propose a simulation-based methodology to estimate MCMC efficiency, grounded on a Kullback divergence criterion requiring an estimate of the entropy of the algorithm successive densities, computed from iid simulated chains. We recently proved in Chauveau and Vandekerkhove (2013) some consistency results in MCMC setup for an entropy estimate based on Monte-Carlo integration of a kernel density estimate based on Gyorfi and Van Der Meulen (1989). Since this estimate requires some tuning parameters and deteriorates as dimension increases, we investigate here an alternative estimation technique based on Nearest Neighbor (NN) estimates. This approach has been initiated by Kozachenko and Leonenko (1987) but used mostly in univariate situations until recently when entropy estimation has been considered in other fields like neuroscience. We show that in MCMC setup where moderate to large dimensions are common, this estimate seems appealing for both computational and operational considerations, and that the problem inherent to a non neglictible bias arising in high dimension can be overcome. All our algorithms for MCMC simulation and entropy estimation are implemented in an R package taking advantage of recent advances in high performance (parallel) computing.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2013 program

2013 JSM Online Program Home

For information, contact jsm@amstat.org or phone (888) 231-3473.

If you have questions about the Continuing Education program, please contact the Education Department.

The views expressed here are those of the individual authors and not necessarily those of the JSM sponsors, their officers, or their staff.

ASA Meetings Department  •  732 North Washington Street, Alexandria, VA 22314  •  (703) 684-1221  •  meetings@amstat.org
Copyright © American Statistical Association.