Data Summaries and Noise Reduction in High-Volume Particle Physics Data

• Activity Number: 315
• Type: Invited
• Date/Time: Tuesday, August 5, 2014 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Computing
• Author(s): Karen Kafadar*+
• Companies: Indiana University
• Keywords: letter value plots ; density estimate ; signal/noise ratio ; B-meson ; target events

Abstract

The analysis of massive, high-volume data sets stresses usual statistical methods and requires new ways of drawing inferences beyond the conventional paradigm (optimal estimation of parameters from a hypothesized distribution or mixture), since the entire data set often cannot be read into software. Data from high-energy particle physics experiments are nearly continuous streams of observations, most of which relate to known events, so the challenge is to identify the very small fraction of events (< 0.01%) that relate to diverse interesting phenomena. We discuss principles of analyzing massive data, present useful displays and analyses of data from high-energy physics experiments with techniques designed to reduce the distraction from noise, and note open issues in analyzing high-volume data in general. (Data from R.L. Jacobsen, UC-Berkeley)