Novel Application of Statistical Tools for Big Data Analyses of Solar Physics

• Activity Number: 203
• Type: Contributed
• Date/Time: Monday, August 10, 2015 : 11:35 AM to 12:20 PM
• Sponsor: Section on Statistics and the Environment
• Author(s): Siavoush Mohammadi and Lars K.S. Daldorff*
• Companies: Infotrek and University of Michigan/NASA GSFC
• Keywords: Explorative analytics ; Heliophysics ; Data extraction ; Big data ; Sun

Abstract

Space plasma simulations are known to generate vast amounts of numerical data. In recent times,with greater availability of computer power, the proportion of data generated has increased exponentially posing new challenges in its analysis. Simulations can be scaled up, but matching methods for analysis have not been developed at the same pace, except in the industry setting where companies utilize big-data techniques to build advanced analysis engines. Although many of these methods need developers working with low-level big-data programming, today there exist powerful out-of-the box solutions that can be easily employed. In this paper, using SAS Visual Analytics as one of these tools, we will demonstrate how existing statistical tools and analytical platforms can be used to analyze simulations of the Sun in a novel way. In our case, simulations generated roughly 660mb/time unit and hundreds of them were needed to provide an adequate analysis. Thus, application of our method allows instant analysis and prevents the need to guess "when" and "where" the interesting physical phenomena occurs, thus effectively getting rid of data which has little or no scientific value.