What Are the Statistical Challenges of Big Data Science?

• Activity Number: 570
• Type: Roundtables
• Date/Time: Wednesday, August 12, 2015 : 12:30 PM to 1:50 PM
• Sponsor: Section on Statistical Computing
• Author(s): Kaiser Fung*
• Companies: New York University
• Keywords: BIg Data ; Data science ; Web data ; Data quality ; Statistical models

Abstract

Something is brewing in the era of Big Data/data science, but we are not sure what is in it yet. It isn't about how fast we can process mountains of data. We discuss what the real challenges are in this new world and where exciting opportunities for statistical research lie. We start with the observation that modern data sets often include characteristics such as unknown or haphazard sampling design, no direct controls, errors due to merging, borrowing from other uses, high granularity, and the need to answer reverse causation questions. Web datastreams used to measure marketing effectiveness or generate recommendations are good examples. We encourage practitioners and researchers to share their perspectives from their own data sets.