Big Data: Does the Song Remain the Same?

• Activity Number: 210
• Type: Invited
• Date/Time: Monday, August 5, 2013 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Education
• Author(s): Chris J. Wild*+ and Antony Unwin
• Companies: University of Auckland and IUniversity of Augsburg
• Keywords: statistical graphics ; big data ; statistical modelling ; conceptual pathways

Abstract

"Everything that's small has to grow . the song remains the same" (Led Zeppelin). As we plan the move to big (or at least much bigger) data, what statistical fundamentals change and what remains the same? To what extent does bigger data impose additional demands on what it takes to comprehend data? Data preparation and cleaning become a much bigger problem, graphics can become overcrowded and can suffer from over plotting, model building can be inefficient and some models cannot be fitted at all. On the other hand better data preparation helps our appreciation of the strengths and weaknesses of the data, we can ask more detailed questions, graphics can still be surprisingly informative and models fit to samples can reveal global features. But how does all this affect what we should teach and when? Computing power progressively solves the technical problems but the conceptual issues will always be as fundamental as ever.