A New Tidy Data Structure to Support Exploration and Modeling of Temporal Data

• Activity Number: 19 - Statistical Computing and Statistical Graphics: Student Paper Award and Chambers Statistical Software Award
• Type: Topic Contributed
• Date/Time: Sunday, July 28, 2019 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Computing
• Author(s): Earo Wang* and Dianne Cook and Rob J Hyndman
• Companies: Monash University and Monash University and Monash Univeristy
• Keywords: time series; data wrangling; tidy data; R; forecasting; data science

Abstract

Mining temporal data for information is often inhibited by a multitude of formats: irregular or multiple time intervals, point events that need aggregating, multiple observational units, and heterogeneous data types. On the other hand, the software supporting time series modeling, makes strict assumptions on the data to be provided, typically requiring a matrix of numeric data with implicit time indexes. Going from raw data to model-ready data is painful. This work presents a cohesive and conceptual framework for organizing and manipulating temporal data, which in turn flows into visualization, forecasting routines. Tidy data principles are extended to temporal data by: (1) mapping the semantics of a dataset into its physical layout; (2) including an explicitly declared index variable representing time; (3) incorporating a "key" comprising single or multiple variables to uniquely identify units over time. This tidy data representation most naturally supports thinking of operations on the data as building blocks, forming part of a "data pipeline" in time-based contexts. The infrastructure of tidy temporal data has been implemented in the R package "tsibble".