Visualization Databases for the Analysis of Large Complex Data Sets

• Activity Number: 47
• Type: Invited
• Date/Time: Sunday, August 2, 2009 : 4:00 PM to 5:50 PM
• Sponsor: Section on Statistical Graphics
• Author(s): Saptarshi Guha*+ and William S. Cleveland
• Companies: Purdue University and Purdue University
• Keywords: data visualization ; distributed computing ; graphical perception ; massive datasets

Abstract

Comprehensive visualization that preserves the information in a large complex data set requires a visualization database (VDB): many displays, some with many pages and with one or more panels per page. A single display results from partitioning the data into subsets, sampling the subsets, and applying a display method to each sample, typically one per panel. Display methods and display viewers can be designed to allow rapid scanning of a large display. Often, it is not necessary to view every page of the display. Computation is embarrassingly parallel and can be greatly reduced by RHIPE a recent merging of the R interactive environment for data analysis and the Hadoop distributed file system and compute engine. VDBs, already successful just with off-the-shelf tools, can be greatly improved by research that rethinks all areas of data visualization in the context of VDBs.