Using Big Data for Practical Work

• Activity Number: 573
• Type: Invited
• Date/Time: Wednesday, August 7, 2013 : 2:00 PM to 3:50 PM
• Sponsor: Section for Statistical Programmers and Analysts
• Author(s): Nancy J. Petersen*+
• Companies: Department of Veterans Affairs
• Keywords: Big data ; statistical programming ; administrative data ; Veterans

Abstract

The Department of Veterans Affairs (VA) uses an electronic medical record system to capture nationwide data on the use of its health care services by Veterans. Annual data are stored in multiple files, representing aspects of care such as outpatient clinic visits, hospitalizations, nursing home care, pharmacy, and laboratory services. These datasets were originally developed for administrative purposes but are also used to answer numerous research questions.

The way that VA researchers access the files has been evolving over time. This has come about because ensuring the security of the data is key and because the number and size of the files has increased. In the past, statistical analysts used a central mainframe computer or used extracts of the large files that were downloaded to their local computing environment. The different approaches that statistical analysts currently use to obtain and analyze these files will be discussed.