Survey Data, Big Data, State Space Models, and Official Statistics

• Activity Number: 624
• Type: Invited
• Date/Time: Thursday, August 13, 2015 : 8:30 AM to 10:20 AM
• Sponsor: Survey Research Methods Section
• Author(s): Siu-Ming Tam*
• Companies: Australian Bureau of Statistics
• Keywords: Predictive distributions ; Kalman Filter ; Crop Yields

Abstract

In official statistics, data collected from scientific sampling methods are considered as the "gold standard" from which statistically valid descriptive and analytic inferences can be made. While Big Data, being digital trails, are generally inexpensive to collect, they invariably suffer from different levels of coverage bias, representational bias and measurement errors. On the other hand, Big Data, where properly harnessed, can be used to provide statistical products more frequently, with more details at the small area or small domain levels, and cheaper, than those provided from censuses and surveys. The challenge to the official statistician is to find an efficient and effective way to harness Big Data, in such a way that the official statistics derived such sources continue to provide the high level of reliability available from survey data. In this talk, we shall present a methodology in which survey data and Big Data are integrated to formulate a State Space model for predicting the finite population parameters of interest to the official statistician. We shall illustrate the methodology using a Big Data source.