Yijun Wei
National Institute of Statistical Sciences, United States Department of Agriculture
508 – Leading the Estimates Towards Known Benchmarks
Deep Learning for Data Imputation and Calibration Weighting
Yijun Wei
National Institute of Statistical Sciences, United States Department of Agriculture
Luca Sartore
United States Department of Agriculture
Jake Abernethy
United States Department of Agriculture
Darcy Miller
United States Department of Agriculture
Kelly Toppin
United States Department of Agriculture
Michael Hyman
United States Department of Agriculture
The USDA's National Agricultural Statistics Service (NASS) surveys are affected by nonresponse and by incomplete responses that may not be homogeneous across farm types and sizes. To address item nonresponse, NASS employs a variety of imputation methods such as ratio imputation, iterative sequential regression, fully conditional specification, K-nearest neighbor, carry forward of previously reported data and manual imputation to provide reliable and consistent values on NASS data. To address unit nonresponse and some other sources of error, NASS currently uses a set of generalized linear regression models to estimate the number of US farms by calibrating their corresponding weights. However, linear models cannot always capture important nonlinear features of the population. Deep learning (including artificial neural network) models are used successfully in numerous other applications in order to capture nonlinear properties efficiently. In this paper, imputation techniques and the adjustment of the survey weights are integrated. A potential unified deep learning method simultaneously adjust survey weights and impute missing values is discussed.