|
Abstract:
|
The concept of "data squashing" was introduced by DuMouchel, Volinsky, Johnson, Cortes and Pregibon (1999) to alleviate the computational burden of the statistical analysis of massive data-sets. The key idea is to squash the original massive data set into a smaller representative sample and then apply the statistical procedure such as MLE to the squashed data. While many exemplifications have been reported to show the practicability of the method, not much research has been made into the accuracy of the method. In this talk we give theoretical arguments to evaluate the accuracy of the data squashing method. To construct the squashed data, DVJCP (1999) partitions the whole data range into bins and matches the local moments of the squashed data to that of the original data on each bin. We show this as a process of density estimation and investigate it in the context of the kernel method.
|
