174 – Big Data: Strategies for Large P, Large N, or Massive Data
Limited-Information Statistics When the Number of Variables Is Large
Mark Reiser
Arizona State University
The Pearson and likelihood ratio statistics are commonly used to test goodness of fit for models applied to data from a multinomial distribution. When data are from a table formed by the cross-classification of a large number of variables, the common statistics may have low power and inaccurate Type I error level due to sparseness. Several statistics have been proposed that use components of the Pearson statistic obtained from marginal distributions. These statistics have mostly been applied to item response models or factor analysis of categorical variables and have very good performance for Type I error rate and power when the data table is formed from a moderate number of variables. However, there are limitations when the number of variables becomes larger than 20. This paper compares the performance of statistics based on marginal distributions as well as computational resources required when the number of variables is large. The comparison includes test statistics from Christoffersson (1975), Reiser (1996, 2008), Bartholomew and Leung (2002), Tollenaar and Mooijaart (2003), and Maydeu-Olivares and Joe (2005).
