Comparison of Empirical Size and Power of Goodness-of-Fit Tests for Multiple Logistic Regression Model Under Varied Sample Size Conditions

• Activity Number: 123 - Binary and Ordinal Outcome Regression
• Type: Contributed
• Date/Time: Monday, July 30, 2018 : 8:30 AM to 10:20 AM
• Sponsor: Biometrics Section
• Author(s): Pengcheng Lu* and Jonathan D Mahnken
• Companies: and University of Kansas Medical Center
• Keywords: Logistic regression; goodness of fit; power of hypothesis test; sample size

Abstract

The disadvantages of the use of chi-square-like goodness-of-fit test in logistic regression remain for sparse data, for example when continuous covariates are included in the model. We examine the performance of some goodness-of-fit tests, namely the Hosmer-Lemeshow (HL) test, the unweighted sum of squares (USS) test and the cumulative sums of residuals (CUSUM) test, which are based on the discrepancy between fitted value and observed value, by using simulated sparse data under different sample size conditions. Our study demonstrates the strong dependence of power and less dependence of type I error rate on sample size across those tests. For wrongly specified models the HL test has power less than 50% to detect the lack-of-fit when sample size is less than 1000, when sample size is less than 2000 both the USS and the CUSUM test have larger power than the HL test in most scenarios, but the USS test outperformed over the CUSUM test. When sample size exceeds 15000 all tests have high powers with no differentiation. On the other hand it is unsurprising that the coefficient weight of the missing covariate plays an important role in the power of all tests to detect lack-of-fit as well.