The Pythagorean Law of Mutual Information Identity: A New Look at Logistic Regression Parameters

• Activity Number: 408 - SPAAC Poster Competition
• Type: Topic Contributed
• Date/Time: Tuesday, July 31, 2018 : 2:00 PM to 3:50 PM
• Sponsor: Scientific and Public Affairs Advisory Committee
• Author(s): Michelle Liou* and Jiun-Wei Liou and Philip E. Cheng
• Companies: Academia Sinica and Academia Sinica and Academia Sinica
• Keywords: Information Identity; Logistic Regression; Pythagorean Law; Categorical Data; Mutual Information; Contingency Table

Abstract

The Pythagorean law of mutual information identity was proposed to attribute the conditional mutual information in an IxJxK contingency table to the sum of two orthogonal parts of deviance residuals: one resulting from fitting the maximum likelihood estimates (MLEs) of common odds ratios (ORs) to the IxJ tables across individual levels of K, and the other referring to the deviation of the fitted MLE tables from those by equating common ORs to one (Cheng, Liou & Aston, 2010). In this poster, we explained the key ideas behind the Pythagorean law and illustrated its potential use, among other things, in categorical data analysis. By using a real data example including one target variable along with three predictors, we illustrated the proposed information approach to estimating logistic regression parameters and compared the parameter estimates with those using the conventional logistic regression. We concluded that the conventional approach to modeling associations between categorical variables could lead to incorrect inference problems if their mutual information identity was not adequately specified.