Empirical Relationships Between Classification and Model Complexity in Neural Networks

• Activity Number: 294
• Type: Contributed
• Date/Time: Tuesday, August 5, 2003 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Computing
• Author(s): Doug Landsittel*+
• Companies: University of Pittsburgh
• Keywords: neural networks ; degrees of freedom ; classification

Abstract

Although substantial literature has focused on the application of neural networks to classification, frequent questions persist about their true utility. These concerns primarily relate to their black box nature (i.e., poor understanding of the underlying model form and complexity) and resulting potential for overfitting. Classification accuracy depends not only on adequate model complexity for fitting the data, but also on sufficient parsimony for generalization. Achieving this balance with neural networks has been problematic because of our limited ability to quantify their model complexity and interpret the underlying model. This study will apply a methodology for generalized degrees of freedom to selecting models that achieve accurate training set classification, but also restrict model complexity for improved generalization. Neural networks with a range of model structures are fit to simulated data of varying complexity. Empirical relationships are derived between the degrees of freedom and subsequent classification on validation sets (for each level of data complexity). Guidelines are given on resulting practical issues (e.g., choosing the number of hidden units).