Self-Tuning, Nonlinear Parameter Estimation: A 21st-Century Approach to Statistics

• Activity Number: 283 - Deep Learning Methods
• Type: Contributed
• Date/Time: Tuesday, August 9, 2022 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Mary Gregg* and Lucas Koepke and Angela Folz and Michael Frey
• Companies: National Institute of Standards and Technology and National Institute of Standards and Technology and National Institute of Standards and Technology/University of Colorado Boulder and National Institute of Standards and Technology
• Keywords: Machine learning; Artificial neural network; Parameter estimation; Synthetic training data; Linear regression

Abstract

Simple linear regression (SLR) is an archetypal parameter estimation problem with wide-ranging applications. The SLR parameter of key interest is the regression slope, usually estimated by the method of least-squares (LS). We propose a nonlinear, self-tuning estimator of slope based around an artificial neural network (ANN) architecture. This nonlinear estimator is tuned – the ANN is trained – using synthetic data generated from knowledge of the posterior distribution of the LS estimator. By training the ANN on purely synthetized data tailored to the data set at hand, this methodology bridges the needs of ANNs with the reality of applied analysis, opening the door for a 21st century approach to statistical estimators. We demonstrate proof of concept, showing that the ANN estimator works well in the problem of SLR, and discuss how the idea is extendable to general estimation scenarios.