B-DeepONet: An Enhanced Bayesian DeepONet for Solving Noisy Parametric PDEs Using Accelerated Replica Exchange SGLD

• Activity Number: 327 - On Surrogate Modeling of Emerging Issues in Physical and Engineering Simulators
• Type: Topic Contributed
• Date/Time: Tuesday, August 9, 2022 : 2:00 PM to 3:50 PM
• Sponsor: Section on Physical and Engineering Sciences
• Author(s): Guang Lin* and Christian Moya and Zecheng Zhang
• Companies: Purdue University and Purdue University and Purdue University
• Keywords: Parametric PDE; deep operator network; noisy data; Bayesian method; Langevin diffusion; replica exchange

Abstract

Deep Operator Networks~(DeepONet) is a neural network used to approximate the operators, including the solution operator of parametric PDE. DeepONets have shown remarkable approximation ability. However, the performance of DeepONets deteriorates when the training data is polluted with noise, a scenario that occurs in practice. To enable DeepONet to handle noisy data, we propose a Bayesian DeepONet based on replica exchange Langevin diffusion~(reLD). Replica exchange uses two particles; one particle exploits the loss function and is used to predict. The other particle trains a different DeepONet that explores the loss function landscape to escape the local minima via swapping. Compared to DeepONets trained with state-of-the-art gradient-based optimization algorithms ( Adam), the proposed Bayesian DeepONet greatly improves the training convergence for noisy scenarios, accurately estimates the uncertainty. Lastly, to further reduce the high computational cost of reLD training of DeepONets, we propose (1) an accelerated training framework that exploits the DeepONets architecture to reduce its cost up to 25% without compromising the performance and (2) a transfer learning strategy