Implementation of Deep Neural Networks via CUDA GPUs: An Application in Large-Scale Image Classification

• Activity Number: 614
• Type: Contributed
• Date/Time: Thursday, August 7, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Junjing Lin*+ and Carrie Segal and Sreenivasa Rao Jammalamadaka
• Companies: University of California, Santa Barbara and University of California, Santa Barbara and University of California, Santa Barbara
• Keywords: machine learning ; large scale methods ; deep learning ; statistical computing ; parallel computing

Abstract

In the pattern recognition arena, deep neural networks (DNN) has recently been proved to outperform many other machine learning techniques due to its advantages of the deep-layer structure and the distributed representation. However, a major concern of DNN is the long training time. The training of DNN is usually gradient-based, which is computationally intensive for large data sets. Thanks to the inherently parallel nature of the model, the huge amount of floating point operations in the training steps, and the relatively low data transfer, it is feasible to exploit the computing power of Graphic Processing Units (GPU) with Nvidia's scalable Compute Unified Device Architecture (CUDA) framework. In this project, we conducted the scaling analysis for various DNN model complexities and different number of GPU cores versus CPU on a supercomputer cluster. The performance is measured in training time and accuracy level. The results show that using GPU has paramount advantages over using CPU.