Bi-Orthogonal Tensor Decomposition for Image Style Matching

• Activity Number: 178 - Novel Applications and Extensions of Dimension Reduction Methods
• Type: Contributed
• Date/Time: Monday, July 29, 2019 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Yutong Li* and Ruoqing Zhu and Annie Qu
• Companies: University of Illinois at Urbana-Champaign and University of Illinois Urbana-Champaign and University of Illinois at Urbana-Champaign
• Keywords: Tensor Decomposition; Neural Style Matching; Convolutional Neural Network; Dimension Reduction; Computer Vision

Abstract

We seek to match the style of an art piece with a collection of images generated by various neural style transfer algorithms. In current literature, this is done by directly comparing the gram matrices computed from all the features maps in each convolution layer of the images of interest. Due to the increasing abstractness of the convolution layers at lower levels, this approach is noisy, computationally inefficient, and fails to consider the latent connection across all the features maps. To address these issues, we propose the Bi-orthogonal Tensor Decomposition to reduce the collection of feature maps into a lower-dimensional representation. The gram matrices are then re-calculated from this reduced dimensional space to perform the style matching. Numerical experiments show the effectiveness of our method in capturing the true signal of the features maps with a lower-dimensional representation while also subsequently improving the accuracy of style matching.