Decoding the Pixels of the Face Image from the Voxels of fMRI BOLD Activity Patterns

• Activity Number: 198
• Type: Contributed
• Date/Time: Monday, August 10, 2015 : 10:30 AM to 12:20 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Rick Farouni*
• Companies: The Ohio State University
• Keywords: Multivariate Pattern Analysis ; fMRI ; Bayesian Modeling ; Neuroimaging ; Predictive Modeling ; Face Perception

Abstract

The main research goal of the project was to apply a multivariate generative Bayesian encoding-decoding modeling framework to an existing publicly available fMRI dataset with the overall aim to predict the category of, and possibly identify, the face image stimuli from the observed distributed activity of fMRI BOLD responses. Most commonly used approaches to classification of induced mental states rely on discriminative models that map data to causes (e.g. Support Vector Machines), whereas the Bayesian approach we are pursuing is based on the specification of a generative model (Friston, 2008) that maps causes to data, but whose inversion maps brain activity to their consequences (e.g. percepts), not to their causes. Similar to, and building upon, the recent work of Guclu et al. (2014), we inverted a decoding model from an encoding model comprised of two components: (1) a non-linear unsupervised feature model that learns the mapping from unlabeled data (i.e. face images) and (2) a linear voxel model that maps the stimulus features into voxel responses. The results showed that given an fMRI BOLD response activity pattern, we are able to predict to which category of stimulus (i.e. famous faces, unfamiliar faces and scrambled faces) the activity pattern corresponds to. More importantly, the approach we have employed provides us with the opportunity to extend the predictive modeling framework to incorporating EEG and MEG data in a integrative hierarchical Bayesian model.