Conference Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 197 - SPAAC Poster Competition
Type: Topic Contributed
Date/Time: Monday, August 8, 2022 : 2:00 PM to 3:50 PM
Sponsor: Section on Statistical Learning and Data Science
Abstract #322741
Title: Understanding the Properties of Permutation Based Importance for the Inputs from Black Box Machine Learning Models
Author(s): Mohammad Kaviul Anam Khan* and Rafal Kustra
Companies: University of Toronto and University of Toronto
Keywords: Variable Importance Metric; Significance; Black Box Models; Non-linear; Non-Additive
Abstract:

The goal of this study is to identify important predictors for black box machine learning methods, where the prediction function is highly non-linear, non-additive and cannot be represented by statistical parameters. Thus, such black-box models lack interpretability and it is very difficult to identify ``important'' or significant inputs for an outcome from such models. The main target is to investigate applicability of permutation based approach, proposed by Breiman (2001) and then generalized by Fisher et.al (2019) to the common non-linear and non-additive machine learning techniques. Another aim is to decompose the proposed variable importance metric (VIM) to obtain a causal parameter which is a function of the expected conditional average treatment effect squared over the distribution of treatments for multinomial and continuous treatments. A simulation study was then conducted to check the performance of the estimated VIM using split-sampling techniques using multiple known machine learning methods. The estimation technique of VIM was also evaluated under model mis-specifications.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2022 program