Online Program

Risk Intelligent Modeling: Avoiding Common Pitfalls of Black Box Analytics (302465)

 View Presentation

Keywords: Analytics, Pitfalls, Documentation

Modern statistical software packages provide complex computational power through “intelligent” and “user friendly” user interfaces. Such easy-to-use interfaces can obscure underlying assumptions and hide inherent limitations of the statistical models they produce. As a result, the models generated can become “black boxes,” mathematical algorithms that are impossible to understand and/or critically review, even for experienced statisticians. Such models, especially when insufficiently documented, are difficult to explain and defend to stakeholders or third-parties, such as a regulators and auditors. We offer practical advice to help reduce or remediate common pitfalls of statistical models that may have inadequate documentation or non-transparent assumptions. These proposed strategies may limit the potential for invalid results and/or misinformed decision making and are designed to help separate the statistics from the judgments, such that the statistics are academically supportable while the judgments can be challenged based on business knowledge. We bring the two together to help provide transparent, maintainable, and effective statistical models.