|
Abstract:
|
In machine learning, the Rashomon effect occurs when there exist many accurate-but-different models that describe the same data. We quantify the Rashomon effect, study how it can be useful for understanding the relationship between training and test performance, and show that it has implications for the existence of simple-yet-accurate models. We consider the Rashomon set as the set of approximately-equally accurate models for a given problem and study its properties. When the Rashomon set is large, models that are accurate - but that also obey various constraints such as interpretability or fairness can often be obtained. We present the Rashomon ratio as a new measure related to the simplicity of model classes. The Rashomon ratio differs from standard complexity measures from statistical learning theory. For a hierarchy of hypothesis spaces, we discuss how the ratio can help to navigate the trade-off between simplicity and accuracy in the model selection process.
|
