Research map / Learning theory and algorithms / Fairness
Position: Amazing Things Come From Having Many Good Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A perspective piece arguing that the Rashomon Effect (many equally good models for noisy tabular data) should reshape ML practice, covering simpler accurate models, constraint handling, uncertainty, stable variable importance, algorithm choice, and policy.
Problem
Standard ML returns a single model from many near-equally-good models (the Rashomon set), ignoring the resulting flexibility, uncertainty, and interpretability opportunities, especially for noisy tabular data in high-stakes settings.
Approach
Reframe machine learning around finding and exploring the full Rashomon set of good models rather than selecting one model, arguing this resolves the interaction bottleneck, explains the absence of an accuracy-interpretability trade-off in noisy data, and supports policy for default interpretable models.
Key results
- On the FICO dataset, random forest, boosted trees, SVMs, logistic regression, an 8-layer neural network and a 2-layer additive risk model all reach roughly 0.70-0.74 test accuracy (AUC 0.757-0.806).
- A sparse decision tree with 7 leaves, found by GOSDT in 8.1 seconds, reaches about 72% train and test accuracy, comparable to black-box models.
- A sparse GAM with 11 features, obtained by FastSparse in 3.15 seconds, reaches test AUC 0.790 and accuracy 0.723.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use different model classes for FICO prediction | FICO classifiers · FICO test folds | Test Accuracy | No clear effect |
| Use a sparse decision tree instead of black-box classifiers | FICO prediction · FICO | test accuracy: approximately 72% | No clear effect |
| Use a sparse GAM without feature interactions instead of decision trees | FICO prediction · FICO | test accuracy: .723 | No clear effect |
Surprising findings
- Model reliance on the same features differs substantially across top-performing model classes, so variable importance rankings are inconsistent.
- Interpretable models match black-box accuracy on noisy tabular data, so no accuracy-interpretability trade-off appears in many domains.
Limitations
- Perspective piece; most claims are distilled from prior technical papers rather than new experiments.
- The Rashomon effect is argued to be less present for deterministic labels or large-margin data, so the conclusions depend on noise levels.
Experimental setup
Datasets: FICO Explainable ML Challenge dataset COMPAS
Benchmarks: FICO dataset (23 features) COMPAS
Metrics: test accuracy test AUC model reliance cross-validation accuracy
Baselines: random forest boosted trees SVM logistic regression 8-layer neural network CART C4.5
Builds on: TreeFARMS FastSparse GOSDT FasterRisk Rashomon Importance Distribution
Closest papers by meaning
- RashomonGB: Analyzing the Rashomon Effect and Mitigating Predictive Multiplicity in Gradient Boosting (NeurIPS 2024 · similarity 0.90)
- Using Noise to Infer Aspects of Simplicity Without Learning (NeurIPS 2024 · similarity 0.85)
- Dropout-Based Rashomon Set Exploration for Efficient Predictive Multiplicity Estimation (ICLR 2024 · similarity 0.85)
- ElliCE: Efficient and Provably Robust Algorithmic Recourse via the Rashomon Sets (NeurIPS 2025 · similarity 0.84)
- Exploring the cloud of feature interaction scores in a Rashomon set (ICLR 2024 · similarity 0.84)
- SORTeD Rashomon Sets of Sparse Decision Trees: Anytime Enumeration (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.