Research map / Learning theory and algorithms / Prediction and decision losses
How many classifiers do we need?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes how disagreement and a newly defined polarization among classifiers govern majority-vote error as the number of ensemble members grows, proposing a neural polarization law (4/3) and an extrapolation estimator from three classifiers.
Problem
Predicting how majority-vote ensemble error changes with the number of classifiers, since existing bounds based on average error and disagreement are loose and uninformative in practice.
Approach
Introduces polarization (a ratio of P(W>1/2) to E[W^2]) that yields tight majority-vote bounds linear in disagreement, conjectures interpolating neural nets are 4/3-polarized, and uses a U-statistic asymptotic result to extrapolate majority-vote error from three sampled classifiers.
Key results
- Empirically, polarization of interpolating classifiers (ResNet18, ResNet50, ResNet101, VGG11, DenseNet40, MobileNet) is mostly below 4/3 across hyperparameters and architectures, supporting the proposed neural polarization law.
- The proposed Corollary bound is strictly tighter than the C-bound and the tandem-loss-based bound on CIFAR-10 and CIFAR-10.1 ensembles.
- Extrapolated majority-vote error estimated from three classifiers closely tracks the true majority-vote error for ensembles of larger size across CIFAR-10, CIFAR-10.1, Fashion-MNIST, and Kuzushiji-MNIST.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed finite-ensemble bound instead of the C-bound or the prior disagreement-based bound | majority-vote error estimation · CIFAR-10 and CIFAR-10.1 | upper-bound tightness | Helped |
| estimate majority-vote error from three classifiers using the proposed extrapolation | majority-vote error estimation · CIFAR-10, CIFAR-10.1, Fashion-MNIST, Kuzushiji-MNIST, and MNIST | majority-vote error estimation accuracy | Helped |
| vary classifier hyperparameters or architecture | neural-network ensembles · CIFAR-10 | polarization | No clear effect |
| evaluate the same ensembles on an out-of-distribution dataset | ResNet18 ensembles trained on CIFAR-10 · CIFAR-10.1 | polarization | No clear effect |
| use the proposed extrapolated error estimators instead of prior majority-vote error bounds | majority-vote error estimation · image-classification model and dataset pairs | estimation accuracy | Helped |
Surprising findings
- Polarization is nearly constant for a dataset regardless of hyperparameters or architecture.
- Pathological examples show the first-order bound L(MV) <= 2 E[L(h)] is tight, so first-order information alone cannot yield useful bounds.
Limitations
- The neural polarization law is a conjecture, not proved, and one interpolating model exceeded 4/3.
- Empirical validation is limited to image classification with small CNN-style models on a few datasets.
- The extrapolation relies on approximations and on tie-free, competent-style assumptions.
Experimental setup
Datasets: CIFAR-10 CIFAR-10.1 MNIST Fashion-MNIST Kuzushiji-MNIST
Benchmarks: CIFAR-10 CIFAR-10.1 MNIST Fashion-MNIST Kuzushiji-MNIST
Metrics: polarization majority vote error rate bound tightness
Baselines: C-bound tandem loss bound (Masegosa et al.) first-order bound (2x average error)
Builds on: C-bound tandem loss competence condition
Closest papers by meaning
- When are ensembles really effective? (NeurIPS 2023 · similarity 0.87)
- An urn model for majority voting in classification ensembles (NeurIPS 2016 · similarity 0.87)
- Exact power indices for plurality-voting ensembles (NeurIPS 2026 · similarity 0.86)
- Don’t Label Twice: Quantity Beats Quality when Comparing Binary Classifiers on a Budget (ICML 2024 · similarity 0.85)
- Classes Are Not Equal: An Empirical Study on Image Recognition Fairness (CVPR 2024 · similarity 0.84)
- When majority rules, minority loses: bias amplification of gradient descent (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.