Research map / Architectures and efficiency / RNNs and normalization
OCCAM: Towards Cost-Efficient and Accuracy-Aware Classification Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OCCAM, a training-free method that estimates per-query classifier accuracy via nearest-neighbour samples under well-separation and solves an ILP to assign classifiers to queries under cost budgets, achieving up to 40% cost reduction with little accuracy loss.
Problem
Choosing which pre-trained classifier (small/cheap vs. large/expensive) to apply to each query so that aggregate accuracy is maximized under a user-specified inference cost budget, without a learned router and with statistical guarantees.
Approach
Estimate each classifier's per-query success probability by averaging labelled-sample accuracies of nearest neighbours (justified by Lipschitz continuity under r-separation), regularize by estimator standard deviation, and assign classifiers to queries via an integer linear program under a cost budget.
Key results
- On CIFAR-10, OCCAM achieves 40% cost reduction with 0.51% accuracy drop (vs 2.22% for Single Best and 1.22% for FrugalMCT).
- On Tiny ImageNet at 40% cost reduction, OCCAM has 2.75% accuracy drop vs 4.01% for Single Best, 7.03% for Random, and 3.88% for FrugalMCT.
- On ImageNet-1K with 101 pre-trained classifiers, OCCAM keeps accuracy drop under 1% across 10-40% cost reductions while Random drops up to 7.05% and Single Best up to 15.09%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OCCAM instead of FrugalMCT | Image classification · CIFAR10, 10%-40% cost reduction | accuracy drop (%): 0.97, 1.13, 1.22 -> 0.56, 0.50, 0.51 | Helped |
| Use OCCAM instead of FrugalMCT | Image classification · CIFAR100, 10%-40% cost reduction | accuracy drop (%): 0.52, 0.79, 1.98 -> 0.34, 0.36, 0.62 | Helped |
| Use OCCAM instead of FrugalMCT | Image classification · Tiny ImageNet-200, 10%-40% cost reduction | accuracy drop (%): 0.86, 1.49, 3.88 -> 0.17, 0.61, 2.75 | Helped |
| Use OCCAM instead of FrugalMCT | Image classification · ImageNet-1K, 10%-40% cost reduction | accuracy drop (%): 0.59, 1.12, 2.35 -> 0.51, 1.05, 2.24 | Helped |
| Use OCCAM with SwinV2-T rather than ResNet-18 features (ablation) | OCCAM · Tiny ImageNet-200, 10%-40% cost reduction | accuracy drop (%): 0.48 -> 0.29; 3.24 -> 2.81; 3.24 -> 2.81 | Mixed |
| Use ResNet-50 features rather than ResNet-18 features (ablation) | OCCAM · Tiny ImageNet-200, 10%-40% cost reduction | accuracy drop (%): 0.48 -> 0.40; 1.02 -> 0.74; 3.24 -> 2.56 | Mixed |
| Use SwinV2-T rather than ResNet-18 features (ablation) | OCCAM · Tiny ImageNet-200, 10%-40% cost reduction | accuracy drop (%): 0.48 -> 0.29; 1.02 -> 0.58; 3.24 -> 2.81 | Mixed |
| Use l1 rather than l∞ distance (ablation) | OCCAM with ResNet-18 features · ImageNet-1K, 10%-40% cost reduction | accuracy drop (%): 0.51, 1.05, 2.24 -> 0.86, 1.51, 3.32 | Hurt |
| Use l2 rather than l∞ distance (ablation) | OCCAM with ResNet-18 features · ImageNet-1K, 10%-40% cost reduction | accuracy drop (%): 0.51, 1.05, 2.24 -> 1.06, 1.65, 3.10 | Hurt |
| Increase K while holding s fixed (ablation) | OCCAM · Tiny ImageNet-200 | accuracy | Helped |
| Increase s while holding K fixed (ablation) | OCCAM · Tiny ImageNet-200 | accuracy | Helped |
| Increase λ above 0 for regularization (ablation) | OCCAM · Tiny ImageNet-200, cost budgets B=0.6 and B=0.8 | accuracy: +1.3%; +0.8% | Helped |
Surprising findings
- Single Best accuracy oscillates heavily with cost on the 101-model ImageNet setting because some expensive models are less accurate than cheaper ones.
- FrugalMCT outperforms OCCAM on ImageNet-1K under l1 and l2 metrics, while OCCAM wins under l-infinity.
Limitations
- Assumes well-separated classification tasks and IID data, which may not hold for other domains such as sentiment analysis or under data drift.
- Optimizes average accuracy rather than worst-case accuracy, which may be inadequate for sensitive applications.
- Gap between OCCAM and the Optimal (true-likelihood) assignment indicates substantial room lost to estimation error.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny ImageNet ImageNet-1K
Benchmarks: CIFAR-10 CIFAR-100 Tiny ImageNet ImageNet-1K
Metrics: accuracy drop (%) at cost reduction levels normalized cost estimation error
Baselines: Single Best Random FrugalMCT
Builds on: FrugalMCT r-separation multiple choice knapsack problem
Closest papers by meaning
- Class-Attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective (AAAI 2024 · similarity 0.82)
- Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks using the Marginal Likelihood (NeurIPS 2024 · similarity 0.82)
- In-Context Learning and Occam's Razor (ICML 2025 · similarity 0.82)
- What Does It Take to Build a Performant Selective Classifier? (NeurIPS 2025 · similarity 0.81)
- Active Learning with LLMs for Partially Observed and Cost-Aware Scenarios (NeurIPS 2024 · similarity 0.81)
- Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning (NeurIPS 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.