Research map / Label-efficient and robust learning / Recognition and long-tail
CAME: Contrastive Automated Model Evaluation
Ru Peng, Qiuyang Duan, Haobo Wang, Jiachen Ma, Yanbo Jiang, Yongjun Tu, Xiu Jiang, Junbo Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use contrastive-accuracy regression (CAME) instead of prior AutoEval methods | CAME · AutoEval across MNIST, CIFAR, COCO, TinyImageNet | MAE: reduces error by about 47.2% on average vs prior SOTA | Improved |
| train with multi-task CE + contrastive loss instead of pre-train then fine-tune (ablation) | CAME · CIFAR-10 | MAE: 1.00 -> 0.49 | Improved |
| co-train with contrastive loss rather than classification only (ablation) | classifier under evaluation · SVHN, USPS, Pascal, Caltech, ImageNet | classification accuracy: USPS 64.08 -> 88.64; SVHN 25.46 -> 23.42; ImageNet 88.83 -> 82.66 | Mixed |
| lower contrastive loss weight to 0.001 (vs 1.0, 0.1) (ablation) | CAME · CIFAR-10 | MAE: MAE 0.71 (1.0), 2.87 (0.1), 0.91 (0.01), 0.49 (0.001) | Mixed |
| swap SimCLR for MoCo-v1/v2 or BYOL (ablation) | CAME · CIFAR-10 | MAE: SimCLR 0.49 vs MoCo-v1 1.20, MoCo-v2 0.88, BYOL 0.59 | Hurt |
| use other backbones instead of DenseNet-40-12 (ablation) | CAME · CIFAR-10 | MAE: 0.49 -> 0.63 to 1.17 (ResNet-18 0.92, VGG-19 0.77) | Hurt |
| change random seed from 0 to 21 or 42 (ablation) | CAME · CIFAR-10 | MAE: 0.49 -> 0.65 (seed 21), 0.51 (seed 42) | No clear effect |
| deviate from default SimCLR RandomResizedCrop strength (0.92), ColorJitter and temperature 0.07 (ablation) | CAME · CIFAR-10 | MAE | Hurt |
| vary number and size of synthetic sample sets (ablation) | CAME · CIFAR-10 | MAE | No clear effect |
Closest papers by meaning
- Energy-based Automated Model Evaluation (ICLR 2024 · similarity 0.88)
- AutoEval Done Right: Using Synthetic Data for Model Evaluation (ICML 2025 · similarity 0.86)
- Adaptive Prediction-Powered AutoEval with Reliability and Efficiency Guarantees (NeurIPS 2025 · similarity 0.86)
- Autonomous Evaluation of LLMs for Truth Maintenance and Reasoning Tasks (ICLR 2025 · similarity 0.85)
- From Jack of All Trades to Master of One: Specializing LLM-based Autoraters to a Test Set (ICML 2025 · similarity 0.84)
- Reproduce, Replicate, Reevaluate. The Long but Safe Way to Extend Machine Learning Methods (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
