Research map / Probabilistic and scientific modeling / Molecules and proteins
Great Models Think Alike: Improving Model Reliability via Inter-Model Latent Agreement
Ailin Deng, Miao Xiong, Bryan Hooi
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add agreement score (CLIP ViT/L-14 + ViT/L-16 multiple foundation models) to temperature scaling | MSP/temperature scaling failure detection · ID datasets (CIFAR10, CIFAR100, STL, BIRDS, FOOD, ImageNet), CNN and ViT | AUROC (%): e.g. CIFAR10 CNN 93.76 -> 98.07 | Improved |
| add agreement score with a single foundation model (CLIP ViT/L-14) to temperature scaling | temperature scaling · ID datasets, CNN and ViT | AUROC (%): e.g. CIFAR100 CNN 87.21 -> 89.63 | Improved |
| add agreement score with multiple foundation models vs single (ablation) | temperature scaling with agreement score · ID datasets, CNN and ViT | AUROC (%): multiple better on CIFAR10/CIFAR100/BIRDS/ImageNet CNN; single better on FOOD (92.33 vs 92.17 CNN, 93.81 vs 93.64 ViT) | Mixed |
| add agreement score to temperature scaling | temperature scaling under distribution shift · CIFAR10.1, CIFAR10-C, CIFAR100-C, ImageNetV2, ImageNet-Sketch | AUROC (%): e.g. CIFAR10-C CNN 78.14 -> 92.76; ImageNetV2 ViT 83.68 -> 84.77 | Improved |
| replace NDCG agreement with Spearman rank correlation (ablation) | agreement measure · ID average over datasets | Avg AUROC (%): 92.67 -> 90.87 | Hurt |
| replace NDCG agreement with CKA (ablation) | agreement measure · ID average over datasets | Avg AUROC (%): 92.67 -> 90.99 | Hurt |
| replace NDCG agreement with Jaccard similarity of k-hop neighborhoods (ablation) | agreement measure · ID average over datasets | Avg AUROC (%): 92.67 -> 92.59 | Hurt |
| increase neighbor candidate pool size n from 2000 to 50000 (ablation) | agreement score with CIFAR10/CIFAR100 neighbors · CIFAR10 and CIFAR100 | AUROC (%): CIFAR10 97.26 -> 97.50; CIFAR100 89.08 -> 90.38 | Improved |
| use a small neighbor pool (n=2000) instead of larger pools (ablation) | agreement score · CIFAR10 and CIFAR100 | AUROC (%): still above MSP (94.15 / 88.42) and T.S. baselines | No clear effect |
Closest papers by meaning
- Improved Training Technique for Latent Consistency Models (ICLR 2025 · similarity 0.86)
- Quantifying Uncertainty in Error Consistency: Towards Reliable Behavioral Comparison of Classifiers (NeurIPS 2025 · similarity 0.85)
- Predicting the Performance of Foundation Models via Agreement-on-the-Line (NeurIPS 2024 · similarity 0.84)
- Improved Techniques for Training Consistency Models (ICLR 2024 · similarity 0.84)
- SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.83)
- Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
