Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Be Confident in What You Know: Bayesian Parameter Efficient Fine-Tuning of Vision Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Bayesian-PEFT, which adds evidential base-rate adjustment and a belief-regularization diversity ensemble to PEFT (VPT, adapter, bias) on ViT to fix severe underconfidence in few-shot adaptation.
Problem
PEFT-adapted vision foundation models are accurate but severely underconfident in few-shot settings, and lack fine-grained uncertainty quantification.
Approach
Adjusts the subjective-logic base rate of an evidential head with a transformation (exponent m) that widens the Dirichlet parameter gap between top classes without changing class ordering, plus an ensemble of evidential models trained with different incorrect-evidence regularization strengths to induce SVGD-like diversity.
Key results
- On 1-shot CIFAR-100, B-PEFT reduces ECE to 0.067 versus 0.393 for the standard VPT model, with accuracy 52.3% vs 48.6%.
- Base-rate adjusted evidential model keeps accuracy equal to the evidential model while cutting 1-shot CIFAR-10 ECE from 0.557 to 0.027.
- B-PEFT achieves lowest ECE among compared calibration methods (TS, PTS, IR-MC) at all CIFAR-100 shot settings, e.g., 0.031 at 10-shot vs 0.036 for TS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add base-rate adjustment to the evidential model (ablation) | Evidential PEFT · Cifar100 1-shot | ECE: 0.499±0.004 -> 0.077±0.004 | Helped |
| Add base-rate adjustment to the evidential model (ablation) | Evidential PEFT · Cifar100 1-shot | accuracy: 51.127±0.435 -> 51.127±0.435 | No clear effect |
| Add a diversity-inducing evidential ensemble (ablation) | Base-rate adjusted evidential PEFT · Cifar100 1-shot | accuracy: 51.127±0.435 -> 52.335±0.610 | Helped |
| Add a diversity-inducing evidential ensemble (ablation) | Base-rate adjusted evidential PEFT · Cifar100 1-shot | ECE: 0.077±0.004 -> 0.067±0.001 | Helped |
| Increase base-rate transformation order m (ablation) | Evidential PEFT · Cifar100 1-shot | ECE: 0.261±0.007 -> 0.092±0.008 -> 0.079±0.003 -> 0.156±0.007 | Mixed |
| Use Bias fine-tuning instead of VPT (ablation) | B-PEFT · Cifar100 1-shot | accuracy: 52.335±0.610 -> 37.825±0.344 | Hurt |
| Use Adapter fine-tuning instead of VPT (ablation) | B-PEFT · Cifar100 1-shot | accuracy: 52.335±0.610 -> 48.732±0.225 | Hurt |
Surprising findings
- Standard PEFT-adapted foundation models are severely underconfident rather than overconfident in few-shot settings, unlike typical deep models.
- Evidential models trained with the standard approach are even more underconfident than CE-trained VPT models.
Limitations
- Gains reported mainly on four image classification datasets with a single supervised ViT backbone.
- Post-hoc hyperparameter m must be chosen by grid search on a 2-shot validation set and interacts with regularization strength.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Food101 Flowers102
Benchmarks: CIFAR-10 CIFAR-100 Food101 Flowers102
Metrics: accuracy expected calibration error vacuity ensemble variance
Baselines: Temperature Scaling Parameterized Temperature Scaling Isotonic Regression (IR-MC) Standard CE model (VPT) Evidential Model
Builds on: Evidential Deep Learning Visual Prompt Tuning Subjective logic Stein Variational Gradient Descent
Closest papers by meaning
- Strong Baselines for Parameter-Efficient Few-Shot Fine-Tuning (AAAI 2024 · similarity 0.89)
- Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual Recognition (CVPR 2025 · similarity 0.87)
- ExPLoRA: Parameter-Efficient Extended Pre-Training to Adapt Vision Transformers under Domain Shifts (ICML 2025 · similarity 0.87)
- Structured Unrestricted-Rank Matrices for Parameter Efficient Finetuning (NeurIPS 2024 · similarity 0.86)
- Parameter Efficient Fine-tuning via Explained Variance Adaptation (NeurIPS 2025 · similarity 0.86)
- Sensitivity-Aware Efficient Fine-Tuning via Compact Dynamic-Rank Adaptation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.