Research map / Vision-language and multimodal / CLIP and zero-shot
Enhancing Few-Shot Class-Incremental Learning via Training-Free Bi-Level Modality Calibration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BiMC, a training-free few-shot class-incremental learning framework that calibrates CLIP's textual and visual prototypes (intra- and inter-modal) plus covariance and nearest-neighbor metrics, treating CLIP as a black box.
Problem
Few-shot class-incremental learning (FSCIL), where models must learn new classes from very few samples without forgetting old ones, usually requiring additional training in base or incremental sessions.
Approach
Uses frozen CLIP with LLM-generated fine-grained class descriptions for textual prototypes and base-session visual prototypes to recalibrate novel-class prototypes, fuses the two modalities via a mixing coefficient, and combines a shared global covariance metric with a category-conditioned nearest-neighbor metric through masked ensemble inference, with no gradient updates.
Key results
- On miniImageNet, BiMC† reaches 93.60 average accuracy with 3.07 performance degradation, versus 88.76 for CLIP zero-shot and 90.66 for FeCAM.
- BiMC† final-session accuracy on miniImageNet is 92.40, matching or exceeding trainable methods CPE-CLIP (82.77), CLIP-M3 (92.50 at 46k trainable params) and LP-DiF (91.68).
- Inter-modal calibration combined with intra-modal calibration yields the best ablation result (92.97 average on miniImageNet with BiMC alone).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BiMC instead of FeCAM | CLIP ViT-B/16 FSCIL · miniImageNet, all sessions | average accuracy (%): 90.66 -> 92.97 | Helped |
| Add ensemble classifier strategy (ablation) | BiMC · miniImageNet, all sessions | average accuracy (%): 92.97 -> 93.60 | Helped |
| Add intra-modal calibration to the visual modality (ablation) | Visual Prototype · miniImageNet | average accuracy (%): 88.55 -> 88.92 | Helped |
| Add intra-modal calibration to the language modality (ablation) | CLIP Zero-Shot · miniImageNet | average accuracy (%): 88.76 -> 91.05 | Helped |
| Add inter-modal calibration to both modalities with intra-modal calibration (ablation) | BiMC · miniImageNet | average accuracy (%): 92.69 -> 92.97 | Helped |
| Add modeling global covariance (MGC) (ablation) | BiMC · miniImageNet | average accuracy (%): 92.97 -> 92.91 | Hurt |
| Add cross-modal category nearest-neighbor metric (CMNN) (ablation) | BiMC · miniImageNet | average accuracy (%): 92.97 -> 93.18 | Helped |
| Use ensemble inference without category masks (ablation) | BiMC† · miniImageNet | average accuracy (%): 93.60 -> 93.37 | Hurt |
| Add category masks to ensemble inference (ablation) | BiMC† · miniImageNet | novel-class accuracy (%): 89.32 -> 91.20 | Helped |
Surprising findings
- Base-session covariance matrices perform poorly for novel classes, so nearest-neighbor scoring is used for novel classes instead of covariance.
- Calibrated classifier reduces prediction entropy relative to uni-modal textual and visual classifiers.
Limitations
- Relies on fixed hyperparameters (lambda, alpha, beta) tuned on base-session validation, with alpha chosen differently per dataset.
- Evaluated only on three standard FSCIL benchmarks with a single CLIP ViT-B/16 backbone.
Experimental setup
Datasets: CIFAR100 CUB200-2011 miniImageNet
Benchmarks: CIFAR100 CUB200-2011 miniImageNet
Metrics: session-wise accuracy average accuracy performance degradation
Baselines: CLIP Zero-Shot Visual Prototype TEEN FeCAM CPE-CLIP CLIP-M3 LP-DiF
Builds on: CLIP TEEN FeCAM CuPL
Closest papers by meaning
- Compositional Few-Shot Class-Incremental Learning (ICML 2024 · similarity 0.86)
- VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning (NeurIPS 2025 · similarity 0.86)
- Rethinking Few-Shot Adaptation of Vision-Language Models in Two Stages (CVPR 2025 · similarity 0.86)
- Pre-trained Vision and Language Transformers Are Few-Shot Incremental Learners (CVPR 2024 · similarity 0.86)
- Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learning (CVPR 2025 · similarity 0.86)
- Post-pre-training for Modality Alignment in Vision-Language Foundation Models (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.