AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

SD-LoRA: Training-Time Structural Distillation into LoRA for Few-Shot Vision-Language Adaptation

Mohammed Rahman Sherif Khan Mohammad, Ardhendu Behera, Sandip Pradhan, Swagat Kumar, Yonghuai Liu

NeurIPS 2026 · Vision-language and multimodal › CLIP and zero-shot · approach: Test-time adaptation of VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SD-LoRA, which adapts frozen CLIP-like vision-language models for few-shot tasks by distilling token-level structure into LoRA during training.

Problem

Few-shot adaptation of vision-language models must learn fine-grained visual distinctions from very limited supervision while preserving deployment efficiency; existing lightweight methods supervise only global image embeddings and underuse token-level information.

Approach

Training-time structural distillation that transfers token-level structure into LoRA weights, so no extra inference cost is added; the abstract indicates it targets the underuse of token-level supervision in global-embedding-based adapters.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add training-time graph teacher with implicit distillation to CLIP-LoRACLIP-LoRA · 11 few-shot benchmarks, 16-shotaccuracy: 83.0 -> 85.0Improved
add graph teacher on top of LoRA+Cache (ablation)LoRA + Cache · Caltech101 4-shotaccuracy: 93.0 -> 96.2Improved
add cache head on top of LoRA+Graph (ablation)LoRA + Graph · EuroSAT 16-shotaccuracy: 93.1 -> 94.0Improved
structural training versus matched 336-trained cache baseline (ablation)SD-LoRA · 4 datasets avg, 1-shotaccuracy: 71.3 -> 73.1Improved
higher training resolution (224 to 336) without structural supervision (ablation)CLIP-LoRA+cache · 4 datasets avg, 1-shotaccuracy: 70.8 -> 71.3Improved
structural distillation on base-to-novel transferSD-LoRA · Base-to-novel avg 11, novel classesaccuracy: 80.73 -> 86.59Improved
structural distillation on base-class accuracySD-LoRA · Base-to-novel EuroSAT/Flowers/DTD baseaccuracy: -0.25 EuroSAT, -1.22 Flowers, -1.16 DTDMixed
structural distillation on ImageNet OOD variantsSD-LoRA · ImageNet-Aaccuracy: 49.63 vs MaPLe 50.90No clear effect
structural distillation on OOD averageSD-LoRA · ImageNet OOD avgaccuracy: 62.85 -> 63.26Improved
structural distillation effect on calibrationSD-LoRA · Cross-dataset transferECE: 5.57 -> 2.65Improved
gains shrink with more shotsSD-LoRA · 11 few-shot benchmarks vs strongest baselineaccuracy: +2.4 (1-shot) to +0.4 (16-shot)Mixed

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 2 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Experimental setup

Baselines: prompt tuning LoRA cache-based adapters

Builds on: LoRA CLIP

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy