Research map / Optimization / Optimizers
LP++: A Surprisingly Strong Linear Probe for Few-Shot CLIP
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LP++, a generalized linear probe for few-shot CLIP whose classifier weights blend visual prototypes with learnable class-wise text-embedding multipliers, optimized by a block-coordinate Majorize-Minimize method with data-driven Lipschitz step sizes.
Problem
Few-shot adaptation of CLIP: the standard linear probe is reported as a weak baseline that ignores text knowledge and requires costly hyperparameter tuning in competing prompt-learning and adapter methods.
Approach
Extend linear-probe logits to f^T(w_k + alpha_k t_k) with learnable class-wise blending parameters alpha_k, and optimize with a block coordinate MM descent whose step sizes come from closed-form Lipschitz constants and whose initialization comes from approximate minima, removing learning-rate and validation grid searches.
Key results
- LP++ averages 63.43% at 1 shot and 74.42% at 16 shots over 11 datasets, versus 36.10% and 70.56% for standard LP with ResNet-50 CLIP.
- LP++ matches or beats Tip-Adapter-F* (63.06% at 1 shot, 74.37% at 16 shots) and prompt-learning methods such as PLOT and ProGrad at most shot counts.
- On 16-shot ImageNet, LP++ takes about 2 seconds versus roughly 17 hours for CoOp and 40 minutes for CLIP-Adapter.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LP++ with learnable class-wise blending of image and text knowledge | CLIP ResNet-50 · 11 benchmarks, 1-shot | accuracy: 62.46 ± 1.89 -> 63.43 ± 0.90 | Helped |
| use LP++ with learnable class-wise blending of image and text knowledge | CLIP ResNet-50 · 11 benchmarks, 2-shot | accuracy: 66.47 ± 0.65 -> 66.20 ± 0.72 | Hurt |
| use LP++ with learnable class-wise blending of image and text knowledge | CLIP ResNet-50 · 11 benchmarks, 4-shot | accuracy: 68.71 ± 0.96 -> 69.16 ± 0.79 | Helped |
| use LP++ with learnable class-wise blending of image and text knowledge | CLIP ResNet-50 · 11 benchmarks, 8-shot | accuracy: 71.96 ± 0.70 -> 72.04 ± 0.46 | Helped |
| use LP++ with learnable class-wise blending of image and text knowledge | CLIP ResNet-50 · 11 benchmarks, 16-shot | accuracy: 74.37 ± 0.35 -> 74.42 ± 0.45 | Helped |
| add text knowledge by learning class-wise blending parameters instead of setting αk = 0 (ablation) | LP++ · 11 benchmarks, 1-shot | accuracy: 35.55 -> 63.43 | Helped |
| add text knowledge by learning class-wise blending parameters instead of setting αk = 0 (ablation) | LP++ · 11 benchmarks, 16-shot | accuracy: 69.75 -> 74.42 | Helped |
| learn class-wise blending parameters instead of fixing αk = 1 (ablation) | LP++ · 11 benchmarks, 1-shot | accuracy: 46.37 -> 63.43 | Helped |
| learn class-wise blending parameters instead of fixing αk = 1 (ablation) | LP++ · 11 benchmarks, 16-shot | accuracy: 69.41 -> 74.42 | Helped |
| use block-coordinate MM instead of single-block gradient descent (ablation) | LP++ · 11 benchmarks, 1-shot | accuracy: 63.04 -> 63.43 | Helped |
| use block-coordinate MM instead of single-block gradient descent (ablation) | LP++ · 11 benchmarks, 16-shot | accuracy: 74.28 -> 74.42 | Helped |
| use the proposed data-informed initialization instead of random initialization (ablation) | LP++ training-free prediction · 11 benchmarks, 1-shot | accuracy: 17.79 -> 59.70 | Helped |
Surprising findings
- Standard linear probe, when given the proposed text-blending and optimization choices, rises from 36.10% to 63.43% at 1 shot, contradicting its reputation as a weak baseline.
- Adam with the Lipschitz-derived learning rate performs poorly (25.44% at 1 shot with the proposed loss), while GD with the same step size matches the grid-searched optimum.
Limitations
- Evaluated only with a ResNet-50 CLIP backbone in the main experiments.
- Relies on the assumption that the MM majorizers can be minimized to global optimality, which holds for convex objective pieces but is only approximated in practice.
Experimental setup
Datasets: ImageNet Caltech101 DTD OxfordPets Flowers102 StanfordCars Food101 FGVCAircraft SUN397 EuroSAT UCF101
Benchmarks: ImageNet Caltech101 DTD OxfordPets Flowers102 StanfordCars Food101 FGVCAircraft SUN397 EuroSAT UCF101
Metrics: average classification accuracy run time
Baselines: Zero-shot CLIP Standard LP CoOp PLOT KgCoOp ProGrad CLIP-Adapter Tip-Adapter-F
Builds on: CLIP CoOp Tip-Adapter Majorize-Minimize
Closest papers by meaning
- A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation (ICLR 2024 · similarity 0.89)
- Text and Image Are Mutually Beneficial: Enhancing Training-Free Few-Shot Classification with CLIP (AAAI 2025 · similarity 0.87)
- Transductive Zero-Shot and Few-Shot CLIP (CVPR 2024 · similarity 0.87)
- Diffusion Feedback Helps CLIP See Better (ICLR 2025 · similarity 0.86)
- Learning to Adapt Frozen CLIP for Few-Shot Test-Time Domain Adaptation (ICLR 2025 · similarity 0.86)
- AmorLIP: Efficient Language-Image Pretraining via Amortization (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.