Research map / Vision-language and multimodal / CLIP and zero-shot
Cross-Modal Few-Shot Learning with Second-Order Neural Ordinary Differential Equations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SONO, which adapts frozen CLIP features with a second-order neural ODE refiner and a text-initialized linear classifier, plus text-as-image augmentation, for few-shot recognition.
Problem
Few-shot adaptation of CLIP suffers from overfitting and the cost of prompt tuning or large-cache adapters; the paper asks whether efficient learnable modules can give strong few-shot accuracy.
Approach
Refine frozen CLIP image features with a second-order Neural ODE (via first-order state augmentation), initialize the classifier from CLIP text embeddings of class prompts so the text encoder need not run each iteration, and augment training with prompts most similar to the few-shot image prototypes ("text-as-image").
Key results
- SONO reaches 77.86% average 16-shot accuracy across 11 datasets, outperforming GraphAdapter by 1.63% and Tip-Adapter by 2.75%.
- On 16-shot ImageNet with ResNet-50, SONO achieves 66.37% vs Tip-Adapter-F 65.51%, trained in 3.5 minutes with 1.54M parameters.
- Domain generalization: average OOD accuracy improves over GraphAdapter by 1.58% (ResNet-50) and 1.80% (ViT-B/16).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Text-Image Augmentation (TAI) (ablation) | SONO · ImageNet 16-shot | accuracy: 65.66 -> 66.37 | Helped |
| add Second-Order NODE model (SNM) (ablation) | SONO with TAI · ImageNet 16-shot | accuracy: 63.26 -> 66.37 | Helped |
| remove TAI and SNM, retaining only the classifier (ablation) | SONO · ImageNet 16-shot | accuracy: 66.37 -> 62.35 | Hurt |
| vary the residual ratio eta (ablation) | SONO · ImageNet 16-shot | accuracy: 64.06, 65.51, 66.20, 66.37, 66.12, 65.93 | Mixed |
| use RK4 instead of Euler, AB, or ABM ODE solver (ablation) | SONO · ImageNet 16-shot | accuracy | Helped |
| use SONO rather than GraphAdapter | CLIP ResNet-50 · ImageNet-V2, ImageNet-Sketch, ImageNet-A, ImageNet-R, and OOD average | accuracy: 57.83 vs 56.58; 37.07 vs 35.89; 27.02 vs 23.07; 60.75 vs 60.86; 45.68 vs 44.10 | Mixed |
| use SONO rather than GraphAdapter | CLIP ViT-B/16 · ImageNet-V2, ImageNet-Sketch, ImageNet-A, ImageNet-R, and OOD average | accuracy: 67.33 vs 66.60; 51.15 vs 49.23; 52.96 vs 50.75; 78.88 vs 77.73; 62.58 vs 60.78 | Helped |
Surprising findings
- Removing the Second-Order NODE component drops 16-shot ImageNet accuracy by 3.11%, larger than removing text-image augmentation (0.71%).
Limitations
- Efficiency and few-shot gains are reported mainly on ImageNet with limited ablations on other datasets.
- Few-shot results rely on a fixed hyperparameter setting (M=50, eta=0.6) chosen empirically.
Experimental setup
Datasets: ImageNet Caltech101 OxfordPets StanfordCars Flowers102 Food-101 FGVC Aircraft DTD SUN397 EuroSAT UCF101 ImageNet-V2
Benchmarks: ImageNet Caltech101 OxfordPets StanfordCars Flowers102 Food-101 FGVC Aircraft DTD SUN397 EuroSAT UCF101 ImageNet-V2
Metrics: accuracy average accuracy training time GFLOPs parameter count
Baselines: CLIP CoOp PLOT CLIP-Adapter Tip-Adapter-F TaskRes GraphAdapter
Builds on: Neural ODE CLIP CoOp Tip-Adapter CuPL
Closest papers by meaning
- Few-shot Learner Parameterization by Diffusion Time-steps (CVPR 2024 · similarity 0.85)
- Meta-Exploiting Frequency Prior for Cross-Domain Few-Shot Learning (NeurIPS 2024 · similarity 0.85)
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.84)
- VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning (NeurIPS 2025 · similarity 0.84)
- Curriculum Multiple Shooting for Robust Training of Neural and Universal Differential Equations (NeurIPS 2026 · similarity 0.84)
- Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data (ICLR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.