Research map / Label-efficient and robust learning / Semi-supervised learning
Learning Textual Prompts for Open-World Semi-Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TP-OWSSL, which uses global-and-local learnable textual prompts plus a forward-and-backward (teacher pseudo-label plus CLIPN validation) strategy to improve image-text alignment for open-world semi-supervised learning on fine-grained datasets.
Problem
Open-world semi-supervised learning must recognize known and unknown classes, and existing text-enhanced methods fail to align images with text well and suffer from noisy image-text matching on unlabeled samples, hurting discrimination of visually similar classes.
Approach
Learn global textual prompts aligned with global visual features and multiscale local textual prompts aligned with top-K local visual features (from the second-to-last ViT block) to filter background noise, and filter unlabeled pseudo-label assignments using a CLIPN-based reliability check (forward teacher matching plus backward negation validation).
Key results
- TP-OWSSL reaches 80.6% all-class accuracy on CUB versus 76.6% for TextGCD (CLIP backbone).
- On Stanford Cars, TP-OWSSL achieves 90.7% all-class accuracy versus 86.9% for TextGCD.
- On Flowers, TP-OWSSL reaches 87.3% all-class accuracy, above TextGCD's 87.2%, while on Oxford Pets it scores 93.9% versus TextGCD's 95.5%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use global-and-local textual prompt learning and forward-and-backward matching validation | TP-OWSSL · CUB, all classes | ACC: 76.6 -> 80.6 | Helped |
| Use global-and-local textual prompt learning and forward-and-backward matching validation | TP-OWSSL · Stanford Cars, all classes | ACC: 86.9 -> 90.7 | Helped |
| Use global-and-local textual prompt learning and forward-and-backward matching validation | TP-OWSSL · Flowers, all classes | ACC: 87.2 -> 87.3 | Helped |
| Use global-and-local textual prompt learning and forward-and-backward matching validation | TP-OWSSL · Oxford Pets, all classes | ACC: 95.5 -> 93.9 | Hurt |
| Add the forward-and-backward strategy to the labeled-only baseline (ablation) | TP-OWSSL · Flowers | accuracy | Helped |
| Add backward reliability validation to forward matching (ablation) | TP-OWSSL · Flowers | accuracy | Helped |
| Add global-and-local textual prompt learning to the forward-and-backward baseline (ablation) | TP-OWSSL · Multiple datasets | accuracy | Helped |
| Add local visual features and local textual prompts (ablation) | TP-OWSSL · Multiple datasets | accuracy | Helped |
| Use multiple scales of local textual prompts instead of a single scale (ablation) | TP-OWSSL · Oxford Pets | accuracy | Helped |
| Vary the number of local textual prompts from 1 to 5 (ablation) | TP-OWSSL · Oxford Pets | accuracy | No clear effect |
| Vary K1 among 1, 50, 100, and 150 (ablation) | TP-OWSSL · Oxford Pets | accuracy | No clear effect |
Surprising findings
- On Oxford Pets, TP-OWSSL (93.9% all-class) underperforms TextGCD (95.5%) and the SimGCD CLIP baseline's unknown-class accuracy is higher than its known-class accuracy, indicating mixed gains across datasets.
Limitations
- Gains are mixed: on Oxford Pets the method is below TextGCD in all-class accuracy.
- Requires a known number of unknown classes, a fundamental assumption of the setting.
- Relies on an external teacher CLIP and CLIPN validation model, and the authors note future work on continuously emerging unknown classes.
Experimental setup
Datasets: CUB Stanford Cars Flowers Oxford Pets
Benchmarks: CUB Stanford Cars Flowers Oxford Pets
Metrics: ACC (all, known, unknown) clustering accuracy (CACC)
Baselines: ORCA GCD SimGCD GPC DCCL PromptCAL CLIP-GCD TextGCD
Builds on: CoOp CoCoOp GalLoP TextGCD CLIPN
Closest papers by meaning
- OwMatch: Conditional Self-Labeling with Consistency for Open-World Semi-Supervised Learning (NeurIPS 2024 · similarity 0.87)
- GOAL: Global-local Object Alignment Learning (CVPR 2025 · similarity 0.86)
- Learning to Prompt with Text Only Supervision for Vision-Language Models (AAAI 2025 · similarity 0.86)
- Targeted Representation Alignment for Open-World Semi-Supervised Learning (CVPR 2024 · similarity 0.85)
- A Similarity Paradigm Through Textual Regularization Without Forgetting (AAAI 2025 · similarity 0.85)
- SemiETS: Integrating Spatial and Content Consistencies for Semi-Supervised End-to-end Text Spotting (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.