Research map / Label-efficient and robust learning / Recognition and long-tail
Exploiting Descriptive Completeness Prior for Cross Modal Hashing with Incomplete Labels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PCRIL, a CLIP-based prompt contrastive recovery framework that recovers unknown labels via learnable label-set prompts and tree search, plus augmentation for pairwise similarity, for cross-modal hashing with incomplete labels.
Problem
Cross-modal hashing degrades when many labels are unknown, because unknown labels make positive and negative image-text pairs uncertain and negative pairs become scarce.
Approach
Learns CLIP text prompts that encode label subsets and contrastively separates an anchor positive label set from negative variants (deletion, addition, replacement) to score label-set completeness; a greedy tree search over these scores recovers missing positives, and complementary mixup plus adaptive negative masking restore pairwise similarity supervision.
Key results
- On Flickr at 30% known labels, PCRIL reaches 77.0 mean mAP at 32 bits versus 64.1 for DCHMT (mean over i-to-t and t-to-i across known ratios).
- Averaged across datasets and known ratios, PCRIL improves about 12.13% mAP over DCHMT, the CLIP-based baseline.
- Learned prompts raise recovery precision on Flickr to 89.7% on average versus 87.5% for conventional prompts and 67.3% for phrasal prompts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PCRIL rather than the strongest listed baseline | Cross-modal hashing with incomplete labels · Flickr, NUS, and COCO at 30%, 50%, and 70% known labels | mAP: 81.4 (4.1) | Helped |
| Add adaptive negative masking to the baseline (ablation) | Baseline CMH · Flickr, NUS, and COCO at 30%, 50%, and 70% known labels | mAP: +6.66% and +6.09% mAP | Helped |
| Add prompt contrastive recovery (ablation) | Baseline CMH with ANM · Flickr, NUS, and COCO | mAP: 3.52% | Helped |
| Add complementary semantic augmentation (ablation) | Baseline CMH with ANM and PCR · Flickr, NUS, and COCO | mAP: 0.60% | Helped |
| Replace ANM with ignoring unobserved pair relationships (ablation) | Baseline CMH · Flickr (30% known) | mAP: 75.0 -> 57.5 | Hurt |
| Replace ANM with assuming all unknown pairs are negative (ablation) | Baseline CMH · Flickr (30% known) | mAP: 75.0 -> 68.9 | Hurt |
| Use learned multi-label prompts instead of a handcrafted phrasal prompt (ablation) | PCRIL · Flickr (30% known) | mAP: 75.0 -> 77.0 | Helped |
| Use recursive label-tree search instead of one-step recovery (ablation) | PCRIL · Flickr (30% known) | mAP: 64.6 -> 77.0 | Helped |
| Use both modalities for recovery rather than a single modality (ablation) | PCRIL · Flickr | mAP: By image 78.2; By text 74.3; Ours 77.0 | Mixed |
| Increase contrastive margin (ablation) | PCRIL prompt contrastive recovery · Flickr | mAP: 0.827 -> 0.814 | Hurt |
Surprising findings
- Original CLIP zero-shot label recovery yields only about 68% precision even at 30% unknown labels on Flickr, and can degrade final performance.
- Text-only recovery has low precision (about 77%) while joint image-text recovery reaches about 90%.
Limitations
- Prompt construction is limited by CLIP's small text token capacity, which hinders richly labeled samples.
- PCR relies on sufficient multi-labeled samples and struggles with single-positive cases.
- Evaluated only with frozen ViT-B/32 CLIP and image-text pairs.
Experimental setup
Datasets: MIRFlickr-25k NUS-WIDE MS COCO IAPR TC-12
Benchmarks: MIRFlickr-25k NUS-WIDE MS COCO IAPR TC-12
Metrics: mAP precision of recovered positive labels
Baselines: DCH SDMCH SCRATCH WCHash DCMH SSAH AGAH DCHMT IU AN
Builds on: CLIP DCMH CoOp-style learnable prompts mixup Assume Negative baseline
Closest papers by meaning
- Robust Self-Paced Hashing for Cross-Modal Retrieval with Noisy Labels (AAAI 2025 · similarity 0.89)
- PromptHash:Affinity-Prompted Collaborative Cross-Modal Learning for Adaptive Hashing Retrieval (CVPR 2025 · similarity 0.88)
- Dual Self-Paced Cross-Modal Hashing (AAAI 2024 · similarity 0.87)
- An End-To-End Graph Attention Network Hashing for Cross-Modal Retrieval (NeurIPS 2024 · similarity 0.87)
- DiCA: Disambiguated Contrastive Alignment for Cross-Modal Retrieval with Partial Labels (AAAI 2025 · similarity 0.87)
- Distribution-Consistency-Guided Multi-modal Hashing (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.