Research map / Label-efficient and robust learning / Semi-supervised learning
DREAM: Decoupled Discriminative Learning with Bigraph-aware Alignment for Semi-supervised 2D-3D Cross-modal Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DREAM, a semi-supervised 2D-3D cross-modal retrieval method that decouples label prediction (label propagation) from learned reliability scoring, adds multimodal negative learning, and aligns modalities via a cross-modal bigraph clustering loss.
Problem
2D-3D cross-modal retrieval under label scarcity, where pseudo-labeling with a single model causes overconfident errors and the heterogeneous 2D/3D gap hampers learning from unlabeled data.
Approach
Decouple label prediction from reliability measurement (learning to selectively learn) with class-specific adaptive thresholds, use negative labels from cross-modal disagreement for unselected samples, and align modalities by clustering a cross-modal bigraph that keeps most similar-pair edges within clusters.
Key results
- On 3D MNIST (2D→3D), DREAM averages 93.45 MAP vs. 93.10 for HOPE and 84.49 for DSCMR.
- On ModelNet10 (2D→3D), DREAM averages 86.17 MAP vs. 84.32 for HOPE.
- On ModelNet40 (2D→3D) with 800-3200 labels, DREAM averages 80.35 MAP vs. 74.79 for HOPE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DREAM instead of HOPE | 2D-3D cross-modal retrieval · 3D MNIST, 2D → 3D (Avg) | MAP: 93.10 -> 93.45 | Helped |
| Use DREAM instead of HOPE | 2D-3D cross-modal retrieval · ModelNet10, 2D → 3D (Avg) | MAP: 84.32 -> 86.17 | Helped |
| Use DREAM instead of HOPE | 2D-3D cross-modal retrieval · ModelNet40, 2D → 3D (Avg) | MAP: 74.79 -> 80.35 | Helped |
| Use DREAM instead of HOPE | 2D-3D cross-modal retrieval · 3D MNIST, 3D → 2D (Avg) | MAP: 91.88 -> 93.05 | Helped |
| Use DREAM instead of HOPE | 2D-3D cross-modal retrieval · ModelNet10, 3D → 2D (Avg) | MAP: 84.05 -> 86.57 | Helped |
| Use DREAM instead of HOPE | 2D-3D cross-modal retrieval · ModelNet40, 3D → 2D (Avg) | MAP: 74.12 -> 80.17 | Helped |
| Use decoupled discriminative learning instead of classifier-based or prototype-based pseudo-labeling (ablation) | DREAM · 2D-3D cross-modal retrieval | MAP | Helped |
| Use bigraph-aware alignment instead of pairwise-label or classwise-anchor alignment (ablation) | DREAM · 2D-3D cross-modal retrieval | MAP | Helped |
| Add the negative learning module (ablation) | DREAM · 2D-3D cross-modal retrieval | MAP | Helped |
| Vary the negative-learning weight α; best reported setting is α = 0.1 (ablation) | DREAM · ModelNet10 with 200 labeled samples | MAP | Mixed |
| Vary the alignment weight β; best reported setting is β = 0.4 (ablation) | DREAM · ModelNet10 with 200 labeled samples | MAP | Mixed |
Limitations
- Evaluated only on three object-level benchmark datasets with 2D/3D pairs drawn from ModelNet and 3D MNIST.
- Hyperparameters alpha and beta were tuned by sensitivity analysis on a single dataset setting.
Experimental setup
Datasets: 3D MNIST ModelNet10 ModelNet40
Benchmarks: 3D MNIST ModelNet10 ModelNet40
Metrics: MAP precision-recall curves precision-top N recall-top N
Baselines: DSCMR MRL ALGCN DA-I-GCN DA-I-GAT DA-P-GCN DA-P-GAT CLF RONO HOPE
Builds on: HOPE label propagation pseudo-labeling complementary/negative learning
Closest papers by meaning
- DiCA: Disambiguated Contrastive Alignment for Cross-Modal Retrieval with Partial Labels (AAAI 2025 · similarity 0.86)
- Cross-Modal and Uni-Modal Soft-Label Alignment for Image-Text Retrieval (AAAI 2024 · similarity 0.85)
- RoDA: Robust Domain Alignment for Cross-Domain Retrieval Against Label Noise (AAAI 2025 · similarity 0.85)
- Learning to Rematch Mismatched Pairs for Robust Cross-Modal Retrieval (CVPR 2024 · similarity 0.85)
- Multi-Level Cross-Modal Alignment for Image Clustering (AAAI 2024 · similarity 0.84)
- Beyond Modality Collapse: Representation Blending for Multimodal Dataset Distillation (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.