Research map / Vision-language and multimodal / Cross-modal learning
Deep Evidential Hashing for Trustworthy Cross-Modal Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Deep Evidential Cross-modal Hashing (DECH), which derives Beta-distribution binomial opinions from hash-code-based evidence to estimate per-pair retrieval reliability, plus a differentiable hashing module trained with a straight-through estimator.
Problem
Cross-modal hashing methods are deterministic and rank retrieved items only by Hamming distance, so they provide no reliability estimate of whether a retrieved pair truly matches.
Approach
Collects positive evidence from Hamming similarity and negative evidence from a subnetwork directly on hash codes (no classifier), places them on a Beta distribution as a binomial opinion, adjusts it with a dissonance-based correction, and defines a reliability score used to filter retrieved pairs; a differentiable hashing layer (L2-normalized sign with STE) avoids continuous relaxation.
Key results
- DECH achieves mAP up to 0.860 on MIRFLICKR25K (I→T, 128 bits) and outperforms 12 cross-modal hashing baselines across all four datasets and bit lengths.
- Filtering retrieved pairs with reliability threshold 0.5 (DECH r=0.5) further raises mAP, e.g., NUS-WIDE I→T at 128 bits from 0.727 to 0.867.
- Ablation shows removing L_kl, L_nzce, or differentiable hashing each lowers mAP (e.g., MIRFLICKR25K I→T 64 bits: 0.853 full vs 0.823 without L_kl).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add evidence correction loss Lkl (ablation) | DECH · MIRFLICKR25K and IAPR TC-12, image-to-text and text-to-image | mAP@ALL: 0.823 -> 0.853; 0.798 -> 0.831; 0.513 -> 0.590; 0.522 -> 0.600 | Helped |
| add near-zero correct-evidence loss Lnzce (ablation) | DECH · MIRFLICKR25K and IAPR TC-12, image-to-text and text-to-image | mAP@ALL: 0.847 -> 0.853; 0.824 -> 0.831; 0.527 -> 0.590; 0.529 -> 0.600 | Mixed |
| add Differentiable Hashing (DH) (ablation) | DECH · MIRFLICKR25K and IAPR TC-12, image-to-text and text-to-image | mAP@ALL: 0.811 -> 0.853; 0.792 -> 0.831; 0.528 -> 0.590; 0.536 -> 0.600 | Helped |
| filter retrieved sample pairs below reliability level 0.5 | DECH · MIRFLICKR25K, NUS-WIDE, IAPR TC-12, and MS-COCO; image-to-text and text-to-image | mAP@ALL | Helped |
| vary the near-zero correct-evidence loss among Lnzce−CE, Lnzce−RA, and Lnzce−RM (ablation) | DECH · MIRFLICKR25K and IAPR TC-12, image-to-text and text-to-image | mAP | No clear effect |
| set γ to 1 rather than too small or too large (ablation) | DECH · IAPR TC-12, 128-bit codes, image-to-text and text-to-image | mAP | Helped |
| set τ around 0.1 (ablation) | DECH · IAPR TC-12, 128-bit codes, image-to-text and text-to-image | mAP | Helped |
Surprising findings
- Reliability-based filtering at threshold 0.5 gives large mAP gains on NUS-WIDE (e.g., 0.727 to 0.867 at 128 bits for I→T).
Limitations
- Reliability filtering improves mAP only by discarding retrieved pairs, so its benefit depends on threshold choice.
- Uses fixed pre-extracted features (pre-trained backbone frozen), so end-to-end feature learning is not explored.
- Evaluated only on image-text retrieval with mAP.
Experimental setup
Datasets: MIRFLICKR25K IAPR TC-12 NUS-WIDE MS-COCO
Benchmarks: MIRFLICKR25K IAPR TC-12 NUS-WIDE MS-COCO
Metrics: mAP@ALL
Baselines: DJSRH JDSH DGCPN DCMH DADH MLSPH HMAH SCCGDH MESDCH MIAN DNPH DHAPH
Builds on: evidential deep learning subjective logic straight-through estimator DCMH
Closest papers by meaning
- Meta-Guided Sample Reweighting for Robust Cross-Modal Hashing Retrieval with Noisy Labels (AAAI 2026 · similarity 0.90)
- Lightweight Contrastive Distilled Hashing for Online Cross-modal Retrieval (AAAI 2025 · similarity 0.90)
- Dual Self-Paced Cross-Modal Hashing (AAAI 2024 · similarity 0.88)
- Robust Self-Paced Hashing for Cross-Modal Retrieval with Noisy Labels (AAAI 2025 · similarity 0.88)
- Statistical Model-driven Similarity Hashing: Bridging Modalities for Efficient Unsupervised Retrieval (AAAI 2025 · similarity 0.88)
- SHE: Streaming-media Hashing Retrieval (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.