Research map / Vision-language and multimodal / Cross-modal learning
An End-To-End Graph Attention Network Hashing for Cross-Modal Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EGATH, an end-to-end supervised cross-modal hashing method that combines CLIP and Transformer feature extractors with a GAT label classifier and a cosine/triplet hash loss, evaluated on MIRFlickr25K, NUS-WIDE and MS-COCO.
Problem
Supervised cross-modal hashing for image-text retrieval suffers from limited global feature representations and weak semantic/label associations, and label co-occurrence modeling is computationally expensive.
Approach
Uses a graph attention network over a label co-occurrence graph (GloVe label embeddings) to predict labels that enrich image and text features, combined with CLIP+Transformer image encoding, a Transformer text encoder, and a hash loss mixing triplet loss on cosine similarity with an alternating discrete optimization (GCDH-style).
Key results
- EGATH reports mAP of 0.8411 (16-bit, I→T) on MIRFlickr25K versus 0.8177 for the best baseline DCHMT.
- On MS-COCO 64-bit I→T, EGATH reaches 0.7945 mAP, while SCAHN reports 0.7528 (SCAHN is higher at 64 bits on COCO for T→I per the text's note).
- Ablation: removing the GAT classifier (EGATH-G) or the triplet loss (EGATH-T) lowers mAP on all three datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the GAT classifier (ablation) | EGATH · MIRFlickr25K I→T 64bit | mAP: 0.8464 -> 0.8637 | Helped |
| Add the GAT classifier (ablation) | EGATH · NUS-WIDE T→I 32bit | mAP: 0.7386 -> 0.7437 | Helped |
| Add the triplet loss (ablation) | EGATH · MIRFlickr25K I→T 64bit | mAP: 0.8622 -> 0.8637 | Helped |
| Add the triplet loss (ablation) | EGATH · NUS-WIDE T→I 32bit | mAP: 0.7390 -> 0.7437 | Helped |
| Use EGATH instead of SCAHN | cross-modal hashing · MS-COCO I→T 64bit | mAP: 0.7528 -> 0.7945 | Helped |
Surprising findings
- The authors note SCAHN outperforms EGATH on MS-COCO at 64 bits in the T→I setting, which they attribute to its contrastive cross-media mechanism.
Limitations
- GAT attention cost grows with the number of labels, limiting scalability to large label sets.
- SCAHN beats EGATH on MS-COCO 64-bit in some settings.
- No error bars or variance across runs reported.
Experimental setup
Datasets: MIRFlickr25K NUS-WIDE MS-COCO
Benchmarks: MIRFlickr25K NUS-WIDE MS-COCO
Metrics: mAP precision-recall curve top-K precision
Baselines: DJSRH JDSH DCHMT MLCAH CDTH UCCH SCAHN DSPH
Builds on: CLIP graph attention network GCDH triplet loss Transformer
Closest papers by meaning
- Adaptive Graph Attention Based Discrete Hashing for Incomplete Cross-modal Retrieval (AAAI 2026 · similarity 0.93)
- Lightweight Contrastive Distilled Hashing for Online Cross-modal Retrieval (AAAI 2025 · similarity 0.92)
- UGGRH: Unsupervised Generative Completion and Graph-attention Refinement for Incomplete Cross-modal Hashing (NeurIPS 2026 · similarity 0.90)
- Self-Supervised Multi-Modal Knowledge Graph Contrastive Hashing for Cross-Modal Search (AAAI 2024 · similarity 0.90)
- Dynamic Masking and Auxiliary Hash Learning for Enhanced Cross-Modal Retrieval (NeurIPS 2025 · similarity 0.90)
- Vision-guided Text Mining for Unsupervised Cross-modal Hashing with Community Similarity Quantization (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.