Research map / Label-efficient and robust learning / Recognition and long-tail
DOGR: Leveraging Document-Oriented Contrastive Learning in Generative Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DOGR, a two-stage generative retrieval framework that adds document-level contrastive learning with prefix-oriented and retrieval-augmented negative sampling on top of identifier generation, and fuses generation probability with a semantic score at inference.
Problem
Generative retrieval models learn only query-to-identifier relations, so they cannot directly model query-document relevance; existing fixes rely only on identifier generation probabilities.
Approach
A two-stage training scheme: first train a T5 model on query->identifier and document->identifier generation, then fine-tune with two contrastive losses over negatives sampled from shared identifier prefixes and from the model's own top-k retrieved identifiers, and rank by P(z|q)·s(q,d) at inference.
Key results
- On NQ320k full test, DOGR reaches R@1 70.2, R@10 89.1, MRR@100 76.8, exceeding the best prior generative baseline by +1.6, +0.3 and +1.2 points.
- On MS MARCO, DOGR achieves MRR@10 22.5, versus 21.6 for LTRGR and 34.8 for dense GTR-base.
- Applied to n-gram, first-n-token and numerical identifiers, DOGR improves R@1 over the base generative model (e.g., 65.2 to 67.2 for n-gram).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add document segment sampling during identifier generation (ablation) | DOGR · NQ320K | R@1: 68.6 -> 70.2 | Helped |
| Add prefix-oriented contrastive loss (ablation) | DOGR · NQ320K | R@1: 69.0 -> 70.2 | Helped |
| Add retrieval-augmented contrastive loss (ablation) | DOGR · NQ320K | R@1: 69.3 -> 70.2 | Helped |
| Add auxiliary generation loss (ablation) | DOGR · NQ320K | R@1: 65.8 -> 70.2 | Helped |
| Add semantic score during inference (ablation) | DOGR · NQ320K | R@1: 68.2 -> 70.2 | Helped |
| Retain generation probability during inference (ablation) | DOGR · NQ320K | R@1: 63.3 -> 70.2 | Helped |
| Use fusion ranking instead of generation-probability ranking with semantic reranking only for conflicts (ablation) | DOGR · NQ320K | R@1: 69.9 -> 70.2 | Helped |
| Increase keyword identifier length from 3 to 8 (ablation) | DOGR · NQ320K | R@1: 66.7 -> 70.2 | Helped |
| Use DOGR with keyword DocID rather than generation-only keyword DocID | DOGR · NQ320K | R@1: 68.2 -> 70.2 | Helped |
| Use DOGR rather than the strongest generative retrieval baseline | Generative retrieval on NQ320K · NQ320K | R@1: +1.6% | Helped |
| Use DOGR rather than the strongest generative retrieval baseline | Generative retrieval on MS MARCO · MS MARCO Dev | MRR@10: 4.2% | Helped |
Surprising findings
- On MS MARCO, DOGR (MRR@10 22.5) remains well below the dense GTR-base baseline (34.8), so the generative approach still lags dense retrieval at scale.
- Removing the generation probability from inference drops R@1 by 9.8 points on NQ320k, showing the identifier probability is essential even with the semantic score.
Limitations
- Identifier conflicts with short identifiers degrade performance (conflict rate 46.9% at length 3), and the method relies on identifier design.
- Performance on the large-scale MS MARCO corpus remains far below dense retrieval baselines.
- Unseen-test performance on NQ320k is only competitive with GENRET and not best.
Experimental setup
Datasets: NQ320k MS MARCO passage ranking
Benchmarks: NQ320k MS MARCO passage ranking
Metrics: Recall@1 Recall@10 MRR@100 MRR@10
Baselines: BM25 DocT5Query DPR ANCE SentenceT5 GTR-base GENRE DSI SEAL DSI-QG NCI GENRET
Builds on: DSI LTRGR GLEN KeyBERT T5
Closest papers by meaning
- Descriptive and Discriminative Document Identifiers for Generative Retrieval (AAAI 2025 · similarity 0.91)
- Generative Retrieval Meets Multi-Graded Relevance (NeurIPS 2024 · similarity 0.91)
- Learning to Rank in Generative Retrieval (AAAI 2024 · similarity 0.90)
- UniGen: A Unified Generative Framework for Retrieval and Question Answering with Large Language Models (AAAI 2024 · similarity 0.88)
- SRA-CL: Semantic Retrieval Augmented Contrastive Learning for Sequential Recommendation (NeurIPS 2025 · similarity 0.86)
- Retrieval is Accurate Generation (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.