Research map / Large language models / Language, safety and interpretability
Revela: Dense Retriever Learning via Language Modeling
Fengyu Cai, Tong Chen, Xinran Zhao, Sihao Chen, Hongming Zhang, Tongshuang Wu, Iryna Gurevych, Heinz Koeppl
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| co-train retriever with LM via in-batch attention instead of REPLUG frozen-LM distillation (ablation) | Revela 3B · CoIR | nDCG@10 (%): 53.9 -> 60.1 | Improved |
| co-train retriever with LM instead of REPLUG (ablation) | Revela 3B · BRIGHT | nDCG@10 (%): 12.2 -> 20.1 | Improved |
| co-train with LM instead of REPLUG (ablation) | Revela 1B · BEIR | nDCG@10 (%): 35.7 -> 42.7 | Improved |
| use Revela instead of Contriever objective (wiki-trained, same LLaMA-3.2-1B) (ablation) | Revela 1B · BEIR | nDCG@10 (%): 42.4 -> 42.7 | No clear effect |
| use Revela instead of Contriever objective (wiki-trained) (ablation) | Revela 1B · CoIR | nDCG@10 (%): 50.3 -> 53.2 | Improved |
| use Revela instead of Contriever objective (code-trained, out-of-domain BEIR) (ablation) | Revela 1B · BEIR | nDCG@10 (%): 32.3 -> 39.6 | Improved |
| use Revela instead of Contriever objective (code-trained) (ablation) | Revela 1B · CoIR | nDCG@10 (%): 52.1 -> 58.6 | Improved |
| increase batch size from 4 to 16 (ablation) | Revela 1B · CoIR | nDCG@10 (%): 55.9 -> 58.6 | Improved |
| increase batch size from 4 to 16 (ablation) | Revela 0.1B · CoIR | nDCG@10 (%): 43.2 -> 48.4 | Improved |
| increase batch size from 4 to 16 (ablation) | Revela 0.5B · BEIR | nDCG@10 (%): 36.8 -> 38.8 | Improved |
| scale retriever backbone from 0.1B to 3B (ablation) | Revela · CoIR | nDCG@10 (%): 48.4 -> 60.1 | Improved |
| use a larger LM (1B instead of 0.1B) with 0.5B retriever (ablation) | Revela 0.5B · BEIR | nDCG@10 (%): 41.6 -> 38.8 | Hurt |
Closest papers by meaning
- BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models (ICLR 2024 · similarity 0.86)
- UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition (ICLR 2024 · similarity 0.86)
- Doc-to-LoRA: Learning to Instantly Internalize Contexts (ICML 2026 · similarity 0.86)
- Can Language Models Actually Retrieve In-Context? Drowning in Documents at Million Token Scale (NeurIPS 2026 · similarity 0.86)
- Retriever-Free Retrieval-Augmented Reasoning via Corpus-Traversing MCTS (NeurIPS 2026 · similarity 0.85)
- In-Context Pretraining: Language Modeling Beyond Document Boundaries (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
