Research map / Vision-language and multimodal / Visual question answering
UniGen: A Unified Generative Framework for Retrieval and Question Answering with Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UniGen, a unified generative model with a shared T5 encoder and two decoders that jointly performs generative document retrieval and grounded QA, bridged by LLM-generated Q- and D-connectors and refined by an iterative enhancement strategy.
Problem
Generative document retrieval and grounded answer generation are typically handled by separate retrieval and reader modules, which prevents simultaneous optimization; query-docid and docid-answer gaps further hinder joint learning.
Approach
A shared encoder with separate retrieval and QA decoders, where LLM-generated Q-Connectors (query context) and D-Connectors (document summaries used as docids) bridge the input-output and docid-answer gaps, and an iterative loop feeds retrieved docs and generated answers back into the LLM to regenerate better Q-Connectors.
Key results
- UniGen-Iter achieves R@1 of 42.34 on MS MARCO vs 37.89 for best baseline NCI (about 11.8% relative gain), and MRR@10 of 74.61 on NQ.
- On open-book QA, UniGen-Iter reaches 24.46 BLEU-1 and 57.83 EM on MS MARCO and NQ, outperforming DPR+FID (18.97 BLEU-1) and Ultron+FID (55.75 EM).
- Ablation shows removing the Q-Connector causes the largest drop, e.g. BLEU-1 from 17.02 to 12.32 for UniGen-Base on MS MARCO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use two iterative-enhancement rounds instead of the base model | UniGen · MS MARCO | MRR@10: 52.64 -> 56.38 | Helped |
| Use two iterative-enhancement rounds instead of the base model | UniGen · NQ | MRR@10: 72.81 -> 74.61 | Helped |
| Add a shared encoder (ablation) | UniGen-Base · MS MARCO | R@1: 37.89 -> 38.75 | Helped |
| Add a Q-Connector (ablation) | UniGen-Base · MS MARCO | BLEU-1: 12.32 -> 17.02 | Helped |
| Add a D-Connector (ablation) | UniGen-Base · MS MARCO | BLEU-1: 15.06 -> 17.02 | Helped |
| Add a shared encoder (ablation) | UniGen-Iter · MS MARCO | R@1: 41.76 -> 42.34 | Helped |
| Add a Q-Connector (ablation) | UniGen-Iter · MS MARCO | BLEU-1: 21.21 -> 24.46 | Helped |
| Add a D-Connector (ablation) | UniGen-Iter · MS MARCO | BLEU-1: 22.35 -> 24.46 | Helped |
Surprising findings
- Closed-book UniGen-Base (367M) outperforms some retrieval-augmented models, which the authors attribute to LLM-generated Q-Connectors.
Limitations
- Relies on gpt-3.5-turbo-0613 API for connector generation, adding cost and dependence on a closed model.
- Iterative enhancement is limited to two iterations in the main model due to memory and time constraints.
- Evaluated only on MS MARCO and NQ with T5-base sized models.
Experimental setup
Datasets: MS MARCO QnA v2.1 Natural Questions (NQ) DocT5query-generated pseudo queries LLM-generated pseudo answers
Benchmarks: MS MARCO QnA v2.1 Natural Questions
Metrics: R@1 R@5 R@10 MRR@10 BLEU-1 ROUGE-L Exact Match F1
Baselines: BM25 DocT5Query DPR ANCE DSI DSI-QG NCI Ultron-PQ T5 BART GPT-3.5 LLaMA2
Builds on: DSI NCI Ultron RAG Fusion-in-Decoder
Closest papers by meaning
- DOGR: Leveraging Document-Oriented Contrastive Learning in Generative Retrieval (AAAI 2025 · similarity 0.88)
- Retrieval-Augmented Visual Question Answering via Built-in Autoregressive Search Engines (AAAI 2025 · similarity 0.88)
- Generative Retrieval Meets Multi-Graded Relevance (NeurIPS 2024 · similarity 0.88)
- Retrieval is Accurate Generation (ICLR 2024 · similarity 0.87)
- Learning to Rank in Generative Retrieval (AAAI 2024 · similarity 0.87)
- Generative Multi-Modal Knowledge Retrieval with Large Language Models (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.