Research map / Large language models / Retrieval and knowledge
Retrieval is Not Enough: Enhancing RAG through Test-Time Critique and Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AlignRAG, a test-time critique-driven iterative refinement framework for RAG in which a LoRA-tuned 8B critic model, trained on contrastive expert-vs-weak critiques, critiques and revises reasoning to align with retrieved evidence, plus an autonomous stopping variant.
Problem
Standard RAG pipelines often produce reasoning that diverges from retrieved evidence (reasoning misalignment), causing factual inconsistencies and unsupported conclusions.
Approach
Train a dedicated retrieval-augmented critic LM on contrastive critiques built from a strong (LLaMA3.1-8B) and weak (Qwen2.5-0.5B) model's responses, so critiques are not self-generated; apply iterative critique-driven revision at test time, with an autonomous variant that emits [Good]/[Bad] tokens to stop refinement.
Key results
- AlignRAG with Qwen2.5-14B + CLM-8B reaches an average of 62.8% across five in-domain benchmarks vs 60.7% for Self-Refine and 61.7% for InstructRAG.
- The 8B CLM improves over Self-Refine by 12.1% on out-of-domain tasks and outperforms a direct 72B CLM by 2.2% (as stated in abstract).
- AlignRAG-auto performs on par with or slightly above the fixed-iteration AlignRAG across datasets and backbones, per the authors.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply AlignRAG test-time critique alignment instead of Self-Refine | Qwen2.5-7B · Five QA benchmarks (Avg.) | accuracy / str-em average: 58.3 -> 62.1 | Helped |
| apply AlignRAG test-time critique alignment instead of Self-Refine | Qwen2.5-14B · Five QA benchmarks (Avg.) | accuracy / str-em average: 60.7 -> 62.8 | Helped |
| apply AlignRAG test-time critique alignment instead of Self-Refine | LLaMA3.1-8B · Five QA benchmarks (Avg.) | accuracy / str-em average: 57.8 -> 60.6 | Helped |
| add an auxiliary frozen LLaMA3.1-8B critic (ablation) | Vanilla RAG with Qwen2.5-7B · Five QA benchmarks (Avg.) | accuracy / str-em average: 44.6 -> 47.3 | Helped |
| train the CLM on weak-model critiques without contrastive signal (ablation) | RAG with Qwen2.5-7B · Five QA benchmarks (Avg.) | accuracy / str-em average: 47.3 -> 49.6 | Helped |
| add contrastive critique synthesis using expert and weak responses (ablation) | RAG with trained CLM and Qwen2.5-7B · Five QA benchmarks (Avg.) | accuracy / str-em average: 49.6 -> 52.9 | Helped |
| replace CPO with CFT for CLM training (ablation) | Qwen2.5-14B · Seven QA benchmarks (Avg.) | accuracy / str-em average: 51.0 -> 53.6 | Helped |
| replace CPO with CFT for CLM training (ablation) | Qwen2.5-14B · SQuAD | accuracy: 20.2 -> 26.6 | Helped |
| add alignment to InstructRAG | Qwen2.5-14B · OOD (avg.) | accuracy: 24.9 -> 33.4 | Helped |
| use AlignRAG-auto dynamic stopping rather than fixed iteration count (ablation) | AlignRAG · NQ and ASQA | accuracy / str-em | Helped |
| use dynamic stopping rather than fixed iteration count (ablation) | AlignRAG · All datasets and model sizes | accuracy | No clear effect |
Surprising findings
- A frozen untrained LLaMA3.1-8B critic already improves over vanilla RAG, and the trained CCS critic surpasses a directly supervised 72B CLM.
Limitations
- Effectiveness diminishes when none of the retrieved documents are relevant to the query.
- Optimal integration may require tuning depending on generator LLM and RAG architecture.
- Main experiments were run once without error bars.
Experimental setup
Datasets: PopQA TriviaQA NaturalQuestions 2WikiMultihopQA ASQA HotpotQA SQuAD
Benchmarks: NQ 2WikiMultihopQA TriviaQA PopQA ASQA HotpotQA SQuAD
Metrics: accuracy str-em
Baselines: Chain-of-thought Vanilla Reasoning RetRobust InstructRAG Self-RAG Self-Refine
Builds on: Self-Refine InstructRAG Self-RAG DPO
Closest papers by meaning
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (ICLR 2024 · similarity 0.92)
- Toward Verifiable Instruction-Following Alignment for Retrieval Augmented Generation (AAAI 2025 · similarity 0.91)
- Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals (NeurIPS 2025 · similarity 0.90)
- Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.90)
- RAGGED: Towards Informed Design of Scalable and Stable RAG Systems (ICML 2025 · similarity 0.90)
- StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.