Research map / Large language models / Retrieval and knowledge

KnowPO: Knowledge-Aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Models

AAAI 2025 · Large language models › Retrieval and knowledge · approach: RAG robustness and evaluation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes KnowPO, a DPO-based preference optimization with SFT that teaches RAG models to follow conflicting context and ignore irrelevant context, using constructed contextual-ignorance and contextual-overinclusion negative samples plus length and error-ratio balancing.

Problem

LLMs in retrieval-augmented generation balance poorly between adherence to conflicting external knowledge and robustness to irrelevant noisy context; instruction tuning alone lacks explicit negative signals, leading to contextual ignorance and contextual overinclusion.

Approach

Builds knowledge-conflict preference pairs (conflicting vs. irrelevant contexts, with error-type negatives generated by GPT-4), aligns response lengths and balances the two error-type pair ratios (R_error=1), and trains with LoRA DPO after an instruction-tuning SFT stage.

Key results

• On SQuAD2.0-Eval with Baichuan2-7B-Chat, KnowPO reaches R_Ad 80.64% and R_Ro 38.77%, versus 58.83% and 21.43% for the best baseline KAFT.
• On RGB with Llama2-13B-Chat, KnowPO reaches R_Ad 83.50% and R_Ro 37.50%, and on KNOT R_Ad 77.03% and R_Ro 38.28%.
• On CMB medical test set, KnowPO improves Baichuan2-7B-Chat R_Ad from 58.69% to 96.23% and R_Ro from 10.66% to 24.12%.

Measured effects reported in the paper

Surprising findings

• Length imbalance between preferred and dispreferred responses biased DPO toward verbose refusals, substantially reducing adherence when lengths were not aligned.
• Counterfactual answers were not required to be correct; only the conflict between the parametric and context answer mattered.

Limitations

• Relies on GPT-4 to generate counterfactual and contextual-overinclusion negatives, and the evaluation uses constructed contexts that the authors say are less challenging than real RAG contexts.
• Evaluated on only two base models, both chat models of 7B and 13B size.
• Only the single-document conflict setting (at most one answer-bearing document) is studied.

Experimental setup

Datasets: SQuAD 2.0 RGB KNOT CMB

Benchmarks: SQuAD 2.0-Eval RGB KNOT CMB

Metrics: R_Ad adherence ratio R_Ro robustness ratio prior-probability adherence analysis conflicting answer match rate

Baselines: Base Prompt CoT COT-VE KAFT CAD CFP

Builds on: Direct Preference Optimization KAFT Chain-of-Thought prompting Context-Aware Decoding

Closest papers by meaning

• DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation (NeurIPS 2025 · similarity 0.87)
• Adapting to Non-Stationary Environments: Multi-Armed Bandit Enhanced Retrieval-Augmented Generation on Knowledge Graphs (AAAI 2025 · similarity 0.87)
• TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees (ICLR 2025 · similarity 0.87)
• A Systematic Exploration of Knowledge Graph Alignment with Large Language Models in Retrieval Augmented Generation (AAAI 2025 · similarity 0.87)
• Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts (ICLR 2024 · similarity 0.87)
• Inference Scaling for Long-Context Retrieval Augmented Generation (ICLR 2025 · similarity 0.86)