Research map / Large language models / Retrieval and knowledge

Beyond Raw Context Transfer: Representation-based Federated Retrieval-Augmented Generation

NeurIPS 2026 · Large language models › Retrieval and knowledge · approach: RAG robustness and evaluation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FedRepRAG, a federated RAG framework where clients keep raw documents local and exchange only query-relevant retrieval embeddings, which a collaboratively trained projector maps into a frozen LLM/VLM's input space as representation tokens.

Problem

Decentralized RAG over sensitive, distributed corpora (e.g., hospitals) typically transfers raw retrieved text, raising privacy risks and inference-time overhead from long contexts.

Approach

Replace raw-context transfer with exchange of compact retrieval embeddings; a two-stage federated-trained MLP projector (semantic alignment via document reconstruction, then task-specific QA alignment) converts each retrieved embedding into one generator input token while the backbone stays frozen.

Key results

• On SLAKE-EN (beta=0.1), FedRepRAG reaches 63.71% average accuracy vs 49.58% for Direct Inference (+14.13 points) and reduces retrieval tokens by 94.62% vs Centralized IC-RAG.
• On PubMedQA, FedRepRAG reaches 59.40% accuracy at beta=0.1 vs 82.20% for Centralized IC-RAG and 54.00% for Direct Inference, with 3 retrieval tokens vs 1194.05.
• On a 300-question KILT-NQ subset, FedRepRAG achieves 25.33 EM vs 10.00 for FedMosaic.
• Stage 1 DP-SGD with sigma=0.5 reduces KILT-NQ context inversion from 20.4% to 0.6% with EM 21.61 vs 21.85 non-DP.

Surprising findings

• Centralized IC-RAG (raw context) is outperformed by FedRepRAG on PubMedQA in accuracy only for the federated setting; on PubMedQA centralized raw-context RAG (82.20%) beats FedRepRAG (59.40%).
• Local IC-RAG under beta=0.1 performs worse than Direct Inference on PubMedQA and KILT-NQ, suggesting irrelevant local context can mislead.

Limitations

• Does not guarantee confidentiality of exchanged representations; residual inversion leakage remains (e.g., 11% context recovery on KILT-NQ at Stage 2).
• Evaluated only on VQA and QA tasks with 10 or 20 simulated clients.
• FedMosaic comparison uses only a 300-question KILT-NQ subset due to cost.

Experimental setup

Datasets: SLAKE-EN ScienceQA PubMedQA KILT-NQ

Benchmarks: SLAKE-EN ScienceQA PubMedQA KILT-NQ

Metrics: accuracy exact match F1 retrieval-context tokens GFLOPs representation inversion success rate membership inference AUC

Baselines: Direct Inference Local IC-RAG Centralized IC-RAG Centralized Rep-RAG Federated IC-RAG FedMosaic

Builds on: FedAvg retrieval-augmented generation context compression

Closest papers by meaning

• SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAG (NeurIPS 2025 · similarity 0.85)
• Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection (ICLR 2025 · similarity 0.85)
• RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering (NeurIPS 2025 · similarity 0.85)
• One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models (AAAI 2025 · similarity 0.85)
• Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers (NeurIPS 2025 · similarity 0.85)
• VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documents (CVPR 2025 · similarity 0.85)