Research map / Large language models / Retrieval and knowledge
Auto-GDA: Automatic Domain Adaptation for Efficient Grounding Verification in Retrieval-Augmented Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Auto-GDA adapts lightweight NLI grounding verifiers to RAG target domains via iterative LLM/augmentation-based synthetic data generation, with teacher-weighted entailment certainty and discrete sample selection, reaching LLM-level ROC at ~10% of latency.
Problem
Lightweight NLI models for verifying grounding of LLM outputs in retrieved evidence perform worse on realistic RAG inputs than LLMs, due to domain mismatch, and labeled target-domain data for supervised adaptation is unavailable.
Approach
Unsupervised domain adaptation via iterative synthetic claim generation (few-shot LLM prompting), label-preserving augmentations, and selection of samples by a decomposed distribution-matching objective that combines distance to target claims, a label-correctness term from teacher entailment certainty, and a model utility term.
Key results
- DeBERTaV2 (Auto-GDA) averages 0.878 ROC across four dataset splits versus 0.708 for non-fine-tuned DeBERTaV2 and 0.883 for GPT-4o, and it outperforms its teacher (MiniCheck-T5 / AlignScore-based teacher) in three of four cases.
- Auto-GDA closes about 96% of the domain adaptation gap relative to fine-tuning on labeled target data (0.885 mean ROC).
- Auto-GDA DeBERTaV2 runs at about 2.12 s per 50 samples, roughly 10% of GPT-4o's 20.47 s, with DAPT, SiFT, and Deep-CORAL giving smaller or inconsistent gains.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune DeBERTaV2 with Auto-GDA synthetic data rather than the non-fine-tuned model | DeBERTaV2 · RAGTruth Summary, RAGTruth QA, LFQA-Verification QA, SummEdits Summary | ROC-AUC: 0.782 -> 0.837; 0.530 -> 0.867; 0.645 -> 0.925; 0.876 -> 0.883 | Helped |
| fine-tune DeBERTaV2 on few-shot generated data instead of leaving it non-fine-tuned (ablation) | DeBERTaV2 · RAGTruth Summary, RAGTruth QA, LFQA-Verification QA, SummEdits Summary | ROC-AUC: 0.782 -> 0.799; 0.530 -> 0.826; 0.645 -> 0.934; 0.876 -> 0.872 | Helped |
| add augmentations but select the generated samples randomly rather than using the Auto-GDA selection objective (ablation) | DeBERTaV2 fine-tuned on few-shot data · RAGTruth Summary, RAGTruth QA, LFQA-Verification QA, SummEdits Summary | ROC-AUC: 0.799 -> 0.777; 0.826 -> 0.783; 0.934 -> 0.919; 0.872 -> 0.862 | Hurt |
| use the Auto-GDA selection objective for augmented data rather than random selection (ablation) | DeBERTaV2 fine-tuning · RAGTruth Summary, RAGTruth QA, LFQA-Verification QA, SummEdits Summary | ROC-AUC: 0.777 -> 0.837; 0.783 -> 0.867; 0.919 -> 0.925; 0.862 -> 0.883 | Mixed |
| fine-tune DeBERTaV2 with Auto-GDA instead of with labeled target data (ablation) | DeBERTaV2 · RAGTruth Summary, RAGTruth QA, LFQA-Verification QA, SummEdits Summary | ROC-AUC: 0.842 -> 0.837; 0.890 -> 0.867; 0.909 -> 0.925; 0.898 -> 0.883 | Mixed |
| apply classical UDA methods DAPT, SiFT, or Deep-CORAL instead of Auto-GDA | DeBERTaV2 · RAGTruth Summary, RAGTruth QA, LFQA-Verification QA, SummEdits Summary | ROC-AUC | Hurt |
| use DeBERTaV2 (Auto-GDA) instead of GPT-4o | Grounding verification · RAGTruth Summary, RAGTruth QA | ROC-AUC: 0.892 -> 0.837; 0.865 -> 0.867 | Mixed |
| use DeBERTaV2 (Auto-GDA) instead of AlignScore | Grounding verification · LFQA-Verification QA, SummEdits Summary | ROC-AUC: 0.904 -> 0.925; 0.894 -> 0.883 | Mixed |
Surprising findings
- Classical UDA methods (DAPT, SiFT, Deep-CORAL) did not yield significant improvements in the NLI domain adaptation setting.
- Random selection of augmented samples decreased performance relative to few-shot data alone, so augmentation helps only when combined with the selection criterion.
Limitations
- Assumes the distribution of evidence documents (knowledge base) is readily available.
- Requires a separate model for each target domain; multi-domain adaptation is not studied.
- Relies on a small labeled validation set (~30 samples) for hyperparameter tuning, deviating from fully unsupervised setting.
Experimental setup
Datasets: RAGTruth LFQA-Verification SummEdits MNLI
Benchmarks: RAGTruth LFQA-Verification SummEdits
Metrics: ROC-AUC inference time (sec per 50 samples)
Baselines: FLAN-T5 BART-large DeBERTaV2 MiniCheck-T5 AlignScore Vectara-2.1 DAPT SiFT Deep-CORAL GPT-3.5 GPT-4o-mini GPT-4o
Builds on: MiniCheck AlignScore DAPT SiFT Deep CORAL
Closest papers by meaning
- Revisiting Source-Free Domain Adaptation: Insights into Representativeness, Generalization, and Variety (CVPR 2025 · similarity 0.87)
- Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass (ICLR 2025 · similarity 0.87)
- Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models (NeurIPS 2025 · similarity 0.86)
- Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models (NeurIPS 2024 · similarity 0.86)
- BLADE: Enhancing Black-Box Large Language Models with Small Domain-Specific Models (AAAI 2025 · similarity 0.86)
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.