AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

SOPO: Socratic Guided Policy Optimization for Span-Level Hallucination Detection

Jiawei Dong, Zilong Bai, Pengtian Zhu, Peng Zhou

NeurIPS 2026 · Reinforcement learning › RL from feedback · approach: RL for LLM reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SOPO, a Socratic guided policy optimization approach that applies RLVR-style training to span-level hallucination detection to address localization ambiguity and reward sparsity.

Problem

Span-level hallucination detection in LLM outputs, where RLVR is hard to apply due to localization ambiguity and reward sparsity.

Approach

Applies Socratic guided policy optimization within an RLVR framework to tackle localization ambiguity and reward sparsity in span-level hallucination detection; specifics are not stated in the available abstract excerpt.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace Socratic guidance with directive Direct-Feedback (ablation)SOPO · RAGTruth span-level (overall)span F1: 60.81 -> 63.31Improved
add Socratic guiding question to step-wise analysis (ablation)SOPO feedback · RAGTruth span-level (overall)span F1Improved
add step-wise analysis to guiding question (ablation)SOPO feedback · RAGTruth span-level (overall)span F1No clear effect
allow ground-truth leakage in teacher feedback (ablation)SOPO · RAGTruth span-level (overall)overall F1: overall F1 gap > 1.0Hurt
add Turn-based Advantage ScalingSOPO · RAGTruth trainingreasoning length per turn: stable ~1,700 tokens per turn vs GRPO inflationImproved
use SOPO instead of GRPOQwen3-4B-2507 · RAGTruth span-level (overall, 10 runs)span F1: 59.49 -> 63.34Improved
use SOPO instead of CAPOQwen3-4B-2507 · LLM-AggreFact (OOD)mean balanced accuracy: 75.29 -> 75.93No clear effect
use SOPO instead of GRPO on OOD benchmarksQwen3-4B-2507 · HaluEval (OOD)accuracy: 67.52 -> 69.77Improved
use FLOP-matched SOPO checkpoint (531 steps) instead of full SOPO (ablation)SOPO · RAGTruth overall case-F1case F1: 83.72 vs GRPO 83.03No clear effect
scale teacher from 8B to 80B (ablation)SOPO · RAGTruth span-level (overall)span F1: 60.47 -> 63.31Improved
use SOPO instead of GRPO with 1.7B student (ablation)Qwen3-1.7B · RAGTruth span-level (overall)span F1: 55.8 -> 57.7Improved
use SOPO instead of GRPO with legacy 8B student (ablation)Qwen3-8B · RAGTruth span-level (overall)span F1: 60.6 -> 61.0Improved

Experimental setup

Builds on: RLVR

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy