Research map / Large language models / Reasoning and chain of thought

ReasoningShield: Safety Moderation over Reasoning Traces of Large Reasoning Models

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces ReasoningShield, a 9.2K-sample benchmark and 1B/3B lightweight moderation models for safety of LRM chain-of-thought traces, trained with SFT then DPO on hard negatives.

Problem

Existing moderation tools trained on answer-level QA data degrade substantially when moderating harmful content hidden in LRM reasoning traces (CoT), so a dedicated CoT moderation benchmark and models are needed.

Approach

Formalizes CoT moderation with a 10-category, 3-level (Safe / Potentially Harmful / Harmful) taxonomy and builds a benchmark with stepwise risk analyses via a multi-LLM majority-vote annotation pipeline plus human-labeled OOD test set; trains small Llama-3.2 models with SFT on consensus samples followed by DPO on hard negatives.

Key results

• ReasoningShield-3B reaches 91.8% average F1 on open-source-LRM CoT moderation, versus 56.2% for LlamaGuard-4-12B and 87.8% for GPT-4o with the ReasoningShield prompt.
• On closed-source (commercial) LRM traces, ReasoningShield-3B averages 81.4% F1 versus 33.7% for LlamaGuard-4.
• Under obfuscation attacks ReasoningShield-3B keeps F1 0.932 while LlamaGuard-4 drops to 0.500.

Surprising findings

• Pilot study shows commercial-LRM-generated traces are hard to moderate for existing tools, with LlamaGuard-1 dropping up to 35.2 F1 points from answer to CoT moderation.
• A 3B specialized model outperforms the much larger GPT-4o even when GPT-4o is given the same optimized prompt.

Limitations

• Focuses only on content safety; agentic risks such as tool misuse are not addressed.
• Dataset access is restricted, which limits reproducibility.
• Some subsets (e.g., Gemma-3 and Mistral prompted variants) show large drops on certain datasets.

Experimental setup

Datasets: AIR-Bench SALAD-Bench Alpaca

Benchmarks: ReasoningShield-Test WildGuard-Test BeaverTails-Test

Metrics: F1 accuracy

Baselines: LlamaGuard-1 LlamaGuard-2 LlamaGuard-3 LlamaGuard-4 WildGuard OpenAI Moderation API GPT-4o MD-Judge

Builds on: DPO LlamaGuard WildGuard

Closest papers by meaning

• The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think (ICLR 2026 · similarity 0.89)
• SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment (NeurIPS 2025 · similarity 0.89)
• FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning (ICLR 2026 · similarity 0.88)
• Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thought (NeurIPS 2024 · similarity 0.88)
• Reasoning Structure of Large Language Models (ICML 2026 · similarity 0.88)
• Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization (NeurIPS 2025 · similarity 0.88)