Research map / Large language models / Reasoning and chain of thought

Seirênes: Adversarial Self-Play with Evolving Distractions for LLM Reasoning

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Seirênes, a shared-parameter adversarial self-play RL framework where one model generates misleading hints (Adversary) and solves problems under them (Reasoner), improving math reasoning across 7 benchmarks.

Problem

LLM reasoners are brittle under non-idealized contexts (superfluous, tangential, or misleading information), while standard RLVR trains only on clean problems.

Approach

A single policy is role-conditioned as Adversary (generates plausible misleading hints, rewarded by the clean-vs-hinted success drop) and Reasoner (solves the original problem under hints with verifier reward), co-evolving online, with decoupled update streams and efficient orchestration (bounded FIFO buffers, latency-aware scheduling, mastery-aware sampling).

Key results

• Seirênes improves the average over the instruction-tuned baseline by +10.2 (Qwen3-4B-Instruct, 47.8 to 58.0), +9.1 (Qwen2.5-7B-Instruct, 13.8 to 22.9), and +7.2 (Qwen3-30B-A3B, 56.7 to 63.9).
• Robustness table: Seirênes obtains the best perturbed average accuracy of 88.2 vs 83.9 for Qwen3-4B-Instruct, and the lowest average robustness degradation (8.06 vs 10.46).
• Hints from the 4B Adversary reduce GPT-5.1 overall accuracy by 3.9 points and Gemini 3 Flash by 5.2 points across seven benchmarks.

Surprising findings

• Adversarial hints from a 4B model degrade top proprietary models (GPT-5.1, Gemini 3 Flash), with Gemini dropping 12.9 points on IMO-Bench.
• Cooperative hint-based baselines (SAGE, Scaf-GRPO, InT) show larger degradation than the instruction-tuned model under generated distractors.

Limitations

• Hint-generation cost: G2 >= 4 could not be fully explored due to compute limits, and per-step inference cost grows with G2.
• Results evaluated mainly on math benchmarks; generalization to other reasoning domains is untested.
• Some baseline numbers and comparisons (e.g., budget-matched) are reported with uncertain or single-run statistics; the paper's own appendix notes the mastery and staleness analyses rely on operational criteria.

Experimental setup

Datasets: DAPO-Math-17K OpenR1-Math

Benchmarks: AIME24 AIME25 AIME26 IMO-Bench MINERVA Math OlympiadBench HMMT 2026 Math-Perturb GSMIR MMLU-Perturb OpenBookQA-Perturb

Metrics: avg@32 accuracy avg@4 accuracy robustness degradation

Baselines: DAPO Dr.GRPO ExGRPO SAGE Scaf-GRPO LUFFY InT

Builds on: GRPO R-Zero SPICE DAPO

Closest papers by meaning

• SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoning (NeurIPS 2025 · similarity 0.90)
• Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning (ICLR 2026 · similarity 0.90)
• Learning Robust Reasoning through Guided Adversarial Self-Play (NeurIPS 2026 · similarity 0.88)
• Self-playing Adversarial Language Game Enhances LLM Reasoning (NeurIPS 2024 · similarity 0.88)
• Are Reasoning LLMs Robust to Interventions on their Chain-of-Thought? (ICLR 2026 · similarity 0.88)
• R-Diverse: Mitigating Diversity Illusion in Self-Play LLM Training (ICML 2026 · similarity 0.87)