Research map / Label-efficient and robust learning / Self-supervised and contrastive
OCNR: Stabilizing Self-Play by Mitigating Iteration-Collapse With One-Class Novelty Rewards
Seungyoo Lee, Giung Nam, Hyungi Lee, Juho Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add one-class novelty reward (OCNR) to R-Zero questioner | R-Zero · Math AVG, Iter 5 | average accuracy: 45.27 -> 52.22 | Improved |
| add OCNR to R-Zero for general-purpose reasoning transfer | R-Zero · MMLU-Pro | accuracy: 54.7 -> 60.2 | Improved |
| add OCNR to Absolute Zero proposer | AZR · AMC | accuracy: 37.5 -> 42.5 | Improved |
| add OCNR to Absolute Zero proposer | AZR · HumanEval+ | pass@1: 71.3 -> 72.5 | Improved |
| double the over-generated data used for solver training (R-Zero 2x) without novelty reward (ablation) | R-Zero · Math AVG, Iter 5 | average accuracy: 45.27 -> 44.02 | Hurt |
| double data for R-Zero (2x) at early iteration (ablation) | R-Zero · Math AVG, Iter 1 | average accuracy: 46.50 -> 47.52 | Improved |
| use sigmoid-based novelty reward instead of distance-based (ablation) | OCNR · t-SNE of generated questions | question novelty | Hurt |
| use more prototypes in the Seen Detector (K=1 to K=3) (ablation) | OCNR Seen Detector · seen detection | AUROC: 0.9152 -> 0.9321 | No clear effect |
| add novelty coefficient lambda_nov >= 0.1 (ablation) | R-Zero · Math AVG, Iter 2 to Iter 5 | average accuracy: 48.10 -> 45.27 for lambda 0.0 | Improved |
| extend R-Zero with OCNR to 10 iterations | R-Zero w/ OCNR · Math AVG, Iter 7 to Iter 10 | average accuracy: 54.51 -> 53.98 | Improved |
Closest papers by meaning
- SCOPE: Self-Play via Co-Evolving Policies for Open-Ended Tasks (NeurIPS 2026 · similarity 0.89)
- Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models (ICML 2024 · similarity 0.87)
- SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Models (ICLR 2026 · similarity 0.87)
- SeRL: Self-play Reinforcement Learning for Large Language Models with Limited Data (NeurIPS 2025 · similarity 0.87)
- Triplets Better Than Pairs: Towards Stable and Effective Self-Play Fine-Tuning for LLMs (NeurIPS 2025 · similarity 0.87)
- Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignment (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
