AI Paper Atlas

Research map / Large language models / Alignment and preferences

Temporal Self-Rewarding Language Models: Decoupling Chosen-Rejected via Past-Future

Yidong Wang, Xin Wang, Cunxiang Wang, Junfeng Fang, Qiufeng Wang, Jianing Chu, Xuran Meng, Shu-Xun Yang, Andrew Zhuoer Feng, Libo Qin, Wei Ye, Shikun Zhang

ICML 2026 · Large language models › Alignment and preferences · approach: Robust and calibrated DPO · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Replace self-rewarding pairs with Temporal SR (anchored rejection + future-guided chosen), half the iterationsLlama3.1-8B · AlpacaEval 2.0win rate (%): 19.69 -> 29.44Improved
Add Anchored Rejection (past initial model rejected responses) only (ablation)Self-Rewarding Llama3.1-8B · AlpacaEval 2.0win rate (%): 19.69 -> 29.06Improved
Add Future-Guided Chosen only (ablation)Self-Rewarding Llama3.1-8B · Arena-Hard-v0.1score: 6.7 -> 14.0Improved
Continue Future-only training for another iteration (ablation)Temporal SR w/o Past · AlpacaEval 2.0win rate (%): 22.05 -> 21.49No clear effect
Add Future-Guided Chosen on top of Anchored Rejection (ablation)Temporal SR Llama3.1-8B · Arena-Hard-v0.1score: 13.4 (w/o Future, iter 3) vs 14.6 (full, iter 1)Improved
Use stronger external judge AutoJ-13B instead of self-judge (ablation)Self-Rewarding and Temporal SR · AlpacaEval 2.0win rate (%): SR 19.7 -> 28.9; Temporal SR 29.4 -> 28.6Improved
Use Temporal SR across judge choices (ablation)Llama3.1-8B · AlpacaEval 2.0 / Arena-Hardwin rate and score: Arena-Hard with AutoJ-13B: 22.4 (Temporal SR) vs 15.2Improved
Apply Temporal SR to Qwen2.5-7BQwen2.5-7B · Arena-Hard-v0.1score: 21.5 -> 34.4Improved
Apply Temporal SR to Mistral-7BMistral-7B · MT-Benchaverage score: 5.68 -> 5.49Mixed
Apply Temporal SR to Llama3.2-3BLlama3.2-3B · MT-Benchaverage score: 4.03 -> 4.32Improved
Apply Temporal SR to Llama3.1-70BLlama3.1-70B · AlpacaEval 2.0win rate (%): 32.91 -> 33.66Improved
Add temporal offline references to GRPOQwen2.5-7B-Math · AIME25accuracy: 8.5 -> 10.8Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy