Research map / Large language models / Reasoning and chain of thought
Time-Frequency Token Advantage Clipping for Training Efficient Large Reasoning Model
Rong Bao, Bo Wang, Xiao Wang, Hongyu Li, Rui Zheng, Leszek Rutkowski, Qi Zhang, Liang Ding, Dacheng Tao
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes time-frequency token advantage clipping to train large reasoning models with token-level control over long chain-of-thought, improving reasoning efficiency.
Problem
Long chain-of-thought reasoning in large reasoning models is inefficient: models overthink simple problems or underthink complex ones, and sequence-level length penalties are too coarse to separate core logic from verbose language.
Approach
Applies token-level advantage clipping guided by time-frequency analysis of tokens so that core reasoning tokens are distinguished from verbose language, instead of sequence-level length penalties.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove historical token count (time dimension) (ablation) | TFAC Qwen3-8B · AIME (Table 4) | Avg@32: 71.4 -> 73.9 | Improved |
| add intra-path frequency clipping (ablation) | TFAC Qwen3-8B · AIME (Table 4) | Avg@32: 70.8 -> 73.9 | Improved |
| add time-count budget (ablation) | TFAC R1-Distill-32B · AIME (Table 4) | Avg@32: 68.4 -> 71.2 | Improved |
| add frequency clipping (ablation) | TFAC R1-Distill-32B · AIME (Table 4) | Avg@32: 68.9 -> 71.2 | Improved |
| add dynamic sampling (ablation) | TFAC · AIME (Table 4) | Avg@32: 73.9 -> 74.2 (Qwen3-8B); 71.2 -> 72.9 (R1-Distill-32B) | Improved |
| apply penalty to advantage clipping instead of terminal reward (TFRS) (ablation) | TFAC · AIME 2024/2025 (Table 2) | Avg@32: 71.9 -> 73.9 (Qwen3-8B, gamma/lambda 0.8); 62.2 -> 66.0 (R1-Distill-32B) | Improved |
| TFAC vs DAPO w. forking token | TFAC · Table 1 benchmarks, Qwen3-8B | Avg@32: MATH-500 95.32 -> 94.96; AIME 2024 77.60 -> 77.80; LCB 58.52 -> 58.15 | Mixed |
| TFAC vs DAPO w. forking token | TFAC · Table 1 benchmarks, R1-Distill-32B | Avg@32: MATH-500 96.90 -> 97.13; AIME 2025 53.20 -> 55.50 | Improved |
| TFAC vs explicit length-penalty methods | TFAC · Table 1 | Avg@32: Qwen3-8B AIME 2025: HAPO 58.00 -> TFAC 69.50 | Improved |
| TFAC vs untrained base on Qwen3-8B MATH-500 | TFAC · MATH-500, Qwen3-8B | Avg@32: 94.25 -> 94.96 | No clear effect |
| TFAC vs DAPO w. fork | TFAC · GPQA (OOD) | Avg@32: 64.92 -> 66.65 (Qwen3-8B); 63.52 -> 65.35 (R1-Distill-32B) | Improved |
| TFAC under small generation budget (~8k) | TFAC · MATH reasoning, token budget scaling | performance | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- HAPO — HAPO: Training Language Models to Reason Concisely via History-Aware Policy Optimization (AAAI 2026) · better on 8 of 8 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Demystifying Long Chain-of-Thought Reasoning (ICML 2025 · similarity 0.91)
- Dynamic Early Exit in Reasoning Models (ICLR 2026 · similarity 0.90)
- Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.90)
- Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens (ICML 2026 · similarity 0.90)
- Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression (ICML 2026 · similarity 0.90)
- TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
