AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

PADA-Coder: Improving Plan-Following Code Generation via Perturbation-Verified Attention Distillation and Dynamic Alignment

Yihong Huang, Ke Qin, Rongzheng Wang, Muquan Li, Jiakai Li, Xiurui Xie, Shuang Liang

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PADA-Coder, which improves plan-following code generation in LLMs by distilling attention toward plan-relevant tokens, verified via perturbation, with dynamic alignment.

Problem

In Plan-then-Code pipelines, adding the plan and verification report enlarges context, so the model's attention drifts to irrelevant tokens and recent code, causing it to overlook critical constraints from the plan.

Approach

Distills attention patterns into the model so it attends to plan-relevant tokens, using perturbation to verify which attention targets matter, combined with dynamic alignment during generation (based on abstract only; details unavailable).

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add attention alignment training (ablation)PADA (Qwen3-4B) · APPSPass@1: 57.8 -> 65.1Improved
add attention alignment training (ablation)PADA (Qwen3-4B) · HumanEvalPass@1: 93.9 -> 96.8Improved
add progress-aware sliding window (ablation)PADA (Qwen3-4B) · APPSPass@1: 61.8 -> 65.1Improved
add progress-aware sliding window (ablation)PADA (Qwen3-4B) · HumanEvalPass@1: 93.2 -> 96.8Improved
add perturbation-verified key token selection (ablation)PADA (Qwen3-4B) · APPSPass@1: 63.1 -> 65.1Improved
add perturbation-verified key token selection (ablation)PADA (Qwen3-4B) · HumanEvalPass@1: 95.1 -> 96.8Improved
add difficulty-aware gating (ablation)PADA (Qwen3-4B) · APPSPass@1: 65.1 -> 65.1No clear effect
add difficulty-aware gating (ablation)PADA (Qwen3-4B) · HumanEvalPass@1: 96.3 -> 96.8No clear effect
use Qwen3-32B as teacher instead of none (ablation)PADA (Qwen2.5-Coder-7B) · APPSPass@1: 58.1 -> 61.3Improved
use similar-scale Qwen3-4B as teacher (ablation)PADA (Qwen2.5-Coder-7B) · APPSPass@1: 58.1 -> 58.4No clear effect
use GPT-5.2 prompted key tokens as teacher (ablation)PADA (Qwen2.5-Coder-7B) · HumanEval, MBPP+, APPSPass@1: 58.1 -> 58.8Mixed
add teacher model to PADA (ablation)PADA (Qwen3-4B) · Code benchmarks average (7 benchmarks)Pass@1 average: 75.3 -> 77.8Improved

Experimental setup

Builds on: plan-then-code

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy