Research map / Large language models / Reasoning and chain of thought

Verifiers in the Loop: Decoding Time Verification for Code Translation

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Decoding Time Verification (DTV), a state-machine controller that interleaves code generation with deterministic compiler/type-checker oracles at structural boundaries, with rollback, escalation, and feedback, evaluated on C-to-Rust and JS-to-TS translation.

Problem

Outcome-only verification of LLM code translation applies compilers/tests only after full generation, so early errors corrupt the autoregressive context and are rarely fixed; learned process reward models are costly and brittle.

Approach

Treat structural boundaries (statements, blocks, functions, programs) as meta steps where deterministic verifiers are invoked on rendered prefixes; commit only unanimously passing prefixes, roll back to the smallest scope with diagnostic-augmented prompts, and escalate scope on repeated failures.

Key results

• DTV/self-refine raises Qwen3-4B compile pass rate from 72.3% to 82.0% on C-to-Rust and from 33.3% to 46.0% on JS-to-TS versus naive/self-refine, with fewer tokens per case.
• In-loop verification alone (single attempt) raises pass rate from 16.3% to 47.0% on C-to-Rust and from 4.7% to 21.3% on JS-to-TS.
• On 100 C-to-Rust ablation cases, removing in-loop recovery (detect-and-abort) drops pass rate from 85.0% to 70.0%, the largest drop among the three mechanisms.

Surprising findings

• On C-to-Rust, DTV's compile-pass gain does not translate into functional correctness: differential-testing success is 8.3% for DTV/self-refine vs 9.3% for naive/self-refine (not significant).
• Best-of-N stays at 7.0% on JS-to-TS for all N up to 32, suggesting resampling repeats the same under-typed structure.

Limitations

• In-loop oracles are compilation-style only; differential testing is reserved for post-hoc evaluation, and effectiveness with behavioral oracles is unaddressed.
• Gains weaken on nonlocal semantic fixes and near-saturated baselines (Gemma C-to-Rust); evaluation limited to two translation tasks and three models.
• Fixed retry and bailout thresholds are not shown to be optimal; wall-clock overhead on TypeScript/ESLint stack.

Experimental setup

Datasets: Project CodeNet TypeWeaver

Benchmarks: CodeNet-derived C-to-Rust set (300 cases) TypeWeaver-derived JS-to-TS set (150 cases)

Metrics: compile pass rate functional success rate via differential testing average tokens per case wall-clock time

Baselines: naive/one-shot naive/self-refine best-of-N S* DTV-no-feedback DTV-no-escalation DTV-detect-and-abort

Builds on: self-refinement S* ROCODE EG-CFG VerMCTS

Closest papers by meaning

• AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement (ICML 2025 · similarity 0.87)
• Variation in Verification: Understanding Verification Dynamics in Large Language Models (ICLR 2026 · similarity 0.87)
• T1: Tool-integrated Verification for Test-time Compute Scaling in Small Language Models (ICLR 2026 · similarity 0.86)
• LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code (ICLR 2025 · similarity 0.86)
• Taming Imperfect Process Verifiers: A Sampling Perspective on Backtracking (ICLR 2026 · similarity 0.86)
• CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer (ICML 2026 · similarity 0.86)