AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Global Context Guidance for Diffusion Large Language Models

Jeimin Jeon, Bumsub Ham

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Parallel and early-exit decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes global context guidance for diffusion LLMs, building on weak-model guidance methods, to steer denoising predictions of the base model.

Problem

Existing guidance methods for diffusion LLMs use a weak model to guide base-model denoising, but the weak model's influence during denoising is unclear and computing its predictions is costly.

Approach

Guidance based on global context rather than a separate weak model's per-step predictions; the abstract is truncated so the exact mechanism is not stated.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Global Context Guidance on global headsLLaDA-8B · GSM8K-CoTaccuracy: 46.7 -> 48.4Improved
add U-CFG weak-model guidanceLLaDA-8B · GSM8K-CoTaccuracy: 46.7 -> 36.7Hurt
add A-CFG weak-model guidanceLLaDA-8B · GSM8K-CoTaccuracy: 46.7 -> 34.7Hurt
replace weak-model guidance with GCGLLaDA-8B · inference costFLOPs per step: 31.8T -> 15.9TImproved
guide local heads instead of global heads (LCG) (ablation)GCG · GSM8K-CoT, Ruleraccuracy: Ruler 29.5 -> 26.0 (LLaDA-8B); GSM8K 5.4 -> 2.7 (Dream-7B)Hurt
apply GCG to least sensitive blocks (LSB) (ablation)GCG · Ruleraverage score: 29.5 -> 23.4Hurt
sweep guidance scale s from 0.1 to 0.5 (ablation)GCG on LLaDA-8B · Ruleraverage score: 29.5 (s=0.3) -> 25.7 (s=0.5)Mixed
swap calibration set (GSM8K-CoT, HumanEval, Ruler) for head selection (ablation)GCG · LLaDA-8B benchmarksaccuracyNo clear effect
apply GCG to Llama-3.2-3B and RoBERTa-baseGCG · ARC-c, HumanEval, MBPP, SQuAD v2, GLUEaccuracy: e.g. HumanEval 51.2 -> 51.8; GLUE 84.70 -> 84.66Mixed

Experimental setup

Builds on: weak-model guidance for diffusion LLMs

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy