Research map / Image generation / Fast diffusion sampling

Importance-Aware OBS Pruning for Diffusion Models

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Training-free diffusion acceleration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes training-free importance-aware OBS pruning for diffusion models, weighting the Hessian by spatial importance maps (CFG magnitude, Canny edges, or detector-masked CFG) to better preserve semantically salient content than OBS-Diff.

Problem

One-shot second-order pruning of text-to-image diffusion models (OBS-Diff) treats all spatial locations uniformly, so at high sparsity it degrades subject fidelity and structure in perceptually important regions.

Approach

Reweight the OBS layer-wise reconstruction objective by a spatial importance map (from CFG delta magnitude, Canny edges, or object-detector masks), equivalently accumulating the Hessian from importance-filtered activations, keeping the closed-form OBS update unchanged.

Key results

• On SD3-Medium at 50% unstructured sparsity, Ours (CFG) reaches ImageReward 0.76 vs 0.71 for OBS-Diff and MUSIQ 66.30 vs 65.98.
• On PixArt-Sigma at 60% unstructured sparsity, Ours (CFG) improves ImageReward from 0.49 (OBS-Diff) to 0.52, and Ours (Canny) reaches 0.53.
• On PixArt-Sigma structured pruning at 40%, ImageReward goes from -0.04 (OBS-Diff) to 0.30 (Ours CFG).

Surprising findings

• CLIP Score often favors the OBS-Diff baseline on individual samples even when human raters prefer the proposed method, with CLIP agreeing with human preference on only 25% of the 20 sampled cases.
• On several unstructured settings the baseline achieves equal or higher CLIP Score than the proposed method (e.g., SD3-Medium 30% sparsity), so gains are mostly in ImageReward and MUSIQ.

Limitations

• CFG-based masks can be noisy or incomplete for abstract prompts, small objects, or cluttered scenes.
• Post-training only; does not explore finetuning to recover quality at very high sparsity.
• Unstructured sparsity gives no practical latency gains without sparse-kernel hardware support.

Experimental setup

Datasets: GCC3M MS-COCO 2017

Benchmarks: MS-COCO 2017 validation prompts

Metrics: CLIP Score ImageReward MUSIQ human rating (5-level)

Baselines: OBS-Diff

Builds on: OBS-Diff Optimal Brain Surgeon classifier-free guidance

Closest papers by meaning

• Attention-Driven Training-Free Efficiency Enhancement of Diffusion Models (CVPR 2024 · similarity 0.88)
• ALTER: All-in-One Layer Pruning and Temporal Expert Routing for Efficient Diffusion Generation (NeurIPS 2025 · similarity 0.87)
• Exploiting Diffusion Prior for Generalizable Dense Prediction (CVPR 2024 · similarity 0.86)
• On improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models (NeurIPS 2024 · similarity 0.86)
• DiTFastAttn: Attention Compression for Diffusion Transformer Models (NeurIPS 2024 · similarity 0.86)
• Exploiting Diffusion Prior for Real-World Image Dehazing with Unpaired Training (AAAI 2025 · similarity 0.86)