Research map / Image generation / Fast diffusion sampling

Aligning Generative Denoising with Discriminative Objectives Unleashes Diffusion for Visual Perception

ICLR 2025 · Image generation › Fast diffusion sampling · approach: Denoising diffusion probabilistic models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ADDP, three plug-and-play alignments for diffusion-based perception (contribution-aware timestep sampling, diffusion-tailored data augmentation, and classifier-free-guidance correctional prompts as an interactive interface), improving Marigold depth, InstructPix2Pix referring segmentation, and InstructCV generalist models without architecture changes.

Problem

Generative diffusion models repurposed for perception show gaps versus discriminative objectives: uneven contribution of denoising timesteps, training-denoising distribution shift causing quality drops at later steps, and underperformance on multi-modal tasks like referring image segmentation.

Approach

Estimate per-timestep contribution factors c_t^2 (from diffusion formulation or via R^2 regression on perception metrics) and use them as timestep sampling probabilities; simulate denoising drift by task-specific augmentation of ground-truth targets with timestep-dependent intensity and x0-prediction; and use negative correctional prompts within classifier-free guidance as an interactive user inte

Key results

• On RefCOCO val, InstructPix2Pix-SD1.5 improves from 60.87 to 66.86 oIoU with ADDP, and SD2.0 from 64.96 to 69.14, catching up with some discriminative baselines.
• On Marigold depth estimation, ADDP lowers average AbsRel rank score to 1.4 (vs 2.4 for Marigold) and improves AbsRel/delta1 on all five zero-shot benchmarks.
• On generalist InstructCV, ADDP improves NYUv2 RMSE from 0.302 to 0.288, ADE20K mIoU from 46.67 to 48.40, and COCO mAP@0.5 from 46.6 to 48.1.

Measured effects reported in the paper

Surprising findings

• Perception quality (IoU) for referring segmentation drops at later denoising steps, and earlier steps contribute far more than later ones.
• Probability-scaling of timesteps outperforms loss-scaling, and x0-prediction is better than epsilon-prediction for the augmentation setup.

Limitations

• RIS still lags the best discriminative methods (e.g., UNINEXT, ReLA), so the paper does not claim state of the art.
• Diffusion models are pretrained for image generation with aesthetic filtering, which may hurt perception generalization.
• The agentic workflow is a proof-of-concept and not an ideal pipeline; the augmentation design may not be optimal.

Experimental setup

Datasets: Hypersim Virtual-KITTI RefCOCO RefCOCO+ G-Ref NYUv2 ADE20K COCO

Benchmarks: ETH3D ScanNet NYUv2 DIODE KITTI RefCOCO RefCOCO+ G-Ref ADE20K COCO

Metrics: AbsRel delta1 oIoU RMSE mIoU mAP@0.5

Baselines: Marigold InstructPix2Pix InstructDiffusion Unified-IO CRIS LAVT ReLA UNINEXT InstructCV

Builds on: DDPM InstructPix2Pix Marigold InstructCV classifier-free guidance

Closest papers by meaning

• Diff-2-in-1: Bridging Generation and Dense Perception with Diffusion Models (ICLR 2025 · similarity 0.92)
• What Matters When Repurposing Diffusion Models for General Dense Perception Tasks? (ICLR 2025 · similarity 0.90)
• Denoising Task Difficulty-based Curriculum for Training Diffusion Models (ICLR 2025 · similarity 0.90)
• DEEM: Diffusion models serve as the eyes of large language models for image perception (ICLR 2025 · similarity 0.90)
• Scaling Properties of Diffusion Models For Perceptual Tasks (CVPR 2025 · similarity 0.89)
• Traversing Distortion-Perception Tradeoff using a Single Score-Based Generative Model (CVPR 2025 · similarity 0.89)