Research map / Image generation / Autoregressive and latent generation

Noise-Guided Predicate Representation Extraction and Diffusion-Enhanced Discretization for Scene Graph Generation

ICML 2025 · Image generation › Autoregressive and latent generation · approach: Classical generative image synthesis

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes NoDIS, which extracts noise-guided predicate representations, expands them with a conditional diffusion feature reconstruction module, and aligns same-class predicates via a learnable discretization mapper to reduce long-tail bias in scene graph generation.

Problem

Scene graph generation under long-tailed predicate distributions suffers biased predictions: tail predicates are under-learned, and independent per-sample predicate representations make the decision layer hard to train.

Approach

Combine a one-step noise-guided denoising extractor with a conditional diffusion model that reconstructs predicate features to increase intra-class diversity, then apply a VQ-style discretization mapper that snaps reconstructed features to learnable predicate prototypes so same-class predicates share consistent representations for the decision layer.

Key results

• On VG PredCls, +NoDIS on PE-Net raises mR@50/100 from 31.5/33.8 to 38.72/41.93 (per the paper's table).
• On Open Images V6, Transformer-NoDIS reaches 48.93 mR@50 and 58.94 F@50 versus 49.8 for SGTR-style baselines' F@50 (reported in Table 2).
• Ablation: adding diffusion enhancement and discretization together improves R@50/mR@50 over the baseline, with a reported 1.34% gain versus the no-module baseline.

Measured effects reported in the paper

Surprising findings

• Introducing diffusion enhancement or discretization alone yields no noticeable gain and can degrade performance; only the combination helps.
• R@K sometimes drops with NoDIS (e.g., Transformer-based PredCls R@K), with the authors attributing this to mR@K emphasizing balanced head/body/tail accuracy.

Limitations

• Gains on R@K are inconsistent; the method may not be optimal for recall-based metrics.
• Parameter count increases by about 40-60M per integrated baseline.

Experimental setup

Datasets: Visual Genome GQA Open Images V4 Open Images V6

Benchmarks: Visual Genome GQA Open Images V4 Open Images V6

Metrics: R@K mR@K F@K wmAP_rel wmAP_phr score_wtd

Baselines: Motifs VCTree Transformer PE-Net BGNN DBiased HML CFA MEET BiC ALF ST-SGG

Builds on: PE-Net Transformer Motifs VQ-VAE DDPM

Closest papers by meaning

• Panoptic Scene Graph Generation with Semantics-Prototype Learning (AAAI 2024 · similarity 0.89)
• DiffVsgg: Diffusion-Driven Online Video Scene Graph Generation (CVPR 2025 · similarity 0.89)
• Leveraging Predicate and Triplet Learning for Scene Graph Generation (CVPR 2024 · similarity 0.88)
• RA-SGG: Retrieval-Augmented Scene Graph Generation Framework via Multi-Prototype Learning (AAAI 2025 · similarity 0.88)
• Conformal Prediction and MLLM aided Uncertainty Quantification in Scene Graph Generation (CVPR 2025 · similarity 0.88)
• Adaptive Visual Scene Understanding: Incremental Scene Graph Generation (NeurIPS 2024 · similarity 0.87)