Research map / Graph learning / Knowledge and temporal graphs

Towards Unbiased and Robust Spatio-Temporal Scene Graph Generation and Anticipation

CVPR 2025 · Graph learning › Knowledge and temporal graphs · approach: Scene graph generation · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes IMPARTAIL, a curriculum-guided loss-masking training framework that adapts object-centric VidSGG and SGA models to learn unbiased estimators on Action Genome, and introduces robustness evaluations under input corruptions.

Problem

Long-tailed predicate distributions in Action Genome cause video scene graph generation (VidSGG) and scene graph anticipation (SGA) models to be biased toward head relationship classes, and they degrade under input distribution shifts.

Approach

Instead of adding architectural debiasing modules or reweighting data, mask the loss of head-class predicate labels with a curriculum-driven masking ratio that increases over epochs, so the model focuses on tail relationships while training with partial gradients; applicable as a plug-in to existing object-centric VidSGG/SGA pipelines.

Key results

• On VidSGG PREDCLS with No Constraint, STTran mR@10 improves from 45.7 to 65.5 (+43.3%) with IMPARTAIL.
• On SGA, SceneSayerSDE with IMPARTAIL improves AGS mR@10 from 6.4 to 7.4 and PGAGS no-constraint mR@10 from 15.4 to 21.8 (reported gains of about 20% for SceneSayer ODE/SDE).
• Under corruptions (Gaussian Noise, Fog, Frost, Brightness, Sun Glare), IMPARTAIL-trained STTran in PREDCLS drops less than baseline, e.g., Gaussian Noise no-constraint mR@10 falls 45.7 to 31.4 for baseline vs 65.5 to 57.5 with IMPARTAIL.

Measured effects reported in the paper

Surprising findings

• Some robustness results show IMPARTAIL underperforming baselines on recall (R@K) under certain corruptions, e.g., Fog and Brightness recall drops of about 10% for DSGDetr in SGCLS/PREDCLS.
• A fixed high masking ratio of 0.9 (reported as 0.99) performed similarly in mean recall to a gradually increasing masking schedule, questioning the added value of the curriculum.

Limitations

• Evaluated only on Action Genome.
• Curriculum schedule offers little gain over a fixed high masking ratio according to the authors' own remark.
• Recall (non-mean) can drop relative to baselines under some corruptions.

Experimental setup

Datasets: Action Genome

Benchmarks: Action Genome

Metrics: Recall@K mean Recall@K

Baselines: STTran DSGDetr STTran+ STTran++ DSGDetr+ DSGDetr++ SceneSayerODE SceneSayerSDE TEMPURA

Builds on: STTran DSGDetr SceneSayer TEMPURA

Closest papers by meaning

• TD²-Net: Toward Denoising and Debiasing for Video Scene Graph Generation (AAAI 2024 · similarity 0.90)
• DiffVsgg: Diffusion-Driven Online Video Scene Graph Generation (CVPR 2025 · similarity 0.89)
• A Closer Look at Dynamic Scene Graph Generation in the Era of Multimodal Large Language Models (NeurIPS 2026 · similarity 0.88)
• HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation (CVPR 2024 · similarity 0.88)
• LASER: A Neuro-Symbolic Framework for Learning Spatio-Temporal Scene Graphs with Weak Supervision (ICLR 2025 · similarity 0.88)
• OED: Towards One-stage End-to-End Dynamic Scene Graph Generation (CVPR 2024 · similarity 0.88)