Research map / Multi-agent systems and games / Human-AI and goal-conditioned
AD3: Implicit Action is the Key for World Models to Distinguish the Diverse Visual Distractors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AD3, which learns an Implicit Action Generator to infer latent actions of visual distractors and trains separate Dreamer-style world models for task-relevant and task-irrelevant components, handling both heterogeneous and homogeneous (agent-shifted) distractors.
Problem
Model-based visual RL struggles to separate task-relevant state from homogeneous distractors (e.g., a shifted agent resembling the controllable one), while prior work mostly addressed heterogeneous distractors like noisy video backgrounds.
Approach
Assume distractors have implicit actions (IABMDP) and infer them with an Implicit Action Generator (TAID inverse dynamics + FIAD forward dynamics, cycle consistency, categorical bottleneck); then train two separated RSSM world models, conditioned on agent actions and inferred implicit actions respectively, with policy learning only in the task-relevant branch.
Key results
- AD3 achieves 658 on Cheetah Run + NBV and 653 on Cheetah Run + AS, versus 474 for the best baseline TIA on AS (table reports consistent top performance across 8 settings).
- On Walker Run + AS, AD3 scores 435 vs. 330 for Dreamer and 157 for Denoised MDP.
- Using ground-truth distractor actions as oracle or inferred implicit actions yields near-perfect task/distractor separation, while using agent action or no action for the distractor model fails.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AD3 instead of TIA | Visual control agent · Cheetah Run + NBV | return: 432 $\pm$ 172 -> 658 $\pm$ 104 | Helped |
| Use AD3 instead of Dreamer | Visual control agent · Cheetah Run + AS | return: 540 $\pm$ 109 -> 653 $\pm$ 130 | Helped |
| Use AD3 instead of TIA | Visual control agent · Walker Run + NBV | return: 293 $\pm$ 129 -> 308 $\pm$ 37 | Helped |
| Use AD3 instead of Dreamer | Visual control agent · Walker Run + AS | return: 330 $\pm$ 51 -> 435 $\pm$ 85 | Helped |
| Use AD3 instead of Denoised MDP | Visual control agent · Finger Spin + NBV | return: 331 $\pm$ 146 -> 491 $\pm$ 189 | Helped |
| Use AD3 instead of Denoised MDP | Visual control agent · Finger Spin + AS | return: 480 $\pm$ 46 -> 515 $\pm$ 228 | Helped |
| Use AD3 instead of TIA | Visual control agent · Hopper Hop + NBV | return: 50 $\pm$ 48 -> 57 $\pm$ 20 | Helped |
| Use AD3 instead of Dreamer | Visual control agent · Hopper Hop + AS | return: 92 $\pm$ 54 -> 99 $\pm$ 31 | Helped |
| Add categorical variables rather than use VQ (ablation) | IAG in AD3 · Cheetah Run + NBV | return: 276 $\pm$ 176 -> 658 $\pm$ 104 | Helped |
| Add agent action to TAID (ablation) | IAG in AD3 · Walker Run + AS | return: 240 $\pm$ 122 -> 435 $\pm$ 85 | Helped |
| Add cycle consistency loss (ablation) | IAG in AD3 · Hopper Hop + AS | return: 62 $\pm$ 66 -> 99 $\pm$ 31 | Helped |
| Add difference reconstruction loss (ablation) | IAG in AD3 · Finger Spin + NBV | return: 549 $\pm$ 273 -> 491 $\pm$ 189 | Hurt |
Surprising findings
- Implicit actions inferred by IAG achieved higher test return than the ground-truth distractor action on Cheetah Run.
- Removing difference reconstruction slightly improved performance on Finger Spin + NBV (549 vs. 491).
Limitations
- Performance on harder combined NBV + AS distractors is lower (e.g., Cheetah Run NBV+AS reaches 642 only after 1M steps), and the method relies on shifted-agent dynamics being inferable.
- Evaluated only on DMC-style tasks plus a few additional environments with simulated distractors; no real-world evaluation.
- Uses the simplest Dreamer backbone for world models, so absolute gains are tied to that backbone.
Experimental setup
Datasets: DeepMind Control Suite (Cheetah Run, Walker Run, Finger Spin, Hopper Hop) Kinetics driving-car videos (NBV backgrounds) Car Racing (OpenAI Gym) Jaco Arm
Benchmarks: DMC Cheetah Run DMC Walker Run DMC Finger Spin DMC Hopper Hop Car Racing + NBV Jaco Arm + NBV Jaco Arm + AS
Metrics: episodic return (mean and std over 4 seeds, 10 test episodes
Baselines: DBC Dreamer TIA Denoised MDP
Builds on: Dreamer TIA Denoised MDP DreamerV2 FICC
Closest papers by meaning
- Latent Action Learning Requires Supervision in the Presence of Distractors (ICML 2025 · similarity 0.86)
- DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors (NeurIPS 2024 · similarity 0.85)
- Grounding Video Models to Actions through Goal Conditioned Exploration (ICLR 2025 · similarity 0.84)
- Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies (NeurIPS 2025 · similarity 0.84)
- Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representations (NeurIPS 2025 · similarity 0.83)
- NEvo: Neural-Guided Evolutionary Video Synthesis for Dynamic Visual Selectivity (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.