Research map / Reinforcement learning / Imitation learning

Imitation from Observations with Trajectory-Level Generative Embeddings

NeurIPS 2026 · Reinforcement learning › Imitation learning · approach: Offline imitation from imperfect data · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TGE, an offline learning-from-observations method that uses a temporal Diffuser's latent embeddings and a log-distance particle entropy reward to label suboptimal D4RL data, then trains IQL or ReBRAC on it.

Problem

Offline imitation learning from observations (single expert state-only episode plus mixed suboptimal data) suffers from support mismatch between offline data and expert, where density-ratio methods give sparse or ill-conditioned signals.

Approach

Use the encoder of a trajectory-level diffusion planner (Diffuser) trained on suboptimal data as a state-only embedding, and define a dense surrogate reward as a log-distance kernel to k-nearest expert embeddings, derived from particle-based cross-entropy minimization, then run standard offline RL with this reward.

Key results

• TGE+ReBRAC reaches 88.86/111.34 style scores on random+few-expert: hopper 88.86 vs DILO 92.66, and halfcheetah 89.70 vs DILO 48.26 on random+few-expert.
• On medium+few-expert ant, TGE+ReBRAC scores 130.23 versus DILO 92.55 and SMODICE 89.11.
• TGE+IQL underperforms DILO on cloned+expert door (0.02 vs 100.90) and hammer (1.34 vs 54.29), which the authors attribute to density-ratio methods exploiting overlap in cloned data.

Surprising findings

• TGE sometimes surpasses the single provided expert demonstration's score, which the authors attribute to the one expert trajectory not being the best behavior in the dataset.
• TGE+IQL performs near zero on cloned+expert door and hammer, while density-ratio DILO does well there.

Limitations

• Diffusion pretraining adds roughly 4-6 hours on a single L40S GPU before reward annotation.
• Underperforms DILO on cloned Adroit door and hammer datasets.
• Results reported only for a single expert observation trajectory and specific mixture protocols.

Experimental setup

Datasets: D4RL hopper D4RL halfcheetah D4RL walker2d D4RL ant D4RL pen D4RL door D4RL hammer

Benchmarks: D4RL MuJoCo locomotion (hopper, halfcheetah, walker2d, ant) D4RL Adroit (pen, door, hammer)

Metrics: normalized D4RL score mean and std over five seeds

Baselines: Diffuser (BC) SMODICE PW-DICE DILO

Builds on: Diffuser IQL ReBRAC DILO SMODICE

Closest papers by meaning

• Diffusion Imitation from Observation (NeurIPS 2024 · similarity 0.91)
• How to Leverage Diverse Demonstrations in Offline Imitation Learning (ICML 2024 · similarity 0.89)
• Robust Visual Imitation Learning with Inverse Dynamics Representations (AAAI 2024 · similarity 0.89)
• Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning (NeurIPS 2024 · similarity 0.89)
• Diffusion Model-Augmented Behavioral Cloning (ICML 2024 · similarity 0.89)
• Markov Balance Satisfaction Improves Performance in Strictly Batch Offline Imitation Learning (AAAI 2025 · similarity 0.89)