Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Following the Human Thread in Social Navigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SDA, a two-stage RL framework that encodes privileged human trajectories into a latent social-dynamics vector during training, then regresses that latent from the robot's state-action history so the policy can find and follow humans without trajectory access, evaluated on Habitat 3.0.
Problem
Social navigation requires a robot to find and follow a human from an egocentric view, but human trajectories (key cues) are only partially observable and privileged information such as humanoid GPS is unavailable at deployment.
Approach
Train a motion policy with a latent encoding of privileged human trajectories (stage 1), then freeze the policy and train an adapter MLP with supervised MSE regression to infer that latent from the robot's recent state and action history (stage 2), so trajectories are needed only at training time.
Key results
- Without privileged information, SDA Stage 2 reaches S 0.91 and SPS 0.45 vs. 0.76 and 0.34 for the Habitat 3.0 baseline.
- SDA Stage 2 raises Following Rate to 0.39 vs. 0.29 for the baseline; Episode Success 0.43 vs. 0.40 (Collision Rate 0.57 vs. 0.48 for the baseline).
- Adding ORCA to humanoids lowers collision rate from 0.57 to 0.37 while raising ES from 0.43 to 0.48.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use state-action history to adapt trajectory-based social dynamics instead of the sensor-only baseline | SDA · Habitat 3.0 Social Navigation | S: 0.76$^{\pm0.02}$ -> 0.91$^{\pm0.01}$ | Helped |
| use state-action history to adapt trajectory-based social dynamics instead of the sensor-only baseline | SDA · Habitat 3.0 Social Navigation | F: 0.29$^{\pm0.01}$ -> 0.39$^{\pm0.01}$ | Helped |
| use state-action history to adapt trajectory-based social dynamics instead of the sensor-only baseline | SDA · Habitat 3.0 Social Navigation | CR: 0.48$^{\pm0.03}$ -> 0.57$^{\pm0.02}$ | Hurt |
| add trajectory input instead of hGPS to the Stage 1 policy (ablation) | SDA Stage 1 · Habitat 3.0 Social Navigation | SPS: 0.62$^{\pm0.01}$ -> 0.46$^{\pm0.01}$ | Hurt |
| adapt trajectories instead of hGPS (ablation) | SDA Stage 2 · Habitat 3.0 Social Navigation | S: 0.70$^{\pm0.02}$ -> 0.91$^{\pm0.01}$ | Helped |
| add hGPS adaptation alongside trajectory adaptation (ablation) | SDA Stage 2 · Habitat 3.0 Social Navigation | S: 0.91$^{\pm0.01}$ -> 0.57$^{\pm0.06}$ | Hurt |
| add ORCA to human motion (ablation) | SDA · Habitat 3.0 Social Navigation | CR: 0.57$^{\pm0.02}$ -> 0.37$^{\pm0.01}$ | Helped |
| add ORCA to human motion (ablation) | SDA · Habitat 3.0 Social Navigation | S: 0.91$^{\pm0.01}$ -> 0.90$^{\pm0.01}$ | Hurt |
| reduce adapter update rate to once every two timesteps (ablation) | SDA · Habitat 3.0 Social Navigation | ES: 0.43$^{\pm0.02}$ -> 0.48$^{\pm0.02}$ | Helped |
| reduce adapter update rate to once every one hundred timesteps (ablation) | SDA · Habitat 3.0 Social Navigation | S: 0.91$^{\pm0.01}$ -> 0.85$^{\pm0.01}$ | Hurt |
| add depth-camera and bounding-box noise (ablation) | SDA · Habitat 3.0 Social Navigation | ES: 0.43$^{\pm0.02}$ -> 0.25$^{\pm0.02}$ | Hurt |
| reduce human speed by half (ablation) | SDA - S2 · Habitat 3.0 Social Navigation | ES: 0.43 $\pm$ 0.02 -> 0.70 $\pm$ 0.01 | Helped |
Surprising findings
- Trajectory-based latent adaptation outperforms adapting humanoid GPS, even though hGPS gives better Stage 1 results.
- Updating the adapter only every 100 steps slightly improves Episode Success, attributed to smoother following.
Limitations
- Evaluated only in simulation (Habitat 3.0), relying on trajectory information not readily available in real-world settings.
- Collision rate increases as the agent follows humans longer, so episode success gains are modest.
- Performance degrades substantially with multiple humans, and the policy was trained only with a single human.
Experimental setup
Datasets: Habitat 3.0 simulated humanoid episodes
Benchmarks: Habitat 3.0 Social Navigation
Metrics: Finding Success (S) SPS Following Rate (F) Collision Rate (CR) Episode Success (ES) Backup-Yield Rate Total Distance Following Distance
Baselines: Heuristic Expert Baseline (Habitat 3.0 SotA policy with humanoid GPS) Baseline+Proximity RMA
Builds on: DD-PPO Rapid Motor Adaptation Habitat 3.0
Closest papers by meaning
- SocialNav: Training Human-Inspired Foundation Model for Socially-Aware Embodied Navigation (CVPR 2026 · similarity 0.90)
- Social-Transmotion: Promptable Human Trajectory Prediction (ICLR 2024 · similarity 0.87)
- SocialCVAE: Predicting Pedestrian Trajectory via Interaction Conditioned Latents (AAAI 2024 · similarity 0.86)
- SocialCircle: Learning the Angle-based Social Interaction Representation for Pedestrian Trajectory Prediction (CVPR 2024 · similarity 0.85)
- Habitat 3.0: A Co-Habitat for Humans, Avatars, and Robots (ICLR 2024 · similarity 0.85)
- SocialMOIF: Multi-Order Intention Fusion for Pedestrian Trajectory Prediction (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.