Research map / Reinforcement learning / Visual and representation RL
Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use cross-modal Transformer fusion instead of State-Depth-Concat | LocoTransformer · Unseen Wide Obs. (trained on Thin Obs.) | Collision Happened: 331.1±192.8 -> 82.2 ± 103.8 | Helped |
| use cross-modal Transformer fusion instead of State-Depth-Concat | LocoTransformer · Moving Obstacle | Distance Moved (m): 16.3 ± 1.7 -> 11.3 ± 2.9 | Hurt |
| use cross-modal Transformer fusion instead of State-Depth-Concat | LocoTransformer · Moving Obstacle | Collision Happened: 88.4 ± 34.0 -> 67.9 ± 18.1 | Helped |
| use cross-modal Transformer fusion instead of State-Depth-Concat | LocoTransformer · Thin Obs. & Sph. | Distance Moved (m): 13.1±2.3 -> 15.2±1.8 | Helped |
| reduce visual tokens from N=4 to N=1 (ablation) | LocoTransformer · Thin Obs. & Sph. | Episode Return: 1551.5 ± 120.4 -> 1204.8 ± 243.6 | Hurt |
| reduce visual tokens from N=4 to N=2 (ablation) | LocoTransformer · Thin Obs. & Sph. | Episode Return: 1551.5 ± 120.4 -> 1418.1± 167.8 | Hurt |
| use L=1 Transformer layer instead of L=2 (ablation) | LocoTransformer · Thin Obs. & Sph. | Episode Return: 1551.5 ± 120.4 -> 1509.7 ± 244.8 | No clear effect |
| increase Transformer layers from L=2 to L=3 (ablation) | LocoTransformer · Thin Obs. & Sph. | Episode Return: 1551.5 ± 120.4 -> 1423.5 ± 100.7 | No clear effect |
| replace spatial cross-modal attention with simple feature concatenation | LocoTransformer · Mountain | Episode Return | Hurt |
Closest papers by meaning
- Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response (ICLR 2024 · similarity 0.88)
- Coupling Vision and Proprioception for Navigation of Legged Robots (CVPR 2022 · similarity 0.88)
- METRA: Scalable Unsupervised RL with Metric-Aware Abstraction (ICLR 2024 · similarity 0.85)
- ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
- Learning to Navigate Efficiently and Precisely in Real Environments (CVPR 2024 · similarity 0.85)
- Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.