Research map / Reinforcement learning / Visual and representation RL
ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents MS-HAB, a GPU-accelerated ManiSkill3 port of the Home Assistant Benchmark with low-level whole-body control, RL and IL baselines, and rule-based event labeling for filtering demonstrations.
Problem
Home-scale mobile manipulation benchmarks lack fast, realistic low-level control, scalable demonstration generation, and controlled behavior filtering in one unified framework.
Approach
Re-implements the HAB on GPU-parallel ManiSkill3 with realistic grasping (no magical grasp), trains per-object RL subtask policies, and uses privileged simulator events to categorize trajectories into success/failure modes for filtering demonstrations used in IL.
Key results
- Achieves 4299.18 SPS at 15.35 GB VRAM versus Habitat 2.0's 1397.65 SPS at 22.60 GB, a 3.08x speedup with 32% less VRAM.
- Per-object RL Pick success on TidyHouse train is 81.75% versus 71.63% for all-object RL policies.
- Close Fridge RL policy achieves 0.00% success on validation scenes because the fridge door opens into a wall.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Scale GPU simulation to 4096 environments | MS-HAB versus Habitat 2.0 · Interact benchmark | samples per second: 1397.65 ± 11.02 -> 4299.18 ± 26.36 SPS | Helped |
| Scale GPU simulation to 4096 environments | MS-HAB versus Habitat 2.0 · Interact benchmark | GPU memory: 22.60 GB -> 15.35 GB | Helped |
| Use MS-HAB live rendering with ray tracing instead of Behavior-1k | ray-tracing benchmark · Ray-tracing benchmark | samples per second: 19.92 ± 0.04 -> 69.90 ± 0.25 SPS | Helped |
| Use MS-HAB live rendering with ray tracing instead of Behavior-1k | ray-tracing benchmark · Ray-tracing benchmark | GPU memory: 7.62 ± 0.04 -> 6.26 ± 0.00 GB | Helped |
| Train per-object Pick policies rather than all-object policies (ablation) | RL Pick policies · TidyHouse and PrepareGroceries Pick | success once rate: TidyHouse train: 71.63 -> 81.75; PrepareGroceries train: 51.88 -> 66.57 | Helped |
| Train per-object Place policies rather than all-object policies (ablation) | RL Place policies · PrepareGroceries Place | success once rate: 53.37 -> 60.22 (Train); 58.63 -> 65.67 (Val) | Helped |
| Train per-object Place policies rather than all-object policies (ablation) | RL Place policies · TidyHouse and SetTable Place | success once rate: TidyHouse Val: 66.07 -> 65.97; SetTable Val: 68.25 -> 67.06 | No clear effect |
| Filter IL demonstrations to place-in-goal trajectories (ablation) | PrepareGroceries Place IL · PrepareGroceries Place | Place : Drop trajectory ratio: 3.17 : 1 (Train); 2.55 : 1 (Val) | Helped |
| Filter IL demonstrations to drop-to-goal trajectories (ablation) | PrepareGroceries Place IL · PrepareGroceries Place | Place : Drop trajectory ratio: 1 : 2.22 (Train); 1 : 2.86 (Val) | Hurt |
| Use a 50/50 split of place and drop demonstrations (ablation) | PrepareGroceries Place IL · PrepareGroceries Place | Place : Drop trajectory ratio: 1 : 1.71 (Train); 1 : 1.41 (Val) | Mixed |
| Increase demonstrations per object (ablation) | IL Pick and Place policies · TidyHouse/PrepareGroceries/SetTable Pick and Place | success once rate: 0.00 ± 0.00 (1 demo) -> 0.62 ± 0.09 (1000 demos) | Helped |
| Use diffusion policy instead of behavior cloning | IL baselines · TidyHouse Pick train | success once rate (%): 61.11 -> 28.37 | Hurt |
Surprising findings
- Diffusion Policy baselines generally underperform simple BC baselines (e.g., TidyHouse Pick train 28.37% vs 61.11%).
- Filtering demonstrations by behavior biases IL policies, but IL policies still perform the opposite behavior fairly often.
Limitations
- Baselines leave significant room for improvement; long-horizon success is low (upper bound below 20% in simplified tasks).
- No transfer to real robots is claimed.
- Navigation is replaced by teleportation, so the benchmark focuses on low-level manipulation.
Experimental setup
Datasets: ReplicaCAD YCB objects HAB episode configurations
Benchmarks: MS-HAB TidyHouse MS-HAB PrepareGroceries MS-HAB SetTable Interact benchmark
Metrics: SPS GPU memory usage success once rate progressive completion rate
Baselines: Habitat 2.0 Behavior-1k RoboCasa Diffusion Policy BC (all-object RL)
Builds on: ManiSkill3 Home Assistant Benchmark Habitat 2.0 D4PG M3
Closest papers by meaning
- RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation (NeurIPS 2025 · similarity 0.87)
- DynScene: Scalable Generation of Dynamic Robotic Manipulation Scenes for Embodied AI (CVPR 2025 · similarity 0.87)
- Zero-Shot Robotic Manipulation with Pre-Trained Image-Editing Diffusion Models (ICLR 2024 · similarity 0.87)
- SkiP: When to Skip and When to Refine for Efficient Robot Manipulation (NeurIPS 2026 · similarity 0.87)
- DeformGen: Dynamics-Based Topology Augmentation for Deformable Manipulation Policy Learning (NeurIPS 2026 · similarity 0.87)
- AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.