Research map / 3D vision / 3D generation

Pose6DAug: Physically Plausible Multi-View Object Swapping for Robot Data Augmentation

NeurIPS 2026 · 3D vision › 3D generation · approach: Hand-object interaction generation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Pose6DAug augments successful VLA robot episodes by swapping the manipulated object with a 3D mesh rendered along a 6D pose trajectory, yielding multi-view consistent failure-targeted demonstrations without new data collection.

Problem

VLA policies fail on novel out-of-distribution objects, and collecting multi-view teleoperation data for each failure case is costly; existing 2D video-editing and simulation-based augmentations lack multi-view consistency or physical plausibility and have coverage limits.

Approach

Reuse a policy's own successful episodes by replacing the manipulated object with a reconstructed 3D mesh (SAM3D) rendered at a shared world-frame 6D pose derived from the original trajectory, with background inpainting and geometric perturbations (rotation, translation, scaling) for diversity.

Key results

• On failure episodes, Pose6DAug reaches 22.8% average success rate vs 16.4% for VACE and 15.8% for MimicGen, with 24.5% turnover ratio.
• Training only on augmented data for 8 hardest OOD meshes, Pose6DAug achieves 21.2% success (34/160) and recovers 7 of 8 objects, vs VACE 15.0% (5 of 8) and MimicGen 5.7% (2 of 8).
• On full evaluation, Pose6DAug achieves 49.5% average success vs 47.2% base policy, with OOD improving from 36.9% to 42.9%.

Surprising findings

• MimicGen-augmented data lowered overall performance below the base policy (46.0% vs 47.2%).
• VACE-augmented data yielded limited gains and in some configurations degraded performance.

Limitations

• Substitution constrained by grasp compatibility; drastically different scale or geometry yields implausible grasps.
• Depends on quality of external vision models (SAM3D mesh extraction, video inpainting), inheriting their failure modes.
• Direct mesh rendering may not capture complex lighting, shadows, or reflections.

Experimental setup

Datasets: RoboCasa365 Counter-to-Cabinet demonstrations Objaverse

Benchmarks: RoboCasa365 Counter-to-Cabinet

Metrics: success rate turnover ratio class recovery

Baselines: Base Policy MimicGen VACE

Builds on: MimicGen VACE GR00T

Closest papers by meaning

• BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models (NeurIPS 2025 · similarity 0.87)
• VersaCamVLA: Camera-Configurable VLA Policies for Robotic Manipulation (NeurIPS 2026 · similarity 0.87)
• OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints (CVPR 2025 · similarity 0.87)
• VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Models (NeurIPS 2026 · similarity 0.87)
• ActionUNet: Improving Robustness of VLA Models with Efficient Multi-scale Fine-tuning (NeurIPS 2026 · similarity 0.87)
• Beyond World-Frame Action Heads: Motion-Centric Action Frames for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.87)