Research map / 3D vision / Human pose estimation
SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Distills view-consistent 3D DINO feature fields from sparse RGBD views with a contrastively trained per-point refinement MLP and vote-based point pruning, enabling one-shot dexterous hand grasping transfer on rigid and deformable objects.
Problem
One-shot transfer of dexterous manipulation skills across object poses, shapes, and categories when only sparse fixed-camera RGBD views are available, since existing feature-field methods rely on dense views and are limited to simple parallel grippers.
Approach
Back-project DINO features onto a fused multi-view point cloud, refine them with a lightweight per-point MLP trained contrastively between overlapping views of a single source scene, prune points with low cross-view feature consensus (Hough-like voting), and optimize a 24-DoF hand pose by matching feature fields between demonstration and target.
Key results
- On rigid objects, Ours achieves 100% on Box1->Box2, 80% on Bowl1->Mug, and 90% on Bowl1->BeerBarrel versus 0% and 0%-30% for DFF on the harder transfers.
- On deformable objects, Ours achieves 100% on Monkey->MonkeyScene and 60% on Monkey->SmallBear versus 40% and 0% for DFF.
- Ours outperforms the naive DFF baseline on nearly all targets, while UniDexGrasp++ (simulation) reaches only 6.2%-66.9% success.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SparseDFF instead of DFF for rigid-object grasp transfer | One-shot dexterous grasping · Rigid object setups | success rate: Box1: 90% -> 100%; Box2: 0% -> 100%; Bowl2: 0% -> 80%; CatBowl: 30% -> 60%; Mug: 0% -> 80%; FloatingMug: 20% -> 40%; Bee | Helped |
| Use SparseDFF instead of DFF | One-shot dexterous grasping · Drill demo → Drill | success rate: 100% -> 100% | No clear effect |
| Use SparseDFF instead of DFF | One-shot dexterous grasping · Bowl1 demo → Bowl1 | success rate: 100% -> 100% | No clear effect |
| Use SparseDFF instead of DFF for deformable-object grasp transfer | One-shot dexterous grasping · Deformable object setups | success rate: Monkey: 90% -> 100%; MonkeyScene: 40% -> 100%; SmallBear: 0% -> 60%; BigBear: 20% -> 80%; SmallBear demo → Monkey: 0% -> | Helped |
| Use SparseDFF instead of DFF | One-shot dexterous grasping · SmallBear demo → SmallBear | success rate: 90% -> 90% | No clear effect |
| Add the feature refinement network (ablation) | SparseDFF · Feature energy field for end-effector pose optimization | feature consistency | Helped |
| Add point pruning (ablation) | SparseDFF · End-effector pose optimization | optimization stability | Helped |
Surprising findings
- The naive DFF baseline achieves 90%-100% on same-object transfers but drops to 0% on several cross-object targets such as Bowl1->Bowl2 and Monkey->SmallBear.
Limitations
- Trials per setup are few (10 per configuration), so success-rate estimates are noisy.
- UniDexGrasp++ was evaluated only in simulation with a state-based model, not on the real-world setup.
- Experiments are limited to a fixed four-camera tabletop setup with pre-calibrated sensors.
Experimental setup
Datasets: YCB
Benchmarks: real-world dexterous grasping trials (Shadow Hand on UR10e) UniDexGrasp++ state-based simulation trials
Metrics: success rate
Baselines: DFF UniDexGrasp++
Builds on: Distilled Feature Fields DINO SAM UniDexGrasp++
Closest papers by meaning
- Scaffolding Dexterous Manipulation with Vision-Language Models (NeurIPS 2025 · similarity 0.89)
- Localizing, Structuring, and Rendering: Bridging 3D and 2D Vision-Language-Action Models for Robotic Manipulation (CVPR 2026 · similarity 0.88)
- VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation (CVPR 2025 · similarity 0.87)
- CADGrasp: Learning Contact and Collision Aware General Dexterous Grasping in Cluttered Scenes (NeurIPS 2025 · similarity 0.87)
- OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints (CVPR 2025 · similarity 0.86)
- DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.