Research map / Detection and segmentation / Open-set and few-shot detection
RAGG: Retrieval-Augmented Grasp Generation Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RAGG, a two-stage intent-based 3D grasp generation model that retrieves a similar interaction instance from a knowledge base to guide a diffusion generator (ReDim), then refines the grasp with a KAN-based Progressive Refinement Network using an SSIM loss.
Problem
Intent-based grasp generation suffers from manipulation ambiguity (one intent yields many poses) and modality gaps between text, point cloud, and mesh, and existing methods rely heavily on textual prompts that require relabeling for unseen objects.
Approach
Retrieve the most relevant hand-object interaction instance using joint object-shape (BPS) and intent similarity, use it as a reference inside a diffusion denoiser via a Semantics Calibration Transformer, then refine with a residual KAN network trained with an SSIM loss on hand-object distance structure.
Key results
- On OakInk, average penetration depth 0.506 cm vs 0.5133 cm for GrabNet (hold) and 0.410 vs 0.541 cm (use); intersection volume 3.667 vs 5.165 cm3 (hold).
- Center distance (controllability) averages 61.395 for RAGG vs 33.094 for GrabNet on OakInk.
- Ablation shows PRN reduces penetration, intersection and displacement relative to the original GrabNet refinement and also improves GrabNet when plugged in.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace GrabNet with RAGG | OakInk Hold (average) · OakInk | penetration depth (cm): 0.5133 ± 0.584 -> 0.506 ± 0.497 | Helped |
| replace GrabNet with RAGG | OakInk Hold (average) · OakInk | intersection volume (cm³): 5.165 ± 14.011 -> 3.667 ± 7.080 | Helped |
| replace GrabNet with RAGG | OakInk Hold (average) · OakInk | simulation displacement (cm): 1.964 ± 2.593 -> 1.274 ± 1.318 | Helped |
| replace GrabNet with RAGG | OakInk Hold (average) · OakInk | center distance: 33.094 -> 61.395 | Helped |
| replace GrabNet with RAGG | OakInk Use (average) · OakInk | penetration depth (cm): 0.541 ± 0.595 -> 0.410 ± 0.478 | Helped |
| replace GrabNet with RAGG | OakInk Use (average) · OakInk | intersection volume (cm³): 4.867 ± 12.049 -> 2.973 ± 7.839 | Helped |
| replace GrabNet with RAGG | OakInk Use (average) · OakInk | simulation displacement (cm): 1.604 ± 1.775 -> 1.598 ± 1.898 | No clear effect |
| replace GrabNet with RAGG | OakInk Use (average) · OakInk | center distance: 33.094 -> 61.395 | Helped |
| replace GrabNet with RAGG | OakInk unseen objects · OakInk (Bowl) | intersection volume: 6.102 -> 4.958 | Helped |
| replace GrabNet with RAGG | GRAB out-of-domain objects · GRAB (Wineglass) | intersection volume: 1.213 -> 0.159 | Helped |
| add the diffusion generation model instead of the generation baseline (ablation) | RAGG ablation · OakInk | penetration depth: 0.672 -> 0.554 | Helped |
| add Joint Retrieval (ablation) | RAGG generation model · OakInk | center distance: 57.767 -> 61.508 | Helped |
Surprising findings
- The original GrabNet refinement reduces penetration but increases simulation displacement because the hand releases the object.
Limitations
- Retrieval uses only the top-1 reference (v set to 1), so guidance depends on a single retrieved instance.
- Some metrics are mixed across categories (e.g., several objects where RAGG is worse on displacement or penetration), and perceptual evaluation uses only five volunteers.
Experimental setup
Datasets: OakInk GRAB
Benchmarks: OakInk GRAB
Metrics: penetration depth solid intersection volume simulation displacement center distance (controllability) human perceptual success rate
Baselines: GrabNet OakInk intent-based GrabNet variant
Builds on: GrabNet DDPM MDM Kolmogorov-Arnold Network SSIM
Closest papers by meaning
- Contact Map Transfer with Conditional Diffusion Model for Generalizable Dexterous Grasp Generation (NeurIPS 2025 · similarity 0.89)
- LatentHOI: On the Generalizable Hand Object Motion Generation with Latent Hand Diffusion. (CVPR 2025 · similarity 0.88)
- G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis (CVPR 2024 · similarity 0.87)
- DiffGrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model (AAAI 2025 · similarity 0.86)
- Grasp as You Say: Language-guided Dexterous Grasp Generation (NeurIPS 2024 · similarity 0.86)
- Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.