Research map / Detection and segmentation / Weakly supervised detection and segmentation
Reasoning Mamba: Hypergraph-Guided Region Relation Calculating for Weakly Supervised Affordance Grounding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes R-Mamba, which builds hypergraphs over DINO-ViT features of exocentric and egocentric images and uses a hypergraph-guided state space (SS2D/Mamba) block with hypergraph evolution to localize affordance regions under image-level weak supervision.
Problem
Weakly supervised affordance grounding (WSAG) must localize affordance regions on egocentric images using only image-level affordance labels, but existing CAM-based methods treat regions as isolated and ignore relationships among object parts.
Approach
Model many-to-many region relations with hypergraphs (vertices = feature points, hyperedges = clusters/distance neighborhoods), evolve the hypergraph using egocentric guidance to keep affordance-relevant parts, and reason over vertices and hyperedges globally with bidirectional SS2D scanning in a Mamba-based HSS block.
Key results
- On AGD20K-Seen, R-Mamba reaches KLD 1.173, SIM 0.414, NSS 1.247, improving over LOCATE (1.226, 0.401, 1.177).
- On AGD20K-Unseen, R-Mamba reports KLD 1.372, SIM 0.380, NSS 1.190, versus LOCATE's 1.405, 0.372, 1.157.
- Integrating the method into WSMA improves HICO-IIF KLD from 1.465 to 1.422 and NSS from 1.012 to 1.030.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| integrate R-Mamba with WSMA | WSMA · AGD20K-Seen | KLD: 1.176 -> 1.143 | Helped |
| integrate R-Mamba with WSMA | WSMA · AGD20K-Unseen | KLD: 1.335 -> 1.310 | Helped |
| integrate R-Mamba with WSMA | WSMA · HICO-IIF | KLD: 1.465 -> 1.422 | Helped |
| add geometric concentration loss Lgc (ablation) | Base · AGD20K-Seen | KLD: 1.732 -> 1.487 | Helped |
| add geometric concentration loss Lgc (ablation) | Base · AGD20K-Unseen | KLD: 1.971 -> 1.732 | Helped |
| add cosine similarity loss Lsim (ablation) | Base · AGD20K-Seen | KLD: 1.732 -> 1.210 | Helped |
| add cosine similarity loss Lsim (ablation) | Base · AGD20K-Unseen | KLD: 1.971 -> 1.408 | Helped |
| add Lgc and Lsim (ablation) | Base · AGD20K-Seen | KLD: 1.732 -> 1.173 | Helped |
| add Lgc and Lsim (ablation) | Base · AGD20K-Unseen | KLD: 1.971 -> 1.372 | Helped |
| add hypergraph modeling (ablation) | R-Mamba · AGD20K-Seen | KLD: 1.453 -> 1.173 | Helped |
| add hypergraph modeling (ablation) | R-Mamba · AGD20K-Unseen | KLD: 1.796 -> 1.372 | Helped |
| add HSS block (ablation) | R-Mamba · AGD20K-Seen | KLD: 1.294 -> 1.173 | Helped |
Surprising findings
- Applying the geometric concentration loss alone gives only marginal improvement; the cosine similarity loss between exo and ego branches provides the major gain.
Limitations
- Gap between predictions and ground truth remains substantial, as acknowledged by the authors.
- Hyperparameters (distance threshold, cluster number) are sensitive and tuned on the test metric.
Experimental setup
Datasets: AGD20K HICO-IIF ImageNet
Benchmarks: AGD20K-Seen AGD20K-Unseen HICO-IIF
Metrics: KLD SIM NSS
Baselines: EIL SPA TS-CAM Hotspots Cross-view-AG Cross-view-AG+ LOCATE WSMA
Builds on: Mamba HGNN SS2D (VMamba) DINO LOCATE
Closest papers by meaning
- Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priors (ICLR 2025 · similarity 0.91)
- Weakly Supervised Multimodal Affordance Grounding for Egocentric Images (AAAI 2024 · similarity 0.89)
- Learning 2D Invariant Affordance Knowledge for 3D Affordance Grounding (AAAI 2025 · similarity 0.89)
- Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactions (CVPR 2025 · similarity 0.87)
- GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding (CVPR 2025 · similarity 0.87)
- 3D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worlds (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.