Research map / Vision-language and multimodal / Multimodal LLMs

Probing and Bridging Geometry-Interaction Cues for Affordance Reasoning in Vision Foundation Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Probes vision foundation models for geometric and interaction cues underlying affordance, then fuses DINOv3 part-level PCA prototypes with Flux Kontext verb cross-attention in a training-free, zero-shot pipeline for affordance estimation.

Problem

What internal capabilities enable visual affordance understanding, and how are geometric part structure and verb-conditioned interaction priors manifested in vision foundation models?

Approach

Decomposes affordance into geometric perception (part-level structure from discriminative VFMs like DINO) and interaction perception (verb-conditioned cross-attention in generative models like Flux), and composes them training-free by aligning PCA geometric bases with verb attention via NSS.

Key results

• On AGD20K, Interaction x Geometry fusion reaches KLD 1.493, SIM 0.326, NSS 1.090, versus Only Interaction at KLD 1.825, SIM 0.271, NSS 1.050.
• Fused training-free method is described as approaching weakly-supervised performance; LOCATE reports KLD 1.405, SIM 0.372, NSS 1.157.
• Linear probing on UMD shows models with stronger Probing3D geometric rankings achieve higher mIoU; adding Metric3Dv2 depth and normals improves CLIP and SigLIP most.

Surprising findings

• DINOv2 gains little from explicit depth and normal cues, while CLIP and SigLIP gain notably.
• DINO's geometric part response is attenuated in full object contexts for rings not semantically part of a cup, indicating semantic assimilation of geometry.

Limitations

• Performance constrained by noisiness of noun-conditioned attention maps and instability of generative outputs.
• Simple training-free fusion is a shallow, open-loop signal combination.
• Fused method is described as approaching, not surpassing, weakly-supervised baselines on AGD20K; UMD evaluated only qualitatively for the fusion.

Experimental setup

Datasets: UMD AGD20K

Benchmarks: UMD AGD20K

Metrics: mIoU KLD SIM NSS

Baselines: AffordanceLLM Cross-View-AG LOCATE Affordance-R1 GroundingDINO MaskCLIP EVF-SAM

Builds on: Probing3D Flux Kontext

Closest papers by meaning

• GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding (CVPR 2025 · similarity 0.90)
• SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model (CVPR 2025 · similarity 0.88)
• AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Models (NeurIPS 2025 · similarity 0.88)
• Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactions (CVPR 2025 · similarity 0.87)
• BiAssemble: Learning Collaborative Affordance for Bimanual Geometric Assembly (ICML 2025 · similarity 0.87)
• IAAO: Interactive Affordance Learning for Articulated Objects in 3D Environments (CVPR 2025 · similarity 0.87)