Research map / Vision-language and multimodal / Human and embodied modeling
Talking Points: Describing and Localizing Pixels
Matan Rusanovsky, Shimon Malnick, Shai Avidan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Gaussian attention masks to the Point Descriptor (ablation) | Point Descriptor · LlamaPointInPart test | mPCK: 23.63 -> 78.13 | Improved |
| add LoRA adaptation of the language model in the Point Localizer (ablation) | Point Localizer · LlamaPointInPart test | mPCK: 47.60 -> 78.83 | Improved |
| add localization-based GRPO fine-tuning (train on Bovidae, test on Canidae) | TP zero-shot · AP-10K Canidae | mPCK: 29.85 -> 29.96 | No clear effect |
| add localization-based GRPO fine-tuning (train on Canidae, test on Bovidae) | TP zero-shot · AP-10K Bovidae | mPCK: 28.56 -> 30.36 | Improved |
| replace OMG-LLaVA descriptions with TP generated descriptions | Point Localizer · LlamaPointInPart test | mPCK: 31.03 -> 78.13 | Improved |
| replace DAM descriptions with TP generated descriptions | Point Localizer · LlamaPointInPart test | mPCK: 42.87 -> 78.13 | Improved |
| use TP generated descriptions instead of ground-truth descriptions | Point Localizer · LlamaPointInPart test | mPCK: 78.83 -> 78.13 | No clear effect |
| use TP descriptions instead of ChatGPT-5 descriptions | Point Localizer · LlamaPointInPart 100-sample subset | mPCK (%): 62% -> 78% | Improved |
| use TP descriptions instead of human descriptions | Point Localizer · LlamaPointInPart 100-sample subset | mPCK (%): 56% -> 78% | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- OMG-LLaVA — OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding (NeurIPS 2024) · better on 3 of 3 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding (CVPR 2025 · similarity 0.90)
- Pixel-Aligned Language Model (CVPR 2024 · similarity 0.89)
- Grounding Everything: Emerging Localization Properties in Vision-Language Transformers (CVPR 2024 · similarity 0.88)
- Improved Visual Grounding through Self-Consistent Explanations (CVPR 2024 · similarity 0.88)
- Referencing Where to Focus: Improving Visual Grounding with Referential Query (NeurIPS 2024 · similarity 0.88)
- Mechanisms of Object Localization in Vision-Language Models (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
