Research map / Probabilistic and scientific modeling / Variational and generative sampling
LINGO-Space: Language-Conditioned Incremental Grounding for Space
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LINGO-Space, a probabilistic, incremental space-grounding method that uses an LLM (ChatGPT) semantic parser and a graph-based network predicting mixtures of polar distributions to localize locations referred to by composite instructions.
Problem
Spatially localizing composite natural-language instructions that refer to space (space grounding), which suffers from positional, referential, representational and compositional ambiguity compared with instance grounding.
Approach
Models the target location as a mixture of instance-wise polar distributions (Gaussian distance, von Mises angle) over scene-graph nodes, and updates this distribution incrementally for each referring phrase produced by a ChatGPT-based semantic parser, using a GPS (MPNN + transformer) network.
Key results
- Achieves 80.0 success on the ParaGon benchmark versus 41.5 for ParaGon and 42.5 for SREM, and is best in 11 of 12 single-expression tasks.
- On composite multi-expression tasks, scores 90.5 on the ParaGon-style composite task versus 28.5 for ParaGon and 26.0 for CLIPort.
- Achieves 95.5 on far-seen-colors and 95.0 on far-unseen-colors where baselines drop to 31.5-60.0 on new close/far predicates.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LINGO-Space instead of the strongest baseline | Single-expression grounding · 12 single-referring-expression benchmark tasks | success score | Helped |
| Use LINGO-Space instead of the strongest baseline | Single-expression grounding · SREM benchmark, task 5 | success score: 99.5 vs 98.5 | Hurt |
| Use LINGO-Space instead of the strongest baseline | Single-expression grounding · SREM benchmark, task 8 | success score: 100 vs 100 | No clear effect |
| Use LINGO-Space for novel close/far predicates | Space grounding · LINGO-Space benchmark, close/far tasks | success score | Helped |
| Use LINGO-Space instead of baseline methods | Composite-instruction grounding · Four multiple-referring-expression benchmark tasks | success score | Mixed |
Surprising findings
- SREM fails to attain scores on the simple CLIPort-style task because of its dependence on a predefined instruction structure.
- ParaGon's performance degraded sharply when retrained on the composite task data only (e.g., 28.5 vs. 67.9 reported in its paper).
Limitations
- Does not account for volumes, causing placement failures (e.g., 1.0 lower in behind-seen-colors).
- Assumes an optimal scene-graph generator and single-relation edges from a predefined predicate set.
- Real-world evaluation is a single qualitative quadruped navigation demonstration.
Experimental setup
Datasets: CLIPort benchmark ParaGon benchmark SREM benchmark Composite task (new)
Benchmarks: CLIPort benchmark ParaGon benchmark SREM benchmark LINGO-Space close/far tasks composite task
Metrics: success score binary success score
Baselines: CLIPort ParaGon SREM
Builds on: CLIPort ParaGon SREM GPS graph transformer Grounding DINO
Closest papers by meaning
- RoboGround: Robotic Manipulation with Grounded Vision-Language Priors (CVPR 2025 · similarity 0.86)
- Grounding Language Plans in Demonstrations Through Counterfactual Perturbations (ICLR 2024 · similarity 0.84)
- Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis (NeurIPS 2025 · similarity 0.84)
- SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation (NeurIPS 2025 · similarity 0.84)
- Grounding Everything in Tokens for Multimodal Large Language Models (CVPR 2026 · similarity 0.84)
- RealVLG-R1: A Large-Scale Real-World Visual-Language Grounding Benchmark for Robotic Perception and Manipulation (CVPR 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.