Research map / Multi-agent systems and games / Human-AI and goal-conditioned
LogiCity: Advancing Neuro-Symbolic AI with Abstract Urban Simulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces LogiCity, a customizable first-order-logic-based urban simulator and benchmark with two tasks (long-horizon Safe Path Following and one-step Visual Action Prediction) for evaluating neuro-symbolic AI.
Problem
Existing neuro-symbolic AI benchmarks lack long-horizon, multi-agent reasoning tasks, and rely on fixed, simplistic logical rules over few entities, far from real-world complexity.
Approach
A simulator where concepts (predicates) and FOL rules are abstractions that can be grounded to arbitrary agent compositions and whose complexity is user-configurable; agents follow rules via an SMT solver, and scenes can be rendered to RGB using GPT-4 and Stable Diffusion XL icons.
Key results
- In SPF easy mode, behavior cloning NLM achieves 0.75 TSR and Popper/MaxSynth reach 1.00 TSR, while the best RL baseline NLM-DQN reaches 0.53 TSR.
- In the VAP hard mode, NLM achieves 0.36 weighted accuracy under random agents, beating GNN's 0.28 in the modular setting.
- GPT-4o reaches 59.0% overall accuracy on the VAP hard-mode subset versus 81.2% for human participants and 25.3% for random chance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use random rather than fixed training agent compositions (ablation) | VAP modular GNN and NLM · VAP Easy and Hard | wAcc.: GNN: 0.53 -> 0.54 (Easy), 0.23 -> 0.28 (Hard); NLM: 0.49 -> 0.56 (Easy), 0.24 -> 0.36 (Hard) | Helped |
| Use random rather than fixed training agent compositions (ablation) | VAP E2E GNN and NLM · VAP Easy and Hard | wAcc.: GNN: 0.78 -> 0.82 (Easy), 0.40 -> 0.54 (Hard); NLM: 0.71 -> 0.79 (Easy), 0.46 -> 0.50 (Hard) | Helped |
| Use modular supervision instead of end-to-end supervision (ablation) | VAP GNN and NLM · VAP Easy and Hard, fixed agent setting | wAcc.: GNN: 0.78 -> 0.53 (Easy), 0.40 -> 0.23 (Hard); NLM: 0.71 -> 0.49 (Easy), 0.46 -> 0.24 (Hard) | Hurt |
| Replace abstract predicate groundings with exact semantic point-cloud observations (ablation) | DQN · SPF Easy and Hard | TSR: 0.35 -> 0.12 (Easy); 0.09 -> 0.01 (Hard) | Hurt |
| Replace abstract predicate groundings with exact semantic point-cloud observations (ablation) | MB-Shooting · SPF Easy and Hard | TSR: 0.24 -> 0.23 (Easy); 0.16 -> 0.02 (Hard) | Hurt |
| Use GPT-4o rather than GPT-4 or GPT-3.5 | LLM in-context reasoning · VAP Hard Mini test | Overall accuracy: GPT-4o: 59.0; GPT-4: 39.6; GPT-3.5: 33.0 | Helped |
| Increase model scale from GPT-4o mini to GPT-4o | LLM in-context reasoning · VAP Hard Mini test | Overall accuracy: 29.6 -> 59.0 | Helped |
Surprising findings
- Pure-neural RL baselines (e.g., DQN, PPO, A2C) perform worse than NLM-DQN, a simple NeSy agent, in the SPF task.
- GPT-4o mini scores only 29.6% overall, far below GPT-4o, indicating a strong model-scale dependence for logical reasoning.
Limitations
- Users must pre-define conflict-free rule sets that avoid deadlocks.
- The simulation is deterministic and does not support temporal logic.
- Perception-based VAP models show low concept recognition accuracy, making perceptual noise accumulate.
Experimental setup
Datasets: LogiCity SPF episodes LogiCity VAP rendered images
Benchmarks: LogiCity SPF (easy, medium, hard, expert) LogiCity VAP (easy, hard)
Metrics: trajectory success rate decision success rate score averaged accuracy weighted accuracy recall
Baselines: Popper MaxSynth HRI NLM GNN MLP DQN PPO A2C DreamerV2 MB-shooting NLM-DQN
Builds on: Neural Logic Machines Popper DQN ProLog
Closest papers by meaning
- NeSyPr: Neurosymbolic Proceduralization For Efficient Embodied Reasoning (NeurIPS 2025 · similarity 0.86)
- From Statistical Relational to Neuro-Symbolic Artificial Intelligence (AAAI 2024 · similarity 0.86)
- NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domains (ICLR 2025 · similarity 0.85)
- Right for the Right Reasons: Avoiding Reasoning Shortcuts via Prototypical Neurosymbolic AI (NeurIPS 2025 · similarity 0.85)
- NeSyCoCo: A Neuro-Symbolic Concept Composer for Compositional Generalization (AAAI 2025 · similarity 0.85)
- VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.