Research map / Large language models / Agents, code and math

Position: The Road to Generalizable Neuro-Symbolic Learning Should be Paved with Foundation Models

NeurIPS 2026 · Large language models › Agents, code and math · approach: Inductive and abstract reasoning evaluation

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper arguing that neuro-symbolic learning should replace task-specific neural training with prompted foundation models (neuro-symbolic prompting), supported by experiments on five NeSy benchmarks against Scallop and ISED.

Problem

Traditional neuro-symbolic learning trains neural perception models jointly with symbolic programs, which is compute-hungry, overfits to datasets, and suffers from symbol hallucination; it is unclear what role training plays now that foundation models can be prompted for perception.

Approach

Replace the trained neural perception component of neuro-symbolic systems with prompted foundation models that extract symbols, while a symbolic program performs reasoning; identify compute, data, and program pitfalls of training-based NeSy and argue prompting avoids them.

Key results

• Largest prompted foundation models (e.g., Qwen2.5-VL-72B, InternVL2.5-78B, Gemini-2.0-Flash) match or exceed Scallop on CLUTRR (0.76-0.79 vs 0.400) and CLEVR (up to 0.90 vs 0.750), while lagging on Sum5 (0.815-0.860 vs 0.975) and Leaf (0.405-0.509 vs 0.811).
• Adding 3% Gaussian noise drops Scallop accuracy by 0.19-0.21 on HWF5 and CLEVR-like image tasks versus 0.06-0.07 for prompting on the same datasets.
• In human evaluation on Leaf, most annotators disagreed with Scallop's margin (58.8%) and shape (60.0%) predictions even though they produced correct final answers, with low agreement for texture (kappa 0.05).

Surprising findings

• Scallop's neural perception often reaches correct final answers via hallucinated symbols that humans judge incorrect (symbol hallucination).
• Trained NeSy models appear to memorize dataset noise and biases, whereas prompted foundation models generalize better under slight distribution shift.

Limitations

• Symbols and programs are human-specified; the method does not infer them automatically.
• Prompting results use greedy decoding with single runs and no error bars; GPT-4o evaluated on only two datasets to limit cost.
• Human evaluation covers only 45 questions from 10 annotators on the Leaf dataset.

Experimental setup

Datasets: Sum5 HWF5 CLUTRR CLEVR Leaf

Benchmarks: Sum5 HWF5 CLUTRR CLEVR Leaf

Metrics: accuracy accuracy difference accuracy drop under 3% Gaussian noise symbol ground-truth match inter-annotator agreement (Cohen's kappa)

Baselines: Scallop ISED Chain-of-Thought prompting

Builds on: Scallop ISED Chain-of-Thought prompting SatLM

Closest papers by meaning

• Deciphering Raw Data in Neuro-Symbolic Learning with Provable Guarantees (AAAI 2024 · similarity 0.90)
• Neural-Symbolic Collaborative Distillation: Advancing Small Language Models for Complex Reasoning Tasks (AAAI 2025 · similarity 0.88)
• Right for the Right Reasons: Avoiding Reasoning Shortcuts via Prototypical Neurosymbolic AI (NeurIPS 2025 · similarity 0.88)
• A learnability analysis on neuro-symbolic learning (NeurIPS 2025 · similarity 0.88)
• Neuro-Symbolic Data Generation for Math Reasoning (NeurIPS 2024 · similarity 0.86)
• DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning (ICML 2025 · similarity 0.86)