Research map / Large language models / Agents, code and math
Right for the Right Reasons: Avoiding Reasoning Shortcuts via Prototypical Neurosymbolic AI
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Prototypical Neurosymbolic (PNet) models that anchor concept embeddings to prototypes from a few labelled examples, so that NeSy losses update embeddings relative to class centroids, mitigating reasoning shortcuts under very scarce supervision.
Problem
Neurosymbolic models trained with only final-label supervision learn spurious concepts (reasoning shortcuts) that satisfy background knowledge, and existing mitigations are costly (dense concept labels) or fail on harder benchmarks like rsbench.
Approach
Combine prototypical learning with NeSy losses: each concept gets a centroid computed from a handful of labelled support examples (unlabelled concepts initialised near the labelled centroids via a chi-square-scaled Gaussian), and the embedding update is driven jointly by the constraint loss and distance to prototypes, which theoretically reduces deterministic reasoning shortcuts.
Key results
- On MNIST-EvenOdd with one labelled image per concept, SL+PNet reaches Acc(C) 0.98 and Cls(C) 0.00 versus 0.08 and 0.68 for SL alone.
- On Kand-Logic, DPL+PNet reaches F1(C) 0.94 versus 0.25 for DPL.
- On BDD-OIA, DPL+PNet raises F1(C) to 0.16 from 0.08 and reduces Cls(C) to 0.35 from 0.82.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add prototype extractors to LTN | LTN on MNIST-EvenOdd · MNIST-EvenOdd | F1(C): 0.05 ± 0.04 -> 0.95 ± 0.03 | Helped |
| Add prototype extractors to SL | SL on MNIST-EvenOdd · MNIST-EvenOdd | F1(C): 0.04 ± 0.04 -> 0.98 ± 0.01 | Helped |
| Add prototype extractors to DPL | DPL on MNIST-EvenOdd · MNIST-EvenOdd | F1(C): 0.04 ± 0.05 -> 0.96 ± 0.04 | Helped |
| Add prototype extractors instead of augmented-support pretraining | NeSy models on Kand-Logic · Kand-Logic | F1(C): LTN 0.90 ± 0.01 -> 0.90 ± 0.01; SL 0.90 ± 0.01 -> 0.92 ± 0.02; DPL 0.91 ± 0.04 -> 0.94 ± 0.00 | No clear effect |
| Add prototype extractors to DPL | DPL on BDD-OIA · BDD-OIA | F1(C): 0.08 ± 0.13 -> 0.16 ± 0.15 | Helped |
| Add prototype extractors to DPL | DPL on BDD-OIA · BDD-OIA | Cls(C): 0.82 ± 0.06 -> 0.35 ± 0.11 | Helped |
| Pretrain the backbone on augmented annotated support data | DPL on MNIST-EvenOdd · MNIST-EvenOdd | F1(C): 0.04 ± 0.05 -> 0.55 ± 0.28 | Helped |
| Use one or two concepts with no labelled examples (ablation) | Prototypical NeSy models on Kand-Logic · Kand-Logic | F1(C): DPL 0.94 ± 0.00 -> 0.65 ± 0.03 -> 0.42 ± 0.02; SL 0.92 ± 0.02 -> 0.73 ± 0.03 -> 0.55 ± 0.04; LTN 0.90 ± 0.01 -> 0.64 ± 0 | Hurt |
| Add reconstruction loss alone (ablation) | NeSy models on MNIST-EvenOdd · MNIST-EvenOdd | F1(C): DPL 0.07 ± 0.00; SL 0.01 ± 0.00; LTN 0.05 ± 0.02 | Mixed |
| Add Shannon-entropy regularization alone (ablation) | NeSy models on MNIST-EvenOdd · MNIST-EvenOdd | F1(C): DPL 0.01 ± 0.00; SL 0.01 ± 0.00; LTN 0.55 ± 0.14 | Mixed |
| Add partial concept supervision alone (ablation) | NeSy models on MNIST-EvenOdd · MNIST-EvenOdd | F1(C): DPL 0.01 ± 0.00; SL 0.22 ± 0.01; LTN 0.12 ± 0.00 | Mixed |
| Combine reconstruction loss, entropy regularization, and concept supervision (ablation) | SL on MNIST-EvenOdd · MNIST-EvenOdd | F1(C): 0.35 ± 0.10 | Hurt |
Surprising findings
- Data augmentation pretraining (Pre*) with over 150 extra labelled datapoints improves concept metrics only modestly, far below prototypical models on MNIST-EvenOdd.
- Partial concept supervision and entropy/reconstruction mitigations, alone or combined, largely fail on MNIST-EvenOdd, with F1(C) near 0.01 to 0.10 for several combinations.
Limitations
- Performance degrades as the number of totally unlabelled classes grows (Kand-Logic F1(C) drops from 0.94 to 0.42-0.65 with unspecified concepts for DPL).
- BDD-OIA concept F1 remains low (0.16) even with PNet, indicating the problem is far from solved on real-world data.
- Theoretical analysis relies on idealized assumptions (invertible smooth generative map, embedding separability/clusterability).
Experimental setup
Datasets: MNIST-EvenOdd Kand-Logic BDD-OIA
Benchmarks: MNIST-EvenOdd Kand-Logic BDD-OIA
Metrics: Acc(C) F1(C) Acc(Y) F1(Y) Cls(C)
Baselines: DeepProbLog Logic Tensor Networks Semantic Loss CCN+ Shannon-entropy regularization reconstruction loss partial concept supervision data-augmentation pretraining (Pre*)
Builds on: prototypical networks DeepProbLog semantic loss Logic Tensor Networks rsbench
Closest papers by meaning
- Position: The Road to Generalizable Neuro-Symbolic Learning Should be Paved with Foundation Models (NeurIPS 2026 · similarity 0.88)
- A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts (NeurIPS 2024 · similarity 0.88)
- Analysis for Abductive Learning and Neural-Symbolic Reasoning Shortcuts (ICML 2024 · similarity 0.88)
- DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning (ICML 2025 · similarity 0.88)
- Shortcuts and Identifiability in Concept-based Models from a Neuro-Symbolic Lens (NeurIPS 2025 · similarity 0.88)
- Deciphering Raw Data in Neuro-Symbolic Learning with Provable Guarantees (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.