Research map / Large language models / Agents, code and math
Data-Efficient Learning with Neural Programs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ISED, a sampling-based algorithm that trains DNN components of neural programs (black-box Python or GPT-4 calls) end-to-end from input-output labels by estimating gradients over a sampled symbolic summary of the program.
Problem
Learning the parameters of a neural model whose outputs feed a black-box program (Python code or LLM API call) when only end-to-end input-output labels are available, without differentiable logic programs.
Approach
ISED samples symbols from the DNN's output distributions, executes the black-box program on each sample, and builds a partial symbolic summary (symbol-output pairs) that is differentiated via semiring aggregation (min-max or add-mult) and a loss over the output distribution, avoiding a learned surrogate and yielding stronger signals than REINFORCE.
Key results
- ISED is top performer on 8 of 16 tasks and outperforms neurosymbolic baselines on 9 of 14 logic-programmable tasks.
- On GPT leaf and scene tasks ISED beats the purely neural baseline by 3.82% and 31.42% and zero-shot CLIP by 59.80% and 17.50%.
- On sum8 with 80 samples ISED reaches 87.28% vs IndeCateR 5.36% and REINFORCE 8.32%; on sum16 IndeCateR+ beats ISED (83.52% vs 8.13%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ISED instead of A-NeSI | MNIST-R sum_3 · sum_3 | accuracy: 94.39 ± 0.77 -> 95.10 ± 0.95 | Helped |
| Use ISED instead of IndeCateR | MNIST-R sum_4 · sum_4 | accuracy: 92.58 ± 0.80 -> 94.1 ± 1.6 | Helped |
| Use ISED instead of Scallop | HWF · HWF | accuracy: 96.65 ± 0.13 -> 97.34 ± 0.26 | Helped |
| Use ISED instead of Scallop | DT leaf · DT leaf | accuracy: 81.13 ± 3.50 -> 82.32 ± 4.15 | Helped |
| Use ISED instead of A-NeSI | GPT leaf · GPT leaf | accuracy: 72.40 ± 12.24 -> 79.95 ± 5.71 | Helped |
| Use ISED instead of A-NeSI | scene recognition · scene | accuracy: 61.46 ± 14.18 -> 68.59 ± 1.95 | Helped |
| Use ISED instead of NASR | Visual Sudoku · sudoku | accuracy: 82.78 ± 1.06 -> 80.32 ± 1.79 | Hurt |
| Use ISED instead of IndeCateR+ at k=80 | sum_8 · sum_8 | accuracy: 10.20 ± 1.12 -> 87.28 ± 0.76 | Helped |
| Use ISED instead of IndeCateR+ at k=120 | sum_12 · sum_12 | accuracy: 6.84 ± 2.06 -> 85.72 ± 2.15 | Helped |
| Use IndeCateR+ instead of ISED at k=1600 | sum_16 · sum_16 | accuracy: 83.52 ± 1.75 -> 8.13 ± 1.10 | Hurt |
| Increase ISED sample count from k=80 to k=800 | sum_8 · sum_8 | accuracy: 87.28 ± 0.76 -> 87.72 ± 0.86 | No clear effect |
| Increase ISED sample count from k=120 to k=1200 | sum_12 · sum_12 | accuracy: 85.72 ± 2.15 -> 86.72 ± 0.48 | No clear effect |
Surprising findings
- ISED fails on sum16 (8.13% accuracy), where the sampling-based approach struggles with high-dimensional program inputs.
- Treating programs as black boxes sometimes matches or beats specialized logic-program neurosymbolic frameworks such as Scallop on HWF.
Limitations
- Scales poorly with the dimensionality of program inputs (e.g., sum16 accuracy near chance).
- Sampling strategy is simple categorical sampling; improvements from Bayesian optimization-style sampling are left to future work.
Experimental setup
Datasets: MNIST-R MNIST HWF LEAF-ID Scene recognition dataset SatNet
Benchmarks: MNIST-R HWF LEAF-ID DT leaf LEAF-ID GPT leaf scene recognition Visual Sudoku
Metrics: accuracy standard deviation training time to accuracy sample efficiency
Baselines: DeepProbLog Scallop A-NeSI REINFORCE IndeCateR NASR IndeCateR+ CLIP NGS
Builds on: REINFORCE IndeCateR NASR A-NeSI DeepProbLog Scallop
Closest papers by meaning
- From Perception to Programs: Regularize, Overparameterize, and Amortize (ICML 2023 · similarity 0.87)
- Programming with a Differentiable Forth Interpreter (ICML 2017 · similarity 0.86)
- Neural network gradient-based learning of black-box function interfaces (ICLR 2019 · similarity 0.86)
- CTSketch: Compositional Tensor Sketching for Scalable Neurosymbolic Learning (NeurIPS 2025 · similarity 0.86)
- On the Hardness of Probabilistic Neurosymbolic Learning (ICML 2024 · similarity 0.85)
- DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.