Research map / Large language models / Agents, code and math
DiSciPLE: Learning Interpretable Programs for Scientific Visual Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiSciPLE, an LLM-driven evolutionary search that synthesizes interpretable Python programs interleaving open-world segmentation primitives with regressors, evaluated on three new scientific visual discovery benchmarks.
Problem
Learning accurate and interpretable predictive programs for scientific visual tasks (population density, poverty, aboveground biomass) where LLMs lack prior knowledge of the relationships and black-box models offer no interpretation.
Approach
Uses an LLM as the crossover and mutation operator in an evolutionary search over Python programs built from vision foundation-model primitives, augmented with a stratified program critic (per land-use category evaluation), analytical program simplification (DAG pruning and low-weight feature removal), and a learned linear regressor over LLM-proposed features.
Key results
- On population density in-distribution, DiSciPLE achieves L2-Log 0.2607 vs 0.3974 for the large deep model (Deep - Large).
- Out-of-distribution AGB L1 of 31.10 for DiSciPLE vs 35.27 for the best deep baseline (Deep - Small).
- Ablation on population density: critic and simplification each reduce L2-log error (0.3159 base to 0.2607 full on test set).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use feature-set prediction instead of directly prompting for a predictor (ablation) | DiSciPLE · Population Density | L2 log: 0.3159 -> 0.2906 | Helped |
| Add the program critic (ablation) | DiSciPLE with feature-set prediction · Population Density | L2 log: 0.2906 -> 0.2873 | Helped |
| Add program simplification (ablation) | DiSciPLE with feature-set prediction and critic · Population Density | L2 log: 0.2873 -> 0.2607 | Helped |
| Remove common-sense understanding of primitives by renaming and not describing them (ablation) | DiSciPLE · Population Density | L1 log: 0.2607 -> 0.8401 | Hurt |
| Remove the objective prompt (ablation) | DiSciPLE · Population Density | L1 log: 0.2607 -> 0.4498 | Hurt |
| Use DiSciPLE rather than the strongest deep baseline | Population Density · Population Density in-distribution | L2-Log: 0.3974 -> 0.2607 | Helped |
| Use DiSciPLE rather than the strongest deep baseline | Population Density · Population Density OOD | L2-Log: 0.4460 -> 0.3807 | Helped |
| Use DiSciPLE rather than the strongest baseline | Poverty · Poverty in-distribution | L1: 1.170 -> 1.077 | Helped |
| Use DiSciPLE rather than the strongest baseline | Poverty · Poverty OOD | L1: 1.257 -> 1.134 | Helped |
| Use DiSciPLE rather than the strongest deep baseline | AGB · AGB in-distribution | L1: 21.15 -> 24.79 | Mixed |
| Use DiSciPLE rather than the strongest baseline | AGB · AGB OOD | L1: 35.27 -> 31.10 | Helped |
Surprising findings
- Removing the common-sense objective prompt and renaming primitives leads to worse-than-mean results (L1 log 0.8401), showing LLM prior knowledge is critical to search.
- Two PhD experts spending about 1.5 hours achieved worse AGB error (L1 34.48) than DiSciPLE (24.79).
Limitations
- Only the final computational layer's learnable parameters are differentiably optimized; end-to-end differentiability did not help much.
- Many pipeline operations have zero gradients over large input regions, complicating gradient-based optimization.
- Evaluated only on three geospatial problems in two domains.
Experimental setup
Datasets: ACS Community Surveys 5-year estimates Sentinel-2 images OpenStreetMap masks SustainBench poverty data NASA GEDI aboveground biomass
Benchmarks: DiSciPLE visual program discovery benchmark (population density, poverty, AGB)
Metrics: L2 log error L1 log error L1 RMSE
Baselines: Mean Concept Bottleneck ResNet Zero-shot LLM programs Random Search
Builds on: ViperGPT VisProg symbolic regression concept bottleneck
Closest papers by meaning
- Position: Data-driven Discovery with Large Generative Models (ICML 2024 · similarity 0.84)
- Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation (ICML 2025 · similarity 0.84)
- Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks (NeurIPS 2026 · similarity 0.84)
- EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Models (NeurIPS 2024 · similarity 0.83)
- CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning (ICLR 2026 · similarity 0.83)
- AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomy (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.