Research map / Large language models / Agents, code and math
SEDRAS: Symbolically Evaluated Deep Research And Science
Fredrik Carlsson, Daniel Ward, Joseph Ortiz, Fangyu Liu, Joakim Nivre
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| let model choose data points actively (post-static) instead of receiving samples statically (ablation) | all top LLMs · SEDRAS-2026 raw representation | test accuracy: static up to 0.75 (GPT) -> post-static all below 0.6, often below 0.5 | Hurt |
| increase number of variables embedded in textual query from 4 to 7 (ablation) | Claude Opus 4.5 query conversion · SEDRAS-2026 textual experiments | conversion accuracy (%): 98.1 -> 69.2 (28.9% drop) | Hurt |
| increase number of variables embedded in textual query from 4 to 7 (ablation) | Gemini 3 query conversion · SEDRAS-2026 textual experiments | conversion accuracy (%): 95.8 -> 89.9 | Hurt |
| raw symbolic representation vs textual representations (ablation) | GPT 5.2 · SEDRAS-2026 | test accuracy | Mixed |
| scale UDD complexity to higher levels (ablation) | top LLMs · SEDRAS-2026 | test accuracy | Mixed |
| render textual samples as files (PDF, XLSX) (ablation) | Grok 4.1 · SEDRAS-2026 | test accuracy | Hurt |
Closest papers by meaning
- CURIE: Evaluating LLMs on Multitask Scientific Long-Context Understanding and Reasoning (ICLR 2025 · similarity 0.87)
- SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models (NeurIPS 2024 · similarity 0.87)
- Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning (NeurIPS 2025 · similarity 0.87)
- SciCode: A Research Coding Benchmark Curated by Scientists (NeurIPS 2024 · similarity 0.87)
- SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Research (AAAI 2024 · similarity 0.86)
- MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
