Research map / Large language models / Agents, code and math
HypoSpace: A Diagnostic Benchmark for Set-Valued Hypothesis Generation under Underdetermination and Sublinear Coverage Bounds
Tingting Chen, Beibei Lin, Zifeng Yuan, Qiran Zou, Hongyu He, Anirudh Goyal, Yew-Soon Ong, Dianbo Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase admissible set size from level 1 to level 3 | frontier LLMs · DNA Interaction (GPT-5) | RR: 100.00% -> 48.00% | Hurt |
| increase admissible set to |H_O|=160 | GPT-5 · Causal Inference | NR/RR: 100.0% -> 71.2% (RR) | Hurt |
| increase admissible set to |H_O|=125 | Grok-4 · 3D voxel | RR: 79.2% (VR 100.0%) | Hurt |
| use reasoning models rather than non-reasoning models | LLMs · Causal level 3 | RR: 99.20% vs 6.70% (GPT-5 vs GPT-4o) | Improved |
| add complexity-stratified decoding | Grok-4 · Boolean, complex hypotheses | RR: 0% -> 17.2% | Improved |
| add complexity-stratified decoding | DeepSeek-R1 · Boolean overall RR | RR: 25.6% -> 6.4% | Hurt |
| add complexity-stratified decoding | GPT-5 · Boolean simple hypotheses | RR: -14.3 points (simple) | Hurt |
| add complexity-stratified decoding | 7 LLMs · Boolean genetic interactions | overall RR: improves 4/7 models | Mixed |
| vary temperature and top-p (ablation) | GPT-4o, Claude-Opus-4, Gemini-2.5-Pro · Boolean (Hard) | RR | No clear effect |
| increase sampling budget to 3.0x|H_O| (ablation) | Gemini-2.5-Pro · Boolean (Hard) | RR: -9.9 | Hurt |
| increase sampling budget to 3.0x|H_O| (ablation) | Claude-Opus-4 · Boolean (Hard) | RR: +1.8 | No clear effect |
Closest papers by meaning
- MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search (NeurIPS 2025 · similarity 0.87)
- SciReason: A Controllable Benchmark for Scientific Reasoning in LLMs (NeurIPS 2026 · similarity 0.86)
- ATTS: Asynchronous Test-Time Scaling via Conformal Prediction (ICLR 2026 · similarity 0.86)
- Finite Resources False Discovery Rate Control on Structured Hypothesis Spaces (NeurIPS 2026 · similarity 0.86)
- The Silent Hyperparameter: Quantifying the Impact of Inference Backends on LLM Reproducibility (NeurIPS 2026 · similarity 0.85)
- Measuring Scientific Capabilities of Language Models with a Systems Biology Dry Lab (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
