Research map / Large language models / Reasoning and chain of thought

Helix: Evolutionary Reinforcement Learning for Open-Ended Scientific Problem Solving

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HELIX, combining GRPO-based RL on a DeepSeek-R1-Distill-Qwen model with NSGA-II selection over reward and embedding-based diversity and in-context lineage prompts, evaluated on 20 open-ended scientific tasks.

Problem

Complex scientific problems are domain-specific, open-ended and unbounded, and existing post-training or workflow-driven (evolutionary LLM) approaches explore inefficiently and generalize poorly.

Approach

Couple RL policy updates (GRPO) with an evolutionary candidate population selected by NSGA-II over reward and kNN embedding diversity, and feed lineage ancestors with rewards and feedback into the prompt so the policy learns from past trials.

Key results

• On circle packing (unit square), HELIX reaches a sum of radii of 2.63598308 with a 14B model, reported as a new record and above the best baseline (GPT-4o 2.611, SLSQP 2.519).
• HELIX achieves best results on 17 of 20 tasks versus all baselines, and beats OpenEvolve on 19 tasks per the paper.
• On Adult Income and Bank Marketing, HELIX attains F1 of 82.07 and 80.65, outperforming GPT-4o-based Open Evolve (72.27 and 78.54).

Surprising findings

• The base Qwen model scored 0.00 on Bank Marketing in direct prompting, showing failure of weaker base models without evolution.
• Inductor reward plateaus around 9.6 across model scales even though mean reward keeps rising with model size.

Limitations

• Results are reported as best-of-attempts outcomes, and evaluation uses fixed time limits per task.
• Gains over strong baselines are not uniform; some tasks (e.g., Demultiplexer, symbolic regression chemistry) are won by baselines.

Experimental setup

Datasets: Adult Income Bank Marketing Boston Housing Transparent Conductors LLM-SRBench

Benchmarks: Adult Income Bank Marketing Boston Housing Transparent Conductors Inductor Beam Bending Magnetic Torque Periodic Heat Demultiplexer Packing in Unit Square Packing in Unit Disk Eggholder

Metrics: macro F1 RMSE sum of radii normalized reward per physics task NMSE-based reward

Baselines: Direct Prompt OpenEvolve LightGBM RRL SLSQP Genetic Algorithm LLM-SR LaSR GPT-4o

Builds on: GRPO AlphaEvolve OpenEvolve NSGA-II VERL

Closest papers by meaning

• DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search (ICLR 2026 · similarity 0.87)
• ThetaEvolve: Test-time Learning on Open Problems (ICML 2026 · similarity 0.86)
• Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks (NeurIPS 2026 · similarity 0.86)
• CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning (ICLR 2026 · similarity 0.86)
• ReLaX: Reasoning with Latent Exploration for Large Reasoning Models (CVPR 2026 · similarity 0.86)
• Representation-Based Exploration for Language Models: From Test-Time to Post-Training (ICLR 2026 · similarity 0.86)