Research map / Large language models / Instruction and fine-tuning

How Post-Training Shapes Biological Reasoning Models

NeurIPS 2026 · Large language models › Instruction and fine-tuning · approach: Fine-tuning for reasoning data · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Controlled study of post-training stages (CPT, SFT, RL) for biological reasoning models across DNA, RNA, and protein tasks, showing SFT raises ID but hurts OOD while RL from strong SFT checkpoints recovers OOD performance.

Problem

Post-training is used to build biological reasoning models that combine LLMs with biological foundation models, but how CPT, SFT, and RL shape in-domain versus out-of-domain generalization is poorly understood.

Approach

Holds backbones, tasks and data fixed while varying CPT, SFT epochs/data, RL epochs, backbone and LoRA rank, and evaluates ID and biologically defined OOD splits to map distinct generalization regimes per training stage.

Key results

• SFT on DNA raises in-domain accuracy from ~0.68 to ~0.90 over 16 epochs while OOD accuracy peaks at ~0.73 around 2-4 epochs and declines to ~0.68 by 32 epochs.
• RL from strong SFT checkpoints improves OOD accuracy by about 0.05 in DNA; for proteins, Q4-P RL gains roughly 0.08 absolute F-max OOD.
• CPT on biology text improves OOD for Q1-D by 0.2 after SFT and 0.08 after RL; best ID-OOD trade-off comes from brief SFT followed by larger RL allocation; higher LoRA rank helps SFT while RL peaks at rank 16.

Surprising findings

• OOD accuracy peaks early in SFT and then declines by about 18 points in RNA while ID keeps improving.
• RL initially drops performance for Qwen3-1.7B RNA models before recovering, unlike the larger backbones.

Limitations

• Conclusions rest on a limited set of tasks, benchmarks, and model families.
• Fixed-epoch SFT-RL allocation is not compute-normalized since RL epochs are costlier.
• Evaluation measures final-answer correctness, not validity of intermediate reasoning.

Experimental setup

Datasets: BioReason KEGG pathway benchmark MEDEA target identification benchmark (adapted) BioReason-Pro UniProt protein function dataset FineFineWeb biology subset

Benchmarks: KEGG pathway prediction (BioReason) MEDEA target identification (hepatoblastoma OOD) BioReason-Pro protein function temporal holdout (CAFA-style)

Metrics: accuracy F-max

Baselines: Qwen3-1.7B Qwen3-4B Gemma 4 E2B

Builds on: GRPO BioReason BioReason-Pro MEDEA Evo2 TranscriptFormer ESM-3

Closest papers by meaning

• On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.87)
• Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead (ICLR 2026 · similarity 0.87)
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.86)
• RAST: Reasoning Activation in LLMs via Small-model Transfer (NeurIPS 2025 · similarity 0.85)
• How does RL Post-training Induce Skill Composition? A Case Study on Countdown (NeurIPS 2026 · similarity 0.85)
• Contextualizing biological perturbation experiments through language (ICLR 2025 · similarity 0.85)