Research map / Large language models / AI and society
CycleResearcher: Improving Automated Research via Automated Review
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Post-trains open-source LLMs as a research agent (CycleResearcher) and a peer-review reward model (CycleReviewer) in an iterative SimPO research-review-refinement loop, trained on new Review-5k and Research-14k datasets.
Problem
Automating the full research cycle (literature review, manuscript writing, peer review, refinement) with open-source LLMs, which prior work mostly did with commercial models and prompting rather than policy optimization.
Approach
Treat paper generation as a policy optimization problem with a trained generative reviewer as reward model; iteratively sample papers, score them with CycleReviewer's simulated peer review, and train with a SimPO+NLL loss in rounds.
Key results
- CycleReviewer-123B achieves Proxy MAE 0.92 vs 1.16 for individual expert reviewers (about 26.89% reduction) and 74.24% decision accuracy.
- CycleResearcher-12B reaches an average simulated review score of 5.36 and 35.13% acceptance rate, versus 5.24 for preprints and 5.69 for accepted papers; AI Scientist scores 4.31.
- Human expert evaluation of 10 papers per system: CycleResearcher averages 4.8 overall vs 3.6 for AI Scientist, but below ICLR 2024 accepted (6.4) and submitted (5.5) papers.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train CycleReviewer-123B for review scoring instead of relying on an individual human reviewer | paper review scoring · Reviewer-5k test (proxy n−1) | Proxy MAE: 1.16 -> 0.92; 26.89% reduction | Helped |
| Use CycleResearcher-12B instead of AI Scientist to generate papers | automated research paper generation · CycleReviewer paper evaluation | average overall score: 4.31 -> 5.36 | Helped |
| Use CycleResearcher instead of AI Scientist for generating papers | automated research paper generation · Human evaluation | average overall score: 3.6 vs. 4.8 | Helped |
| Increase rejection-sampling candidates from 1 to 100 | CycleResearcher-12B · CycleReviewer paper evaluation | average score: approximately 5.36 to 7.02 | Helped |
| Add reinforcement learning (ablation) | CycleResearcher-12B · CycleReviewer paper evaluation | average score: 5.12 -> 5.36 | Helped |
| Add iterative training (ablation) | CycleResearcher-12B · CycleReviewer paper evaluation | average score: 5.21 -> 5.36 | Helped |
| Add NLL loss (ablation) | CycleResearcher-12B · CycleReviewer paper evaluation | average score: 4.91 -> 5.36 | Helped |
| Add NLL loss (ablation) | CycleResearcher-12B · CycleReviewer paper evaluation | acceptance rate: 12.03% -> 35.14% | Helped |
Surprising findings
- Removing NLL loss sharply degraded output, causing repetitive text and dropping acceptance rate to 12.03%.
- Rejection sampling with 100 samples raised simulated average score from 5.36 to 7.02, though this is measured by the same automated reviewer used for reward.
Limitations
- Experimental results in generated papers are fabricated by the model, not from real experiments.
- Reward hacking risk since policy and reward models are not updated simultaneously; evaluation is largely by the same automated reviewer.
- Limited to machine learning domain; CycleReviewer knowledge cutoff is January 2024.
Experimental setup
Datasets: Review-5k Research-14k ICLR 2024 OpenReview reviews
Benchmarks: Review-5k test set Research-14k test set arXiv preprints (Sep 2024)
Metrics: Proxy MSE Proxy MAE decision accuracy macro F1 average review score acceptance rate detection accuracy
Baselines: AI Scientist GPT-4o-mini GPT-4o Claude-3.5-Sonnet Gemini-1.5-pro GLM-4 DeepSeek-2.5
Builds on: SimPO DPO AI Scientist Fast-DetectGPT LoRA-GA
Closest papers by meaning
- MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research (NeurIPS 2025 · similarity 0.88)
- AI-Researcher: Autonomous Scientific Innovation (NeurIPS 2025 · similarity 0.87)
- AAAR-1.0: Assessing AI’s Potential to Assist Research (ICML 2025 · similarity 0.87)
- Evaluating AI-based Scientific Knowledge Synthesis with Epidemiological Systematic Reviews (NeurIPS 2026 · similarity 0.87)
- SciResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery (NeurIPS 2026 · similarity 0.87)
- SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.