Research map / Large language models / Language, safety and interpretability
RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds RewriteLM, an instruction-tuned PaLM/PaLM 2 model for cross-sentence text rewriting, using Wiki-edit and CoT-synthesized instruction data, an automatically scored preference ranking for RL, and a new OpenRewriteEval benchmark.
Problem
Existing LLMs and rewriting benchmarks handle text rewriting poorly, especially cross-sentence, open-ended natural-language-instructed rewrites, and prior work relies on human labelers for instruction and preference data.
Approach
Generate rewrite instruction data from Wiki edit summaries (refined by a fine-tuned PaLM2-XXS) and from CoT-prompted synthetic instructions, filter with NLI and edit-ratio heuristics, and build reward-model preference pairs automatically from a heuristic quality score instead of human rankings, then apply RL on the SFT model.
Key results
- Rewrite-RL$_\text{r/w}$-PaLM 2 achieves NLI (s-p) 0.96 and SARI 40.66 on OpenRewriteEval, the best NLI and Updated-R (40.10) among compared models.
- On EditEval, Rewrite-RL$_\text{r/w}$-PaLM 2 gets the best Updated-R on FRU (53.93) and WFI (49.53 Updated-R) while being mostly on par with PEER-11 and InsGPT.
- Human evaluation (80 examples, 5 experts) gives Rewrite-RL$_\text{r/w}$-PaLM 2 the highest average score 1.886 vs ChatGPT 1.83 and Alpaca 13B 1.814.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune PaLM 2-S on the rewriting instruction dataset | PaLM 2-S · OpenRewriteEval | SARI: 28.62 -> 40.92 | Helped |
| Apply reinforcement learning without the synthetic preference dataset (ablation) | Rewrite-PaLM 2 · OpenRewriteEval | SARI: 40.92 -> 40.97 | No clear effect |
| Use synthetic preference data when training the reward model (ablation) | Rewrite-RL-PaLM 2 · OpenRewriteEval | NLI (s-p): 0.94 -> 0.96 | Helped |
| Use synthetic preference data when training the reward model (ablation) | Rewrite-RL-PaLM 2 · OpenRewriteEval | SARI: 40.97 -> 40.66 | Hurt |
| Use synthetic preference data when training the reward model (ablation) | Rewrite-RL-PaLM 2 · OpenRewriteEval | Updated-R: 39.36 -> 40.10 | Helped |
| Apply reinforcement learning using synthetic preference data | Rewrite-PaLM 2 · Human evaluation | AVG: 1.869 -> 1.886 | Helped |
| Apply supervised rewriting fine-tuning to PaLM 2-S | PaLM 2-S · EditEval TRK | SARI: 34.32 -> 40.81 | Helped |
| Apply supervised rewriting fine-tuning to PaLM 2-S | PaLM 2-S · EditEval WNC | SARI: 25.2 -> 37.11 | Helped |
| Apply supervised rewriting fine-tuning to PaLM 2-S | PaLM 2-S · EditEval WFI | SARI: 11.41 -> 26.55 | Helped |
Surprising findings
- Pretrained PaLM 2-S without instruction tuning produces very high edit ratios (1.22) and hallucinations, indicating it rewrites far too much.
Limitations
- Relies on heuristic NLI and edit-ratio scoring for preference data rather than human judgments.
- Evaluation models are closed PaLM-family models and the dataset is not clearly released.
- Human-level instruction success and content preservation not reached, per the authors.
Experimental setup
Datasets: Wikipedia revision history C4 OpenRewriteEval EditEval
Benchmarks: OpenRewriteEval EditEval
Metrics: NLI edit ratio SARI GLEU Updated-ROUGE ROUGE-L human Likert ratings (instruction success, content preservat
Baselines: PaLM PaLM 2 LLaMA Flan-PaLM Alpaca Alpaca-PaLM 2 Vicuna ChatGPT InsGPT PEER-11 OPT GPT-3
Builds on: InstructGPT PEER Self-Instruct Flan-PaLM
Closest papers by meaning
- MaFeRw: Query Rewriting with Multi-Aspect Feedbacks for Retrieval-Augmented Large Language Models (AAAI 2025 · similarity 0.87)
- LongGenBench: Benchmarking Long-Form Generation in Long Context LLMs (ICLR 2025 · similarity 0.86)
- Incorporating Self-Rewriting into Large Language Model Reasoning Reinforcement (AAAI 2026 · similarity 0.86)
- Teaching Large Language Models to Translate with Comparison (AAAI 2024 · similarity 0.86)
- Evaluating the Zero-shot Robustness of Instruction-tuned Language Models (ICLR 2024 · similarity 0.86)
- MAmmoTH2: Scaling Instructions from the Web (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.