Research map / Image generation / Text-to-image diffusion
BARTScore: Evaluating Generated Text as Text Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Score generated text using conditional generation probabilities from a pretrained seq2seq model instead of representation matching | BARTScore · Summarization, NeR18 | Spearman correlation: 0.679 | Helped |
| Fine-tune BARTScore on CNNDM (ablation) | BARTScore · WMT19 MT, average across language pairs | Kendall's Tau: 0.281 -> 0.317 | Helped |
| Further fine-tune on ParaBank2 after CNNDM (ablation) | BARTScore-CNN · WMT19 MT, average across language pairs | Kendall's Tau: 0.317 -> 0.331 | Helped |
| Add the prompt “such as” (ablation) | BARTScore-CNN-Para · WMT19 de-en | Kendall's Tau: 0.205 -> 0.238; 0.033 improvement | Helped |
| Fine-tune on CNNDM and ParaBank2 (ablation) | BARTScore · NeR18 summarization | Spearman correlation: 0.679 -> 0.657 | Hurt |
| Add prompts (ablation) | BARTScore · Summarization, semantic overlap perspectives | Spearman correlation: up to 0.072 | Helped |
| Add prompts (ablation) | BARTScore-CNN · Rank19 factuality | Accuracy: 0.836 -> 0.796 | Hurt |
| Fine-tune on ParaBank2 after CNNDM (ablation) | BARTScore-CNN · Rank19 factuality | Accuracy: 0.836 -> 0.788 | Hurt |
| Fine-tune on CNNDM (ablation) | BARTScore · QAGS-XSUM factuality | Pearson correlation: 0.009 -> 0.184 | Helped |
| Fine-tune on CNNDM then ParaBank2 (ablation) | BARTScore · BAGEL data-to-text | Spearman correlation: 0.247 -> 0.330 | Helped |
| Replace uniform token weights with stop-word exclusion, IDF, or target-token prior weights (ablation) | BARTScore · Text evaluation experiments | Evaluation correlation | Hurt |
| Use PEGASUS or T5 instead of BART as the seq2seq backbone (ablation) | BARTScore · WMT19 MT | Kendall's Tau | Hurt |
Closest papers by meaning
- RevisEval: Improving LLM-as-a-Judge via Response-Adapted References (ICLR 2025 · similarity 0.88)
- Neither Valid nor Reliable? Investigating the Use of LLMs as Judges (NeurIPS 2025 · similarity 0.86)
- Retrieval is Accurate Generation (ICLR 2024 · similarity 0.86)
- Open-Domain Text Evaluation via Contrastive Distribution Methods (ICML 2024 · similarity 0.86)
- WritingBench: A Comprehensive Benchmark for Generative Writing (NeurIPS 2025 · similarity 0.86)
- Generative Judge for Evaluating Alignment (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.