Research map / Large language models / Language, safety and interpretability
Perception Score: A Learned Metric for Open-ended Text Generation Evaluation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Perception Score (large) instead of the strongest reported baseline | text-generation evaluation metrics · ROCStories | Pearson correlation with human judgment: 0.638 -> 0.692 | Helped |
| Use Perception Score (large) instead of the strongest reported baseline | text-generation evaluation metrics · Large Movie Review Conditional | Pearson correlation with human judgment: 0.491 -> 0.494 | Helped |
| Use Perception Score (large) instead of the strongest reported baseline | text-generation evaluation metrics · COCO Image Captions | Pearson correlation with human judgment: 0.504 -> 0.578 | Helped |
| Use Perception Score (large) instead of the strongest reported baseline | text-generation evaluation metrics · Large Movie Review Unconditional | Pearson correlation with human judgment: 0.173 -> 0.249 | Helped |
| Add model uncertainty reweighting (ablation) | Perception Score (base) · ROCStories | Pearson correlation with human judgment: 0.653 -> 0.671 | Helped |
| Add model uncertainty reweighting (ablation) | Perception Score (base) · COCO Image Captions | Pearson correlation with human judgment: 0.551 -> 0.563 | Helped |
| Add data uncertainty reweighting (ablation) | Perception Score (base) · ROCStories | Pearson correlation with human judgment: 0.636 -> 0.671 | Helped |
| Add data uncertainty reweighting (ablation) | Perception Score (base) · COCO Image Captions | Pearson correlation with human judgment: 0.538 -> 0.563 | Helped |
| Use a context-aware learned metric rather than reference-similarity-only scoring | Perception Score · ROCStories adversarial endings | score: 0.3650 | Helped |
Closest papers by meaning
- Open-Domain Text Evaluation via Contrastive Distribution Methods (ICML 2024 · similarity 0.87)
- Benchmarking LLMs' Judgments with No Gold Standard (ICLR 2025 · similarity 0.86)
- Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models (ICLR 2024 · similarity 0.85)
- OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation (CVPR 2025 · similarity 0.85)
- Generative Judge for Evaluating Alignment (ICLR 2024 · similarity 0.85)
- RevisEval: Improving LLM-as-a-Judge via Response-Adapted References (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.