AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

A Semantic QA-Based Approach for Text Summarization Evaluation

Ping Chen, Fei Wu, Tong Wang, Wei Ding

AAAI 2018 · Large language models › Language, safety and interpretability · approach: Controlled text generation · 41 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Apply topic-word question filtering (ablation)QA-based evaluation · DUC 2007, question length 20, confidence 0.8Pearson correlation: 0.6120 -> 0.7725Improved
Apply topic-word question filtering (ablation)QA-based evaluation · DUC 2007, question length 20, confidence 0.9Pearson correlation: 0.6379 -> 0.8363Improved
Apply topic-word question filtering (ablation)QA-based evaluation · DUC 2007, question length 30, confidence 0.8Pearson correlation: 0.6373 -> 0.6310Hurt
Apply topic-word question filtering (ablation)QA-based evaluation · DUC 2007, question length 30, confidence 0.9Pearson correlation: 0.5092 -> 0.5462Improved
Increase question length limit from 20 to 30 words (ablation)QA-based evaluation · DUC 2007, confidence 0.8, no filteringPearson correlation: 0.6120 -> 0.6373Improved
Increase question length limit from 20 to 30 words (ablation)QA-based evaluation · DUC 2007, confidence 0.9, no filteringPearson correlation: 0.6379 -> 0.5092Hurt
Increase question length limit from 20 to 30 words (ablation)QA-based evaluation · DUC 2007, confidence 0.8, with filteringPearson correlation: 0.7725 -> 0.6310Hurt
Increase question length limit from 20 to 30 words (ablation)QA-based evaluation · DUC 2007, confidence 0.9, with filteringPearson correlation: 0.8363 -> 0.5462Hurt
Raise answer confidence threshold from 0.8 to 0.9 (ablation)QA-based evaluation · DUC 2007, question length 20, no filteringPearson correlation: 0.6120 -> 0.6379Improved
Raise answer confidence threshold from 0.8 to 0.9 (ablation)QA-based evaluation · DUC 2007, question length 20, with filteringPearson correlation: 0.7725 -> 0.8363Improved
Raise answer confidence threshold from 0.8 to 0.9 (ablation)QA-based evaluation · DUC 2007, question length 30, no filteringPearson correlation: 0.6373 -> 0.5092Hurt
Raise answer confidence threshold from 0.8 to 0.9 (ablation)QA-based evaluation · DUC 2007, question length 30, with filteringPearson correlation: 0.6310 -> 0.5462Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy