Research map / Large language models / Language, safety and interpretability
A Large-Scale Dataset for Argument Quality Ranking: Construction and Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Bi-LSTM GloVe with BERT-V | Argument quality ranking on IBM-Rank-30k · IBM-Rank-30k test (WA and MACE-P labels) | Pearson r: $.44$ -> $.48$ (WA); $.43$ -> $.49$ (MACE-P) | Helped |
| Fine-tune BERT rather than use frozen BERT-V features (ablation) | BERT-V argument quality ranker · IBM-Rank-30k test | Pearson r: $.48$ -> $.51$ (WA); $.49$ -> $.52$ (MACE-P) | Helped |
| Add the topic to the argument input after [SEP] (ablation) | BERT-FT argument quality ranker · IBM-Rank-30k test | Pearson r: $.51$ -> $.52$ (WA); $.52$ -> $.53$ (MACE-P) | Helped |
| Add the topic to the argument input after [SEP] (ablation) | BERT-FT argument quality ranker · IBM-Rank-30k test | Spearman ρ: $.47$ -> $.48$ (WA); $.5$ -> $.52$ (MACE-P) | Helped |
| Add the topic to the argument input after [SEP] (ablation) | BERT-FT argument quality ranker · SwanRank cross-domain | RRSE: $0.91$ -> $0.81$ | Helped |
| Add the topic to the argument input after [SEP] (ablation) | BERT-FT argument quality ranker · UKPRank | Spearman ρ: $.63$ -> $.62$ | Hurt |
| Use BERT-FT-TOPIC instead of the published Swanson cross-domain model | SwanRank cross-domain argument ranking · SwanRank cross-domain weighted average | RRSE: $0.89$ -> $0.81$ | Helped |
| Use BERT-FT-TOPIC instead of BERT-FT (ablation) | Argument quality ranking · SwanRank cross-domain, four topic pairs | RRSE: $0.8$ -> $0.65$ (GM); $0.82$ -> $0.71$ (GC); $1.1$ -> $0.96$ (DP); $0.98$ -> $1.01$ (EV) | Mixed |
| Replace BERT-FT with BERT-FT-TOPIC (ablation) | Argument quality ranking · UKPRank | Pearson r: $.45$ -> $.46$ | Helped |
| Replace BERT-FT with BERT-FT-TOPIC (ablation) | Argument quality ranking · UKPRank | Spearman ρ: $.63$ -> $.62$ | Hurt |
| Use weighted-average scores rather than MACE-P scores as quality labels | BERT-FT-TOPIC argument ranker · IBM-Rank-30k test | Spearman ρ: $.48$ (WA) vs $.52$ (MACE-P) | Mixed |
| Use argument length as the ranking signal | Arg-Length baseline · IBM-Rank-30k test | Pearson r: $.21$ (WA); $.22$ (MACE-P) | Hurt |
Closest papers by meaning
- Corpus Wide Argument Mining—A Working Solution (AAAI 2020 · similarity 0.87)
- OpenDebateEvidence: A Massive-Scale Argument Mining and Summarization Dataset (NeurIPS 2024 · similarity 0.87)
- Fine-Grained Argument Unit Recognition and Classification (AAAI 2020 · similarity 0.87)
- An Extension-Based Argument-Ranking Semantics: Social Rankings in Abstract Argumentation (AAAI 2025 · similarity 0.85)
- Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models? (NeurIPS 2025 · similarity 0.83)
- From Crowdsourced Data to High-quality Benchmarks: Arena-Hard and Benchbuilder Pipeline (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.