Research map / Probabilistic and scientific modeling / Molecules and proteins
NAIPv2: Debiased Pairwise Learning for Efficient Paper Quality Estimation
Penghai Zhao, Jinyu Tian, Qinghua Xing, Xin Zhang, Zheng Li, Jianjun Qian, Ming-Ming Cheng, Xiang Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use debiased pairwise training instead of NAIPv1 regression | LLaMA-3 8B paper quality estimator · NAIDv2 test | AUC: 0.605 -> 0.782 | Improved |
| use pairwise training with pointwise inference instead of pointwise training (ablation) | NAIPv2 · NAIDv2 test | AUC: 0.633 -> 0.782 | Improved |
| use pairwise training with pointwise inference instead of concatenated pairwise inference (ablation) | NAIPv2 · NAIDv2 test | AUC: 0.720 -> 0.782 | Improved |
| restrict pairs to same time and hierarchical cluster (ablation) | NAIPv2 · NAIDv2 test | AUC: 0.739 -> 0.782 | Improved |
| group pairs by publication time only (ablation) | NAIPv2 · NAIDv2 test | AUC: 0.739 -> 0.736 | No clear effect |
| group pairs by GPT-generated keywords (ablation) | NAIPv2 · NAIDv2 test | Spearman rho: 0.400 -> 0.007 | Hurt |
| use RTS instead of mean reviewer score as label (ablation) | NAIPv2 · NAIDv2 test | AUC: 0.753 -> 0.782 | Improved |
| use mode aggregation instead of mean (ablation) | NAIPv2 · NAIDv2 test | AUC: 0.753 -> 0.730 | Hurt |
| add curriculum learning from easy to hard pairs (ablation) | NAIPv2 · NAIDv2 test | AUC: 0.770 -> 0.782 | Improved |
| emphasize hard pairs (small RTS gap) (ablation) | NAIPv2 · NAIDv2 test | performance | Hurt |
| hierarchical cluster max distance too small or too large (ablation) | NAIPv2 · NAIDv2 test | performance | Hurt |
Closest papers by meaning
- From Replication to Redesign: Exploring Pairwise Comparisons for LLM-Based Peer Review (NeurIPS 2025 · similarity 0.85)
- Not Search, But Scan: Benchmarking MLLMs on Scan-Oriented Academic Paper Reasoning (ICLR 2026 · similarity 0.82)
- Agent Reviewers: Domain-specific Multimodal Agents with Shared Memory for Paper Review (ICML 2025 · similarity 0.82)
- AAAR-1.0: Assessing AI’s Potential to Assist Research (ICML 2025 · similarity 0.82)
- PRISMM-Bench: A Benchmark of Peer-Review Grounded Multimodal Inconsistencies (ICLR 2026 · similarity 0.82)
- Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
