Research map / Large language models / Language, safety and interpretability

When Should LLMs Be Less Specific? Selective Abstraction for Reliable Long-Form Text Generation

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Statistical frameworks for LLM evaluation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Selective Abstraction, which replaces low-confidence atomic claims in long-form LLM outputs with less specific but more reliable abstractions instead of deleting them, evaluated via risk-coverage curves and AURC on FactScore and LongFact.

Problem

Binary abstention in selective generation is too coarse for long-form text, where a single uncertain detail forces discarding large amounts of otherwise reliable content.

Approach

Generalize uncertain atomic claims along a sequence of less specific abstractions (e.g., date -> year -> decade -> century), select the most specific one whose verbalized confidence exceeds a threshold, and define coverage as entropy-based retained information measured via Wikidata set sizes.

Key results

• Atom-wise SA improves AURC over redaction by 17.43% on average on FactScore and 13.77% on LongFact, up to 27.73% for gpt-oss-120b on FactScore.
• At matched coverage, atom-wise SA achieves about 10-12 percentage points lower risk than Inline and Self-Revision baselines on average.
• The fact-checking agent reaches F1 0.93 against 102 manually checked claims.

Surprising findings

• Inline and Self-Revision operating points lie close to the redaction curve and sometimes exceed it, behaving more like atom deletion than abstraction.
• Thresholded prompt variants of Inline and Self-Revision did not produce a meaningful trade-off.

Limitations

• Verbalized confidence scores are imperfect and may be poorly calibrated.
• Atoms are treated as independent, ignoring dependencies between claims.
• Information measure relies on Wikidata counts under a uniform-prior assumption, a sparse proxy for real-world entity counts.

Experimental setup

Datasets: FactScore LongFact-Objects

Benchmarks: FactScore LongFact-Objects

Metrics: AURC risk coverage (entropy-based information retention) F1 of fact-checker against human labels

Baselines: Selective Redaction Inline Abstraction Self-Revision

Builds on: selective prediction hierarchical selective classification FactScore SAFE SPINACH

Closest papers by meaning

• Graph-based Uncertainty Metrics for Long-form Language Model Generations (NeurIPS 2024 · similarity 0.86)
• ClashEval: Quantifying the tug-of-war between an LLM’s internal prior and external evidence (NeurIPS 2024 · similarity 0.86)
• Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse (ICLR 2025 · similarity 0.85)
• Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs (ICLR 2024 · similarity 0.85)
• To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts (ICLR 2025 · similarity 0.85)
• Information Discernment in Large Language Models (NeurIPS 2026 · similarity 0.85)