Research map / Label-efficient and robust learning / Domain adaptation and generalization
CROSSNEWS: A Cross-Genre Authorship Verification and Attribution Benchmark
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CROSSNEWS, a cross-genre authorship dataset pairing news articles and tweets by the same journalists, benchmarks verification and attribution models, and proposes SELMA, a zero-shot LLM-embedding method.
Problem
Authorship models generalize poorly across genres and topics, and existing datasets are single-genre or too small for modern embedding-based methods, so cross-genre authorship verification and attribution lack suitable benchmarks.
Approach
Build a large cross-genre dataset linking the same journalists' news articles and tweets (silver set via Wikidata-Twitter linkage, manually verified gold set), and propose SELMA, which uses an instruction-tuned LLM embedding model (e5-mistral-7b-instruct) with a style instruction for zero-shot verification and attribution.
Key results
- SELMA achieves the best accuracy on Article-Article and Article-Tweet attribution setups among all models, outperforming other methods especially in cross-genre settings (Table 3).
- LUAR gets 67.8% accuracy on Article-Tweet verification training pairs versus N-gram's 58.3%, and LIP reaches 84.9% accuracy on Tweet-Tweet verification.
- N-gram, a top single-genre method, drops to near random on Article-Tweet verification (55.8-58.3% accuracy range), showing poor cross-genre transfer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SELMA embeddings instead of direct LLM prompting | CROSSNEWS verification · Article-Tweet | accuracy: 78.5 -> 64.3 | Hurt |
| Use SELMA embeddings instead of direct LLM prompting | CROSSNEWS verification · Article-Article | accuracy: 85.8 -> 77.1 | Hurt |
| Use SELMA embeddings instead of direct LLM prompting | CROSSNEWS verification · Tweet-Tweet | accuracy: 84.9 -> 53.3 | Hurt |
| Add task-description prompt (ablation) | SELMA · CROSSNEWS attribution, Article-Article | accuracy: 52.8 -> 56.9 | Helped |
| Use task-description prompt rather than no prompt (ablation) | SELMA · CROSSNEWS attribution, Article-Tweet | accuracy: 15.8 -> 18.4 | Helped |
| Use LIP prompt rather than no prompt (ablation) | SELMA · CROSSNEWS attribution, Tweet-Article | accuracy: 18.7 -> 21.5 | Helped |
| Add task-description prompt (ablation) | SELMA · CROSSNEWS attribution, Tweet-Tweet | accuracy: 31.0 -> 33.3 | Helped |
| Restrict documents to a single topic (ablation) | authorship verification models · CROSSNEWS Article-Article | accuracy: 70.4 -> 64.5 (-5.9%) | Hurt |
| Restrict documents to a single topic (ablation) | authorship attribution models · CROSSNEWS Article-Article | accuracy: 45.6 -> 65.8 (+20.2%) | Mixed |
| Restrict documents to a single topic (ablation) | authorship verification models · CROSSNEWS Tweet-Tweet | accuracy: 69.4 -> 52.2 (-17.2%) | Hurt |
| Restrict documents to a single topic (ablation) | authorship attribution models · CROSSNEWS Tweet-Tweet | accuracy: 38.7 -> 37.1 (-1.6%) | Mixed |
Surprising findings
- Topic impacts verification and attribution in opposite ways: verification accuracy drops when topics are isolated while attribution accuracy rises for articles.
- SELMA with a 7B embedding model outperforms prompting LLaMA-3 70B for verification and attribution in several settings.
Limitations
- Silver set linkage accuracy is estimated at 93.6%, so label noise is present in training data.
- Gold set is limited to 500 authors from three news organizations and English-language Twitter/X posts.
Experimental setup
Datasets: CROSSNEWS silver set CROSSNEWS gold set Wikidata Twitter/X API
Benchmarks: CROSSNEWS
Metrics: accuracy F1 R@8 average rank
Baselines: N-gram PPM O2D2 PART LUAR STEL TaskOnly prompting PromptAV LIP
Builds on: N-gram authorship method LUAR PART STEL PromptAV LIP
Closest papers by meaning
- ALISON: Fast and Effective Stylometric Authorship Obfuscation (AAAI 2024 · similarity 0.81)
- A Synthetic Dataset for Personal Attribute Inference (NeurIPS 2024 · similarity 0.80)
- ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer (AAAI 2024 · similarity 0.79)
- MUSE: Machine Unlearning Six-Way Evaluation for Language Models (ICLR 2025 · similarity 0.79)
- STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasings (ICML 2025 · similarity 0.79)
- Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models (ICLR 2026 · similarity 0.78)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.