Research map / Label-efficient and robust learning / Recognition and long-tail

Political Bias Prediction Models Focus on Source Cues, Not Semantics

AAAI 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Data augmentation and generalization bias · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that political bias predictors trained on source-labeled news largely learn source cues, and proposes RatAA, a rationalization-based sentence selector combined with adversarial source removal, that cuts source-cue reliance while lowering bias accuracy to about 65%.

Problem

Political bias labels for news articles are assigned from the publishing source, so bias classifiers can learn to identify the source via superficial cues rather than semantic bias content.

Approach

Identify source-indicative sentence patterns via gradient-times-embedding attribution, measure whether bias-trained models can predict the source and flip bias when source cues are inserted, and propose RatAA: a rationalization-style sentence selector with adversarial source reversal (plus an optional uniform-distribution regularizer) so the bias predictor only sees source-irrelevant sentences.

Key results

• Bias-trained baselines predict the news source with 54%-78% accuracy from frozen embeddings (RoBERTa 0.78, BERT 0.73).
• Inserting a single source-cue sentence flips predicted bias often; BERT and RoBERTa average about 62% and 57% flip rates.
• RatAA and RatAA+Reg reach 67.6% and 63% bias accuracy versus about 80% for baselines, with a source-probe accuracy of 0.45 and 0.40 and much lower flip rates.

Measured effects reported in the paper

Surprising findings

• Baseline models trained only for bias prediction can identify the source with up to 78% accuracy, showing source information is embedded in bias embeddings.
• Adversarial adaptation alone leaves a center-bias source cue flip rate near 80%, showing AA alone is insufficient.

Limitations

• Accuracy of the proposed model drops about 15 points relative to baselines because source-correlated but bias-relevant cues are removed.
• Evaluated on a single public dataset (Baly et al. 2020); generalization to other datasets is untested.
• Hyperparameters such as k1, k2 and temperature were set by hand and the early-stopping choice was made without test-set selection.

Experimental setup

Datasets: Baly et al. (2020) political bias news dataset

Benchmarks: Baly et al. (2020) dataset held-out test split

Metrics: accuracy (bias) accuracy (source) bias flip rate under sentence insertion

Baselines: HLSTM HAN GNN Transformer Encoder BERT RoBERTa AA

Builds on: Adversarial Adaptation (Baly et al. 2020) Rationalization (Lei, Barzilay, and Jaakkola 2016) Hierarchical Attention Network Gumbel-softmax

Closest papers by meaning

• When Neutral Summaries Are Not That Neutral: Quantifying Political Neutrality in LLM-Generated News Summaries (Student Abstract) (AAAI 2025 · similarity 0.84)
• Linear Representations of Political Perspective Emerge in Large Language Models (ICLR 2025 · similarity 0.83)
• The Power of LLM-Generated Synthetic Data for Stance Detection in Online Political Discussions (ICLR 2025 · similarity 0.82)
• Quantifying Political Polarization through the Lens of Machine Translation and Vicarious Offense (AAAI 2024 · similarity 0.82)
• Merging AI Incidents Research with Political Misinformation Research: Introducing the Political Deepfakes Incidents Database (AAAI 2024 · similarity 0.81)
• Examining Alignment of Large Language Models through Representative Heuristics: the case of political stereotypes (ICLR 2025 · similarity 0.81)