Research map / Architectures and efficiency / Attention and transformers
Debiasing Attention Mechanism in Transformer without Demographics
Shenyu Lu, Yipei Wang, Xiaoqian Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a method to debias transformer attention without access to sensitive/demographic labels, addressing fairness in transformers under privacy and legal constraints.
Problem
Fairness issues in transformers; existing debiasing methods require sensitive attributes (age, gender), which raise privacy concerns or violate regulations, and existing demographic-free fairness methods are limited.
Approach
Debiases the attention mechanism of transformers in a fairness-without-demographics setting, so that no sensitive labels are needed during training. Details beyond the abstract are not available in the provided text.
Closest papers by meaning
- Fairness-Aware Structured Pruning in Transformers (AAAI 2024 · similarity 0.87)
- Spectral Conditioning of Attention Improves Transformer Performance (NeurIPS 2025 · similarity 0.86)
- Transformers Learn Low Sensitivity Functions: Investigations and Implications (ICLR 2025 · similarity 0.86)
- Understanding Differential Transformer Unchains Pretrained Self-Attentions (NeurIPS 2025 · similarity 0.85)
- kNN Attention Demystified: A Theoretical Exploration for Scalable Transformers (ICLR 2025 · similarity 0.85)
- Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers (Student Abstract) (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
