Research map / Large language models / Language, safety and interpretability
Format as a Prior: Quantifying and Analyzing Bias in LLMs for Heterogeneous Data
Jiacheng Liu, Mayi Xu, Qiankun Pi, Wenli Li, Ming Zhong, Yuanyuan Zhu, Mengchi Liu, Tieyun Qian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| convert homogeneous inputs into heterogeneous formats with equivalent content | LLM QA · HotpotQA | accuracy: -9% | Hurt |
| convert homogeneous inputs into heterogeneous formats with equivalent content | LLM QA · MuSiQue | accuracy: -12% | Hurt |
| present both conflicting evidences in same format (plain text) instead of different formats (ablation) | ten LLMs · ConflictBank pairs | DCR: heterogeneous 3.01%-24.02%, markedly higher when homogeneous | Improved |
| increase model size within Qwen3 series (ablation) | Qwen3 · ConflictBank pairs | DCR | No clear effect |
| increase entries in structured evidence (4 -> 8 -> 12) vs texts (ablation) | KG evidence · Information richness, KGs vs texts | FPR for structured: 0.334 -> 0.412 -> 0.472 | Improved |
| increase entries in structured evidence (4 -> 12) vs texts (ablation) | Table evidence · Information richness, tables vs texts | FPR for structured: 0.238 -> 0.366 | Improved |
| corrupt structure tokens of structured evidence (ablation) | structured evidence vs texts · Structure quality | FPR for structured | Hurt |
| add attention-balancing reweighting at decoding | Qwen3-8B, Mistral-7B, Llama-3.1-8B · ConflictBank pairs | DCR | Improved |
| add attention-balancing reweighting at decoding | Qwen3-8B, Mistral-7B, Llama-3.1-8B · ConflictBank pairs | FPR | No clear effect |
| add attention-balancing reweighting | heterogeneous RAG-style QA · HotpotQA | accuracy: +6.5% | Improved |
| add attention-balancing reweighting | heterogeneous RAG-style QA · MuSiQue | accuracy: +9.5% | Improved |
Closest papers by meaning
- Cognitive Bias and Reassignment: Who Can Contribute High Quality LLM Data (AAAI 2025 · similarity 0.88)
- ReFF: Reinforcing Format Faithfulness in Language Models Across Varied Tasks (AAAI 2025 · similarity 0.88)
- Fixing It in Post: A Comparative Study of LLM Post-Training Data Quality and Model Performance (NeurIPS 2025 · similarity 0.87)
- Investigating and Mitigating Undesirable Biases in Large Language Models (AAAI 2025 · similarity 0.87)
- CEB: Compositional Evaluation Benchmark for Fairness in Large Language Models (ICLR 2025 · similarity 0.87)
- UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulation (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
