Research map / Vision-language and multimodal / Multimodal LLMs
Social Debiasing for Fair Multi-modal LLMs
Harry Cheng, Yangyang Guo, Qingpei Guo, Ming Yang, Tian Gan, Weili Guan, Liqiang Nie
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dataset resampling on top of fine-tuning (ablation) | LLaVA-7B+FT · CMSC test | MaxSkew@C: 0.8058 -> 0.7741 | Improved |
| add SFLoss on top of fine-tuning (ablation) | LLaVA-7B+FT · CMSC test | MaxSkew@C: 0.8058 -> 0.5838 | Improved |
| add SFLoss on top of resampling (full CSD) (ablation) | LLaVA-7B+Resample · CMSC test | MaxSkew@C: 0.7741 -> 0.5633 | Improved |
| add resampling on top of fine-tuning (ablation) | Bunny-8B+FT · CMSC test | MaxSkew@C: 0.9026 -> 0.4008 | Improved |
| add SFLoss on top of fine-tuning (ablation) | Qwen-VL-7B+FT · CMSC test | MaxSkew@C: 1.0227 -> 0.8041 | Improved |
| add SFLoss to resampling (full CSD) (ablation) | LLaVA-13B+Resample · CMSC test | MaxSkew@C: 0.4292 -> 0.4192 | No clear effect |
| replace naive fine-tuning with CSD | LLaVA-13B+FT · CMSC test | MinSkew@C: -1.5107 -> -0.8167 | Improved |
| replace naive fine-tuning with CSD | Qwen-VL-7B+FT · FairFace | MaxSkew@C: 0.6916 -> 0.2607 | Improved |
| fine-tune the full model instead of only the language component (ablation) | LLaVA-7B · CMSC test | MaxSkew@C: 0.9839 -> 0.8058 | Improved |
| fine-tune the visual component instead of language (ablation) | LLaVA-7B · CMSC test | MaxSkew@C: 0.9839 -> 1.0036 | No clear effect |
| use full-parameter CSD instead of LoRA (ablation) | LLaVA-7B · CMSC test | MaxSkew@C: 0.9945 -> 0.5633 | Improved |
| raise Bunny learning rate from 1e-7 to 5e-6 (ablation) | Bunny-8B · TextVQA | accuracy: 65.20 -> 41.06 | Hurt |
Closest papers by meaning
- Interpretable Debiasing of Vision-Language Models for Social Fairness (CVPR 2026 · similarity 0.89)
- Joint Vision-Language Social Bias Removal for CLIP (CVPR 2025 · similarity 0.88)
- Mitigating Social Bias in Large Language Models: A Multi-Objective Approach Within a Multi-Agent Framework (AAAI 2025 · similarity 0.88)
- SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples (CVPR 2024 · similarity 0.88)
- SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models (AAAI 2024 · similarity 0.88)
- Investigating and Mitigating Undesirable Biases in Large Language Models (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
