Research map / Vision-language and multimodal / CLIP and zero-shot
CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies data balancing for mitigating gender-occupation and representation biases in CLIP, proposing the M4 subsampling algorithm and training over 150 models on WebLI.
Problem
CLIP-style multimodal models absorb societal stereotypes (representation and association biases) from web data, and it is unclear how effective data balancing is for mitigating them and how it affects model quality.
Approach
Introduces Multi-Modal Moment Matching (M4), a scalable constrained subsampling algorithm that decorrelates sensitive attributes from labels and proxies (first- and second-order statistics) via a dual/SGD solver, and uses two-stage training to study how CLIP learns and unlearns biases.
Key results
- Data balancing with proxies reduces representation bias in CLIP models, and fine-tuning on balanced data for only 10% of training effectively counters representation bias.
- Balancing improves 10-shot classification on average (e.g., ImageNet 50.4 baseline vs 50.9 balanced) but hurts retrieval.
- SigLIP ViT-B/16 with data balancing and quality filtering improves COCO img2txt@5 from 86.0 to 87.1 and ImageNet 0-shot from 77.0 to 77.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Balance training data with M4, with or without proxies | CLIP · ILSRCV2012 | representation bias (gender parity) | Helped |
| Add proxy variables to M4 balancing (ablation) | CLIP · ILSRCV2012 | representation bias (gender parity) | Helped |
| Add proxy constraints to data balancing (ablation) | CLIP · FairFace, UTKFace, and MIAP | association bias | Hurt |
| Fine-tune on balanced data | CLIP · ILSRCV2012 | representation bias (gender parity) | Helped |
| Fine-tune on balanced data | CLIP · FairFace, UTKFace, and MIAP | association bias | No clear effect |
| Balance training data | CLIP ViT-B/16 · Classification benchmarks and COCO/Flickr retrieval | task performance | Mixed |
| Apply M4 balancing with proxies | SigLIP ViT-B/16 · FairFace | association bias: 38.8 -> 29.9 | Helped |
| Apply M4 balancing with proxies | SigLIP ViT-B/16 · UTKFace | association bias: 38.2 -> 33.7 | Helped |
| Apply M4 balancing with proxies | SigLIP ViT-B/16 · MIAP | association bias: 28.4 -> 20.5 | Helped |
| Apply M4 balancing with proxies | SigLIP ViT-B/16 · ImageNet-1k, 0-shot | accuracy: 77.0 -> 77.5 | Helped |
| Apply M4 balancing with proxies | SigLIP ViT-B/16 · COCO image-to-text @5 | Recall@5: 86.0 -> 87.1 | Helped |
| Apply M4 balancing with proxies | SigLIP ViT-B/16 · SigLIP classification and retrieval benchmarks | task performance | Helped |
Surprising findings
- Adding proxies hurts association bias reduction even though it helps representation bias, due to competing constraints.
- Fine-tuning on balanced data is less effective for association bias, which changes gradually with training duration regardless of order.
Limitations
- Focuses only on perceived gender and occupations, and on contrastive rather than generative models.
- Statistical metrics cannot capture the full notion of fairness; which attributes and label associations are acceptable is not addressed.
Experimental setup
Datasets: WebLI (English subset) ImageNet-ILSVRC2012 FairFace UTKFace MIAP COCO Flickr Adult Default of Credit Card Clients
Benchmarks: ImageNet FairFace UTKFace MIAP COCO retrieval Flickr retrieval Colorectal Cars Birds UC CIFAR100 DTD
Metrics: mean parity association bias parity zero-shot accuracy few-shot accuracy recall@5 demographic parity classification error balanced error
Baselines: baseline (no intervention) balanced SigLIP without intervention correlation remover reduce-to-binary threshold optimizer
Builds on: CLIP SigLIP Big Vision
Closest papers by meaning
- Diffusion Feedback Helps CLIP See Better (ICLR 2025 · similarity 0.89)
- MASS: Overcoming Language Bias in Image-Text Matching (AAAI 2025 · similarity 0.89)
- Demystifying CLIP Data (ICLR 2024 · similarity 0.88)
- CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning (NeurIPS 2024 · similarity 0.88)
- What Makes CLIP More Robust to Long-Tailed Pre-Training Data? A Controlled Study for Transferable Insights (NeurIPS 2024 · similarity 0.88)
- Post-pre-training for Modality Alignment in Vision-Language Foundation Models (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.