Research map / Label-efficient and robust learning / Recognition and long-tail
SCISSOR: Mitigating Semantic Bias through Cluster-Aware Siamese Networks for Robust Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCISSOR, a Siamese-network debiasing module trained with cluster-aware triplet losses on a frozen pretrained encoder to suppress semantic-cluster shortcuts, improving OOD F1 on text and image classification.
Problem
Shortcut learning hurts OOD generalization; the paper argues that imbalanced label distributions within semantic clusters of sample embeddings act as shortcuts, not only superficial features.
Approach
Cluster sample embeddings with Markov Clustering, label clusters as balanced or imbalanced, build quadruplets (anchor, positive, intermediate, negative) that include cluster information, and train a lightweight Transformer-based Siamese debiasing module with triplet loss on top of a frozen PLM/vision encoder before training a linear classification head.
Key results
- SCISSOR reports +5.3 F1 on GYAFC, +7.3 on Yelp, +7.7 on Chest-XRay and +1 on Not-MNIST over baselines.
- For ViT on Chest-XRay, SCISSOR reaches 83.92 accuracy vs 72.38 for the baseline ViT, and for BERT on Yelp it reaches 90.65 vs 81.17.
- Ablation shows replacing MCL with K-means yields almost identical performance, and SCISSOR beats the Triplet variant by about 2 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SCISSOR to BERT | BERT · GYAFC | accuracy: 70.40 -> 78.20 | Helped |
| Add SCISSOR to BERT | BERT · Yelp | accuracy: 81.17 -> 90.65 | Helped |
| Add SCISSOR to RoBERTa | RoBERTa · GYAFC | accuracy: 73.66 -> 81.34 | Helped |
| Add SCISSOR to RoBERTa | RoBERTa · Yelp | accuracy: 75.76 -> 87.79 | Helped |
| Add SCISSOR to ViT | ViT · Chest-XRay | accuracy: 72.38 -> 83.92 | Helped |
| Add SCISSOR to ViT | ViT · Not-MNIST | accuracy: 88.87 -> 90.89 | Helped |
| Add SCISSOR to Swin | Swin · Chest-XRay | accuracy: 84.54 -> 88.65 | Helped |
| Add SCISSOR to Swin | Swin · Not-MNIST | accuracy: 92.72 -> 92.74 | No clear effect |
| Add SCISSOR to DINOv2 | DINOv2 · Chest-XRay | accuracy: 68.94 -> 73.59 | Helped |
| Add SCISSOR to DINOv2 | DINOv2 · Not-MNIST | accuracy: 85.40 -> 85.75 | Helped |
| Replace MCL with K-means clustering (ablation) | SCISSOR · GYAFC and Yelp | accuracy: 76.52 vs 78.20; 88.50 vs 90.65 | No clear effect |
| Replace MCL with K-means clustering (ablation) | SCISSOR · Chest-XRay and Not-MNIST | accuracy: 82.20 vs 83.92; 89.44 vs 90.89 | No clear effect |
Surprising findings
- Replacing Markov clustering with K-means gives nearly identical results, suggesting the specific clustering algorithm matters little.
- RAZOR rewriting lowers LLaMA accuracy by about 2 points on GYAFC, and IRM underperforms the baseline on some small datasets.
Limitations
- Clustering via MCL has O(n^2) cost, mitigated only by approximate heuristics the paper does not evaluate.
- No standard adversarial OOD benchmark exists for the identified semantic shortcuts, so OOD evaluation relies on a cross-validation proxy.
- Theoretical lemmas rely on strong assumptions (bounded Hessian, bounded weights), and the Hopkins statistic values are near 0 across all models.
Experimental setup
Datasets: GYAFC Yelp Chest-XRay Not-MNIST
Benchmarks: GYAFC Yelp Chest-XRay Not-MNIST
Metrics: accuracy F1
Baselines: IRM RAZOR LC Triplet K-means variant of SCISSOR
Builds on: triplet loss Markov Clustering Algorithm IRM
Closest papers by meaning
- Interpretable Debiasing of Vision-Language Models for Social Fairness (CVPR 2026 · similarity 0.85)
- Joint Vision-Language Social Bias Removal for CLIP (CVPR 2025 · similarity 0.85)
- A Unified Debiasing Approach for Vision-Language Models across Modalities and Tasks (NeurIPS 2024 · similarity 0.84)
- Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity (ICML 2024 · similarity 0.84)
- Low-Biased General Annotated Dataset Generation (CVPR 2025 · similarity 0.84)
- MoDE: CLIP Data Experts via Clustering (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.