Research map / Label-efficient and robust learning / Semi-supervised learning
SimLabel: Similarity-Weighted Semi-supervision for Multi-annotator Learning with Missing Labels
Liyun Zhang, Zheng Lian, Hong Liu, Takanori Takebe, Yuta Nakashima
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace skipping with similarity-weighted soft labels for missing annotations | QuMATL · AMER2 | average accuracy: 0.86 -> 0.88 | Improved |
| replace skipping with similarity-weighted soft labels | PADL · AMER (13 annotators) | average accuracy: 0.79 -> 0.83 | Improved |
| add confidence-based iterative refinement (ablation) | SimLabel with QuMATL · AMER2 | average accuracy: 0.88 -> 0.90 | Improved |
| add confidence-based iterative refinement (ablation) | SimLabel with MaDL · AMER (13 annotators) | average accuracy: 0.84 -> 0.86 | Improved |
| add confidence refinement on top of soft labels (ablation) | SimLabel with PADL · AMER (40% removed) | average accuracy: 0.64 -> 0.64 | No clear effect |
| add similarity-weighted soft labels under 40% extra missing | QuMATL · STREET-Ha (40% removed) | average accuracy: 0.52 -> 0.56 | Improved |
| add similarity-weighted soft labels (lower DIC is better) | QuMATL · AMER2 | DIC: 0.22 -> 0.17 | Improved |
| use Cohen's kappa instead of Pearson for similarity (ablation) | SimLabel with QuMATL · AMER2 | average accuracy: 0.86 -> 0.90 | Improved |
| use Cohen's kappa instead of Krippendorff's alpha (ablation) | SimLabel with QuMATL · AMER2 | average accuracy: 0.88 -> 0.90 | Improved |
| raise confidence threshold from 0.6 to 0.8 (ablation) | SimLabel with QuMATL · AMER2 | average accuracy: 0.90 -> 0.86 | Hurt |
| lower confidence threshold from 0.6 to 0.5 (ablation) | SimLabel with QuMATL · AMER2 | average accuracy: 0.90 -> 0.89 | Hurt |
| combine max probability with entropy term in confidence (ablation) | SimLabel with QuMATL · AMER2 | average accuracy: 0.87 -> 0.90 | Improved |
Closest papers by meaning
- Unbiased Multi-Label Learning from Crowdsourced Annotations (ICML 2024 · similarity 0.87)
- TLLC: Transfer Learning-based Label Completion for Crowdsourcing (ICML 2025 · similarity 0.86)
- Coupled Confusion Correction: Learning from Crowds with Sparse Annotations (AAAI 2024 · similarity 0.86)
- Collaborative Refining for Learning from Inaccurate Labels (NeurIPS 2024 · similarity 0.86)
- Multi-Instance Multi-Label Classification from Crowdsourced Labels (AAAI 2025 · similarity 0.86)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
