Research map / Label-efficient and robust learning / Domain adaptation and generalization
TLLC: Transfer Learning-based Label Completion for Crowdsourcing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TLLC, a transfer-learning label completion method for crowdsourcing that pretrains a Siamese network on high-confidence instances and fine-tunes per-worker copies to complete each worker's missing labels.
Problem
Sparse crowdsourced label matrices make label completion hard; worker modeling is insufficient because each worker annotates only a few instances.
Approach
Build a source domain from all high-confidence instances (confident-learning-style filtering) to pretrain a Siamese network, then transfer and fine-tune per-worker copies on each worker's own annotated instances, and complete missing labels via distances between new embeddings and class centroids.
Key results
- On Income, MV accuracy after TLLC completion is 74.97% vs 71.17% after WSLC; AALI reaches 75.37% vs 64.87%.
- On Leaves, AALI accuracy after TLLC is 72.40% vs 64.06% after WSLC; MV 68.88% vs 63.80%.
- On Music genre, no statistically significant difference between TLLC and WSLC; TLLC underperforms on data with adversarial workers.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TLLC instead of WSLC for label completion | MV aggregation · Income | aggregation accuracy: 71.17 -> 74.97 | Helped |
| Use TLLC instead of WSLC for label completion | label aggregation methods · Income | aggregation accuracy: 71.83 -> 74.67; 71.15 -> 74.80; 71.83 -> 72.67; 64.87 -> 75.37; 71.00 -> 74.67 | Helped |
| Use TLLC instead of WSLC for label completion | label aggregation methods · Leaves | aggregation accuracy: 63.80 -> 68.88; 63.80 -> 67.19; 63.85 -> 69.01; 64.58 -> 69.79; 64.06 -> 72.40; 63.80 -> 68.75 | Helped |
| Use TLLC instead of WSLC for label completion | label aggregation methods · Music genre | aggregation accuracy: 77.86 -> 70.58; 77.86 -> 71.43; 77.89 -> 70.53; 78.86 -> 78.00; 77.83 -> 72.11; 77.86 -> 78.57 | Mixed |
| Add instance filtering (ablation) | TLLC + MV · Income | aggregation accuracy: 71.83 -> 74.97 | Helped |
| Add pretraining (ablation) | TLLC + MV · Income | aggregation accuracy: 74.16 -> 74.97 | Helped |
| Add transfer learning (ablation) | TLLC + MV · Income | aggregation accuracy: 71.66 -> 74.97 | Helped |
| Vary new embedding dimension (ablation) | TLLC + MV · Income | aggregation accuracy: 2: 74.94; 4: 71.83; 6: 71.66; 8: 73.33; 10: 72.66 | Mixed |
| Vary number of epochs (ablation) | TLLC + MV · Income | aggregation accuracy: 2: 74.94; 4: 72.33; 6: 73.00; 8: 72.16; 10: 72.83 | Mixed |
| Vary batch size (ablation) | TLLC + MV · Income | aggregation accuracy: 8: 71.83; 16: 72.50; 32: 74.94; 64: 73.33; 128: 73.16 | Mixed |
Surprising findings
- On Music genre, TLLC gives no significant gain over WSLC, which the authors attribute to adversarial workers with many labels being amplified by per-worker fine-tuning.
- Workers' annotation quality converges after WSLC completion, erasing worker-specific characteristics.
Limitations
- Lacks robustness to adversarial workers who annotate many instances.
- Performance on Music genre dataset is not significantly better than WSLC.
- Worker modeling time complexity is quadratic in number of instances.
Experimental setup
Datasets: Income Leaves Music genre
Benchmarks: Income Leaves Music genre
Metrics: aggregation accuracy Friedman test with Nemenyi post-hoc
Baselines: WSLC No completion (raw label matrix)
Builds on: WSLC confident learning Jung & Lease PMF label completion Ben-David domain adaptation bound
Closest papers by meaning
- Label Distribution Propagation-based Label Completion for Crowdsourcing (ICML 2025 · similarity 0.92)
- Unbiased Multi-Label Learning from Crowdsourced Annotations (ICML 2024 · similarity 0.87)
- Coupled Confusion Correction: Learning from Crowds with Sparse Annotations (AAAI 2024 · similarity 0.86)
- Multi-Instance Multi-Label Classification from Crowdsourced Labels (AAAI 2025 · similarity 0.85)
- KFNN: K-Free Nearest Neighbor For Crowdsourcing (NeurIPS 2024 · similarity 0.85)
- Label Aggregation for Composite Crowd Tasks by Worker Ability Constraint Satisfaction (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.