Research map / Label-efficient and robust learning / Recognition and long-tail
Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling
Benedikt Boecking, Willie Neiswanger, Eric Xing, Artur Dubrawski
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IWS-LSE instead of active learning with sample-label queries | Downstream classifier · Six text classification tasks | test-set AUC | Improved |
| Use IWS-LSE instead of Snuba | Downstream classifier · Six text classification tasks | test-set AUC | Mixed |
| Use IWS-AS with human feedback rather than simulated-oracle feedback | IWS-AS · Amazon and IMDB | test-set AUC | No clear effect |
| Use IWS-AS LF queries instead of active-learning sample queries | Amazon · Amazon | queries to match mean test AUC: 200 iterations vs. about 600 iterations | Improved |
| Use IWS-AS LF queries instead of active-learning sample queries | IMDB · IMDB | queries to match mean test AUC: 200 iterations vs. more than 1000 iterations | Improved |
| Use IWS-LSE on COCO image tasks | Downstream image classifier · COCO image tasks | test-set AUC | Improved |
| Use higher-order n-gram LFs instead of unigram LFs (ablation) | Text LF family · Text classification tasks | test-set performance | No clear effect |
| Vary the IWS-LSE accuracy threshold r across a broad range above chance (ablation) | IWS-LSE-a and IWS-LSE-ac · Text datasets | test-set performance | No clear effect |
| Set the IWS-LSE accuracy threshold too close to 1.0 (ablation) | IWS-LSE-a and IWS-LSE-ac · Text datasets | test-set performance | Hurt |
| Vary the IWS-LSE-ac final-set-size parameter m̃ from 50 to 200 (ablation) | IWS-LSE-ac · Text datasets | test-set performance: 50 to 200 | No clear effect |
| Use real-user LF feedback instead of sample labeling | IWS-AS user study · Amazon | median response time: 4.2s vs. 7.9s | Improved |
| Use real-user LF feedback instead of sample labeling | IWS-AS user study · IMDB | median response time: 3.2s vs. 19.s | Improved |
Closest papers by meaning
- Characterizing the Impacts of Semi-supervised Learning for Weak Supervision (NeurIPS 2023 · similarity 0.88)
- Creating Training Sets via Weak Indirect Supervision (ICLR 2022 · similarity 0.88)
- Ordinal Programmatic Weak Supervision and Crowdsourcing for Estimating Cognitive States (Student Abstract) (AAAI 2023 · similarity 0.87)
- Stronger Than You Think: Benchmarking Weak Supervision on Realistic Tasks (NeurIPS 2024 · similarity 0.87)
- Weak Supervision Performance Evaluation via Partial Identification (NeurIPS 2024 · similarity 0.86)
- A General Framework for Learning from Weak Supervision (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
