Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
AutoWS-Bench-101: Benchmarking Automated Weak Supervision with 100 Labels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CLIP features with AutoWS on in-distribution image tasks | AutoWS methods · CIFAR-10 | accuracy | Helped |
| Use CLIP features for out-of-distribution or text-only tasks | AutoWS methods · Diverse and text tasks | accuracy | Hurt |
| Use multipolar rather than unipolar LFs (ablation) | Snuba · Diverse and text tasks | accuracy | Helped |
| Use multipolar rather than unipolar LFs as class count grows (ablation) | Snuba · Image tasks | coverage | Hurt |
| Use CLIP representations instead of other feature representations | AutoWS methods · Image tasks | coverage | Hurt |
| Increase LF cardinality up to a point (ablation) | Snuba · CLIP-applicable datasets | accuracy | Helped |
| Change Snuba's LF selection and abstention metric from its default (ablation) | Snuba · MNIST | accuracy | No clear effect |
| Increase the number of labeled examples from 100 to 1000 (ablation) | Snuba · MNIST | accuracy | No clear effect |
| Combine multiple embedding types instead of using the best single embedding (ablation) | GOGGLES · CIFAR-10 | accuracy: 0.5436 -> 0.2313 | Hurt |
| Combine multiple embedding types instead of using the best single embedding (ablation) | GOGGLES · Navier-Stokes | accuracy: 0.9353 -> 0.4868 | Hurt |
| Replace automatic LF selection with human-in-the-loop selection (ablation) | IWS · Navier-Stokes + ResNet-18 | accuracy: 0.7995 -> 0.9753 | Helped |
| Replace automatic LF selection with human-in-the-loop selection (ablation) | IWS · Yelp + Raw features | coverage: 1.0000 -> 0.5377 | Hurt |
Closest papers by meaning
- Stronger Than You Think: Benchmarking Weak Supervision on Realistic Tasks (NeurIPS 2024 · similarity 0.93)
- Weak Supervision Performance Evaluation via Partial Identification (NeurIPS 2024 · similarity 0.90)
- A General Framework for Learning from Weak Supervision (ICML 2024 · similarity 0.87)
- Pearls from Pebbles: Improved Confidence Functions for Auto-labeling (NeurIPS 2024 · similarity 0.87)
- Learning with Partial-Label and Unlabeled Data: A Uniform Treatment for Supervision Redundancy and Insufficiency (ICML 2024 · similarity 0.87)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.