Research map / Label-efficient and robust learning / Test-time adaptation
Task Discovery: Finding the Tasks that Neural Networks Generalize on
Andrei Atanov, Andrey Filatov, Teresa Yeo, Ajay Sohmshetty, Amir Zamir
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use an adversarial train-test split based on a randomly initialized network task instead of a random split | CIFAR-10 multi-class classification · CIFAR-10 | accuracy: 0.78±0.04 -> 0.41±0.10 | Hurt |
| Use an adversarial train-test split based on a randomly initialized network task instead of a random split | CelebA hair_blond classification · CelebA hair_blond | accuracy: 0.94±0.00 -> 0.42±0.02 | Hurt |
| Use an adversarial train-test split based on a randomly initialized network task instead of a random split | ImageNet classification · ImageNet | top-1 accuracy: 0.59±0.01 -> 0.29±0.00 | Hurt |
| Use the discovered task set instead of randomly initialized network tasks | ResNet-18 task discovery · CIFAR-10 | average task similarity: 0.68 -> 0.51 | Improved |
| Use a SimCLR-pretrained embedding as the task-discovery input (ablation) | Task discovery · CIFAR-10 | similarity to human-labelled tasks | Improved |
| Vary the similarity-loss weight λ (ablation) | Shared-embedding task discovery · CIFAR-10 | average agreement score | No clear effect |
| Change any one stochasticity source between training runs (ablation) | Agreement-score measurement · CIFAR-10 human- and random-labelled tasks | agreement score: 1 | Hurt |
| Use fewer training images (ablation) | Agreement-score measurement on human-labelled tasks · CIFAR-10 | agreement score | Hurt |
| Use fewer training images (ablation) | Agreement-score measurement on random-labelled tasks · CIFAR-10 | agreement score | Mixed |
| Use the discovered tasks rather than human-labelled tasks | Agreement-score measurement across test domains · CIFAR-10 training; multiple test domains | agreement score stability | Improved |
| Discover tasks on TinyImageNet instead of evaluating human-labelled binary tasks | ResNet-18 · TinyImageNet | agreement score: below 0.65 -> above 0.8 | Improved |
Closest papers by meaning
- Scaling can lead to compositional generalization (NeurIPS 2025 · similarity 0.86)
- To Grok or not to Grok: Disentangling Generalization and Memorization on Corrupted Algorithmic Datasets (ICLR 2024 · similarity 0.84)
- Do Neural Networks Need Gradient Descent to Generalize? A Theoretical Study (NeurIPS 2025 · similarity 0.83)
- Unveiling Memorization–Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise (NeurIPS 2026 · similarity 0.83)
- Understanding Generalization through Decision Pattern Shift (NeurIPS 2026 · similarity 0.83)
- Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
