ICLR 2021 accepted papers on Label-efficient and robust learning
139 of the 860 papers accepted at ICLR 2021 (16.2%) are on Label-efficient and robust learning, compared with 11.5% at ICLR 2020. They fall into 9 topics.
See ICLR 2021 in the interactive map
Submitting to ICLR? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Recognition and long-tail · 65 papers
- Prototypical Contrastive Learning of Unsupervised Representations (469 citations)
- Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval (329 citations)
- Revisiting Few-sample BERT Fine-tuning (210 citations)
- AdaSpeech: Adaptive Text to Speech for Custom Voice (77 citations)
- Learning Invariant Representations for Reinforcement Learning without Reconstruction (73 citations)
- On Learning Universal Representations Across Languages (65 citations)
- Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning (60 citations)
- Model Patching: Closing the Subgroup Performance Gap with Data Augmentation (58 citations)
- Long-tail learning via logit adjustment (52 citations)
- Learning from others' mistakes: Avoiding dataset biases without modeling them (50 citations)
- Isotropy in the Contextual Embedding Space: Clusters and Manifolds (49 citations)
- Trusted Multi-View Classification (48 citations)
and 53 more in the interactive map.
Zero-/few-shot and meta-learning · 16 papers
- Free Lunch for Few-shot Learning: Distribution Calibration (179 citations)
- A Universal Representation Transformer Layer for Few-Shot Image Classification (67 citations)
- BOIL: Towards Representation Change for Few-shot Learning (60 citations)
- MELR: Meta-Learning via Modeling Episode-Level Relationships for Few-Shot Learning (50 citations)
- IEPT: Instance-Level and Episode-Level Pretext Tasks for Few-Shot Learning (47 citations)
- Improving Zero-Shot Voice Style Transfer via Disentangled Representation Learning (26 citations)
- Adaptive and Generative Zero-Shot Learning (24 citations)
- Contrastive Syn-to-Real Generalization (19 citations)
- Zero-shot Synthesis with Group-Supervised Learning (12 citations)
- Isometric Propagation Network for Generalized Zero-shot Learning (11 citations)
- Meta-GMVAE: Mixture of Gaussian VAE for Unsupervised Meta-Learning (11 citations)
- Class Normalization for (Continual)? Generalized Zero-Shot Learning (9 citations)
and 4 more in the interactive map.
Self-supervised and contrastive · 15 papers
- Learning and Evaluating Representations for Deep One-Class Classification (105 citations)
- What Should Not Be Contrastive in Contrastive Learning (91 citations)
- Semantic Re-tuning with Contrastive Tension (83 citations)
- SEED: Self-supervised Distillation For Visual Representation (71 citations)
- $i$-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning (52 citations)
- Representation Learning via Invariant Causal Mechanisms (39 citations)
- Viewmaker Networks: Learning Views for Unsupervised Representation Learning (22 citations)
- On Self-Supervised Image Representations for GAN Evaluation (13 citations)
- Self-supervised Learning from a Multi-view Perspective (11 citations)
- For self-supervised learning, Rationality implies generalization, provably (7 citations)
- Negative Data Augmentation (7 citations)
- Generalized Multimodal ELBO (4 citations)
and 3 more in the interactive map.
Knowledge distillation · 10 papers
- FastSpeech 2: Fast and High-Quality End-to-End Text to Speech (506 citations)
- Knowledge distillation via softmax regression representation learning (55 citations)
- Understanding and Improving Lexical Choice in Non-Autoregressive Translation (55 citations)
- Rethinking Soft Labels for Knowledge Distillation: A Bias–Variance Tradeoff Perspective (44 citations)
- Distilling Knowledge from Reader to Retriever for Question Answering (39 citations)
- MixKD: Towards Efficient Distillation of Large-scale Language Models (29 citations)
- Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Study (26 citations)
- Undistillable: Making A Nasty Teacher That CANNOT teach students (19 citations)
- Knowledge Distillation as Semiparametric Inference (6 citations)
- Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors
Noisy labels · 9 papers
- Robust early-learning: Hindering the memorization of noisy labels (121 citations)
- Sharpness-aware Minimization for Efficiently Improving Generalization (104 citations)
- MoPro: Webly Supervised Learning with Momentum Prototypes (47 citations)
- Unlearnable Examples: Making Personal Data Unexploitable (45 citations)
- Robust Curriculum Learning: from clean label detection to noisy label self-correction (39 citations)
- Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization (31 citations)
- Learning with Feature-Dependent Label Noise: A Progressive Approach (31 citations)
- Learning with Instance-Dependent Label Noise: A Sample Sieve Approach (26 citations)
- When Optimizing $f$-Divergence is Robust with Label Noise (6 citations)
Test-time adaptation · 8 papers
- Tent: Fully Test-Time Adaptation by Entropy Minimization (283 citations)
- Understanding the failure modes of out-of-distribution generalization (49 citations)
- Overfitting for Fun and Profit: Instance-Adaptive Data Compression (30 citations)
- BREEDS: Benchmarks for Subpopulation Shift (17 citations)
- Improving Transformation Invariance in Contrastive Representation Learning (11 citations)
- Coping with Label Shift via Distributionally Robust Optimisation (7 citations)
- No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networks (3 citations)
- Systematic generalisation with group invariant predictions
Semi-supervised learning · 6 papers
- PseudoSeg: Designing Pseudo Labels for Semantic Segmentation (205 citations)
- Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data (83 citations)
- In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning (72 citations)
- On Data-Augmentation and Consistency-Based Semi-Supervised Learning (15 citations)
- CO2: Consistent Contrast for Unsupervised Visual Representation Learning (10 citations)
- Semi-supervised Keypoint Localization (4 citations)
Domain adaptation and generalization · 5 papers
- Domain Generalization with MixStyle (41 citations)
- Self-training For Few-shot Transfer Across Extreme Task Differences (17 citations)
- Robust and Generalizable Visual Representation Learning via Random Convolutions (16 citations)
- Integrating Categorical Semantics into Unsupervised Domain Translation (2 citations)
- DINO: A Conditional Energy-Based GAN for Domain Translation
OOD and anomaly detection · 5 papers
- Explainable Deep One-Class Classification (102 citations)
- SSD: A Unified Framework for Self-Supervised Outlier Detection (44 citations)
- In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness (9 citations)
- Multiscale Score Matching for Out-of-Distribution Detection (7 citations)
- Removing Undesirable Feature Contributions Using Out-of-Distribution Data (1 citation)
