Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning

Learning to Learn with Contrastive Meta-Objective

NeurIPS 2025 · Label-efficient and robust learning › Zero-/few-shot and meta-learning · approach: Meta-augmentation and meta-objectives · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ConML, a learner-agnostic contrastive meta-objective that uses task identity in mini-batch episodic meta-training to align models from the same task and discriminate models from different tasks in model space.

Problem

Meta-learners do not exploit the task identity available in mini-batch episodic training as supervision, which could improve generalization to unseen tasks.

Approach

Extends contrastive learning from representation space to model space: inner-task distance (alignment between models trained on subsets of the same task) minus inter-task distance (discrimination between models of different tasks in a batch), added to the episodic loss, with per-meta-learner mappings from models to vectors (weights, prototypes, hypernetwork outputs, or ICL probe outputs).

Key results

• On miniImageNet 5-way 1-shot, MAML improves from 48.75% to 56.25% and ProtoNet from 48.90% to 51.03% with ConML, using one shared configuration (B=32, K=1, lambda=0.1).
• Across 10 meta-learner/setting combinations in the mini/tieredImageNet table, ConML improves every reported accuracy, e.g. CAML 96.15% to 97.03% on miniImageNet 5-way 1-shot.
• In ICL on synthetic function classes, ConML reduces relative minimal error to 0.42-0.81 of baseline ICL, with shot savings of 3.9-11.3 examples.

Measured effects reported in the paper

Surprising findings

• Euclidean distance works much better than cosine for ProtoNet, while cosine works generally for other learners.
• The paper reports that the naive contrastive loss can cause model collapse when lambda is set too high.

Limitations

• Memory cost grows linearly with K, roughly 2x memory versus the baseline at K=1.
• The theoretical guarantee relies on strong assumptions such as unit-norm model representations and a specific distance choice, and the paper notes the squared-cosine form was not implemented in experiments.
• Few-shot experiments for some baselines lack error bars, and compute resources are not reported in detail.

Experimental setup

Datasets: miniImageNet tieredImageNet Meta-Dataset

Benchmarks: miniImageNet tieredImageNet Meta-Dataset

Metrics: accuracy relative minimal error shot spare memory relative time

Baselines: MAML FOMAML Reptile MatchNet ProtoNet SCNAPs CAML MELR LastShot P>M>F

Builds on: MAML ProtoNet CAML Simple CNAPS InfoNCE

Closest papers by meaning

• The Close Relationship Between Contrastive Learning and Meta-Learning (ICLR 2022 · similarity 0.91)
• Function Contrastive Learning of Transferable Meta-Representations (ICML 2021 · similarity 0.88)
• Meta-Learning Requires Meta-Augmentation (NeurIPS 2020 · similarity 0.88)
• Any-Way Meta Learning (AAAI 2024 · similarity 0.88)
• Provable Meta-Learning with Low-Rank Adaptations (NeurIPS 2025 · similarity 0.87)
• Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models (ICML 2024 · similarity 0.87)