Research map / Label-efficient and robust learning / Recognition and long-tail
Threshold-Consistent Margin Loss for Open-World Deep Metric Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the OPIS variance-based metric for threshold inconsistency in deep metric learning and a Threshold-Consistent Margin (TCM) regularizer that penalizes hard positive/negative pairs outside cosine margins to improve threshold consistency while preserving accuracy.
Problem
Deep metric learning embeddings show non-uniform intra-/inter-class structure, so a single fixed distance threshold yields inconsistent FAR/FRR across test classes and data distributions (threshold inconsistency), and high accuracy does not guarantee consistency.
Approach
A variance-based OPIS metric measures utility-curve dispersion across test classes within a calibration FAR range without a hold-out set; TCM is a regularizer added to any base DML loss that applies hinge-style penalties to hard positive and hard negative pairs beyond two cosine margins, averaged over hard counts.
Key results
- On iNaturalist-2018 with ResNet50 (512-d), RS@k+TCM improves R@1 from 83.9 to 84.8 with ViT-B/16 and reduces OPIS from 0.37e-3 to 0.17e-3 (up to 77.3% OPIS reduction across settings).
- TCM improves R@1 by more than 1.0% across base losses (e.g., ProxyNCA +1.4, ArcFace +1.0, SAP +1.7, RS@k +1.5 on ResNet50) while lowering OPIS by up to 60.7% relative.
- TCM adds under 1.5% to per-epoch training time at batch size 392 on Cars-196 with ViT-B/16.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add TCM to Recall@k Surrogate | ViT-B/16 with Recall@k Surrogate · iNaturalist-2018 | R@1: 83.9 to 84.8 | Helped |
| Add TCM to Recall@k Surrogate | ViT-B/16 with Recall@k Surrogate · iNaturalist-2018 | OPIS: 0.37× 10^{-3} to 0.17× 10^{-3} | Helped |
| Use only the positive TCM term (ablation) | ViT-B/16 with Recall@k Surrogate · iNaturalist-2018 | R@1: 83.9 to 79.0 | Hurt |
| Use only the positive TCM term (ablation) | ViT-B/16 with Recall@k Surrogate · iNaturalist-2018 | OPIS: 0.37 to 0.06 | Helped |
| Use only the negative TCM term (ablation) | ViT-B/16 with Recall@k Surrogate · iNaturalist-2018 | R@1: 83.9 to 72.5 | Hurt |
| Use only the negative TCM term (ablation) | ViT-B/16 with Recall@k Surrogate · iNaturalist-2018 | OPIS: 0.37 to 0.14 | Helped |
| Remove the indicator functions from TCM (ablation) | ViT-B/16 with Recall@k Surrogate · iNaturalist-2018 | R@1: 83.9 to 78.1 | Hurt |
| Remove the indicator functions from TCM (ablation) | ViT-B/16 with Recall@k Surrogate · iNaturalist-2018 | OPIS: 0.37 to 0.31 | No clear effect |
| Add TCM to Smooth-AP | ResNet50 with Smooth-AP · iNaturalist-2018 | R@1: +1.7 | Helped |
| Add TCM to Smooth-AP | ResNet50 with Smooth-AP · iNaturalist-2018 | OPIS: 0.330 to 0.168 | Helped |
| Add contrastive loss as a regularizer instead of TCM | ResNet50 with Smooth-AP · iNaturalist-2018 | R@1: 67.4 to 67.1 | Hurt |
| Add contrastive loss as a regularizer instead of TCM | ResNet50 with Smooth-AP · iNaturalist-2018 | OPIS: 0.330 to 0.10 | Helped |
Surprising findings
- High accuracy in the high-accuracy regime comes with a Pareto trade-off where accuracy improvements tend to increase threshold inconsistency for existing DML methods.
- Using the contrastive loss as regularizer gives the lowest OPIS but reduces R@1, whereas TCM improves both.
Limitations
- OPIS requires a sufficient number of samples per class, so it is unsuitable for few-shot evaluation.
- TCM can fail under significant train-test distribution shift or strong label noise.
- Margin hyperparameters (m+, m-) require grid search and cannot be directly estimated from the calibration range.
Experimental setup
Datasets: iNaturalist-2018 Stanford Online Product CUB-200-2011 Cars-196 MNIST
Benchmarks: iNaturalist-2018 Stanford Online Product CUB-200-2011 Cars-196
Metrics: Recall@k OPIS epsilon-OPIS (10%-OPIS)
Baselines: Smooth-AP Recall@k Surrogate ProxyNCA ArcFace ProxyAnchor ROADMAP Triplet loss Contrastive loss
Builds on: Smooth-AP Recall@k Surrogate ProxyAnchor ArcFace One-Threshold-for-All (OTA)
Closest papers by meaning
- Stabilizing Sample Similarity in Representation via Mitigating Random Consistency (ICML 2025 · similarity 0.85)
- Towards Improved Proxy-Based Deep Metric Learning via Data-Augmented Domain Adaptation (AAAI 2024 · similarity 0.85)
- Detecting Brittle Decisions for Free: Leveraging Margin Consistency in Deep Robust Classifiers (NeurIPS 2024 · similarity 0.85)
- Collapse-Aware Triplet Decoupling for Adversarially Robust Image Retrieval (ICML 2024 · similarity 0.85)
- Deep Disentangled Metric Learning (AAAI 2025 · similarity 0.85)
- Mean Field Theory in Deep Metric Learning (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.