Research map / Learning theory and algorithms / Prediction and decision losses
Relational Surrogate Loss Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace cross-entropy with ReLoss | ResNet-56 · CIFAR-10 test | Top-1 accuracy: 94.32 ± 0.25 -> 94.57 ± 0.08 | Helped |
| Replace cross-entropy with ReLoss | ResNet-56 · CIFAR-100 test | Top-1 accuracy: 73.61 ± 0.11 -> 74.15 ± 0.14 | Helped |
| Add ReLoss | DARK with HRNet-W48 · COCO test-dev | AP: 76.2 -> 76.4 | Helped |
| Add ReLoss | MacBERT-large · DuReader 2.0 test | ROUGE-L: 50.8 -> 64.9 | Helped |
| Add ReLoss to NAT-Base | NAT-Base · WMT16 EN-RO | BLEU: +0.83 BLEU | Helped |
| Optimize surrogate loss by metric rank correlation instead of output approximation (ablation) | ReLoss on image classification · CIFAR-10 | Top-1 accuracy: 94.11 ± 0.42 -> 94.57 ± 0.08 | Helped |
| Optimize surrogate loss by metric rank correlation instead of output approximation (ablation) | ReLoss on image classification · CIFAR-100 | Top-1 accuracy: 73.88 ± 0.32 -> 74.15 ± 0.14 | Helped |
| Add gradient penalty during ReLoss learning (ablation) | ReLoss on ResNet-56 · CIFAR-100 | Top-1 accuracy: 73.03 ± 0.26 -> 74.15 ± 0.14 | Helped |
| Add the regular loss alongside ReLoss (ablation) | ResNet-56 · CIFAR-10 | Top-1 accuracy: 93.82 ± 0.32 -> 94.57 ± 0.08 | Helped |
| Train ReLoss alternately with the prediction model (ablation) | ResNet-56 · CIFAR-10 | Top-1 accuracy: 94.57 ± 0.08 -> 94.65 ± 0.21 | Mixed |
| Train ReLoss alternately with the prediction model (ablation) | ResNet-56 · CIFAR-100 | Top-1 accuracy: 74.15 ± 0.14 -> 74.18 ± 0.31 | Mixed |
| Transfer a ReLoss learned on ImageNet rather than learn one on the target dataset (ablation) | Image classification models · CIFAR-10 | Top-1 accuracy: 94.57 ± 0.08 -> 94.61 ± 0.12 | No clear effect |
Closest papers by meaning
- Auto Seg-Loss: Searching Metric Surrogates for Semantic Segmentation (ICLR 2021 · similarity 0.86)
- Adversarial Surrogate Losses for Ordinal Regression (NeurIPS 2017 · similarity 0.85)
- SoDeep: A Sorting Deep Net to Learn Ranking Loss Surrogates (CVPR 2019 · similarity 0.85)
- Multi-Similarity Loss With General Pair Weighting for Deep Metric Learning (CVPR 2019 · similarity 0.85)
- Addressing the Loss-Metric Mismatch with Adaptive Loss Alignment (ICML 2019 · similarity 0.85)
- Trading off Consistency and Dimensionality of Convex Surrogates for Multiclass Classification (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.