Research map / Optimization / Optimizers
Automatic and Harmless Regularization with Constrained and Lexicographic Optimization: A Dynamic Barrier Approach
Chengyue Gong, Xingchao Liu, qiang liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace tuned-lambda L2 regularization with Lexico | EffNet-B0 / AlphaNet-A0 fine-tuning · Oxford Flower (AlphaNet-A0) | top-1 accuracy: 97.7±0.2 -> 97.6±0.1 | No clear effect |
| Replace tuned-lambda L2 regularization with Lexico | EffNet-B0 · Oxford Flower | top-1 accuracy: 97.2±0.1 -> 97.1±0.1 | No clear effect |
| Replace tuned-lambda L2 regularization with Lexico | AlphaNet-A0 · Stanford Car | top-1 accuracy: 91.0±0.1 -> 91.6±0.1 | Improved |
| Replace linearly-combined UDA loss with Lexico | UDA · CIFAR-10 100 labels | test accuracy (%): 75.28±2.39 -> 77.61±1.83 | Improved |
| Replace linearly-combined UDA loss with Lexico | UDA · CIFAR-10 250 labels | test accuracy (%): 91.18±1.08 -> 91.86±0.85 | Improved |
| Replace linearly-combined UDA loss with Lexico | UDA · CIFAR-10 1000 labels | test accuracy (%): 94.56±0.21 -> 94.53±0.15 | No clear effect |
| Replace linearly-combined UDA loss with Lexico | UDA · CIFAR-100 1000 labels | test accuracy (%): 54.85±0.83 -> 56.91±0.78 | Improved |
| Replace linearly-combined UDA loss with Lexico | UDA · CIFAR-100 2500 labels | test accuracy (%): 66.37±0.28 -> 67.06±0.26 | Improved |
| Replace linearly-combined UDA loss with Lexico | UDA · CIFAR-100 10000 labels | test accuracy (%): 67.42±0.22 -> 67.57±0.14 | No clear effect |
| Replace linear combination (best lambda 5) with Lexico | Swin-Base UperNet · Cityscapes val | mIoU: 79.6±0.2 -> 79.7±0.1 | Improved |
| Replace linear combination (best lambda 10) with Lexico | Swin-Base FCN auxiliary · Cityscapes val | mIoU: 78.3±0.2 -> 78.5±0.1 | Improved |
| Dynamic barrier (ours) instead of linear combination for Pareto profiling | sparse representation learning · Toy data, BERT-base NG20/IMDb | Pareto front coverage | Improved |
Closest papers by meaning
- Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models (ICML 2019 · similarity 0.86)
- Iterative Regularization for Learning with Convex Loss Functions (JMLR 2016 · similarity 0.84)
- Near-Optimal Solutions of Constrained Learning Problems (ICLR 2024 · similarity 0.84)
- Improving Deep Learning Optimization through Constrained Parameter Regularization (NeurIPS 2024 · similarity 0.83)
- A Generalization Result for Convergence in Learning-to-Optimize (ICML 2025 · similarity 0.83)
- Learning to Reject with a Fixed Predictor: Application to Decontextualization (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
