Research map / Architectures and efficiency / Pruning and efficient tuning
Local Scale Equivariance with Latent Deep Equilibrium Canonicalizer
Md Ashiqur Rahman, Chiao-An Yang, Michael N. Cheng, Lim Jun Hao, Jeremiah Jiang, Teck-Yian Lim, Raymond A. Yeh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DEC latent canonicalizer instead of augmentation plus equivariance loss (AugL) | ViT/Swin/DINOv2 segmentation · Locally scaled segmentation | mIoU: AugL 79.05 -> 80.86 on DINOv2 | Improved |
| canonicalize the input image (Canon) instead of latent features (ablation) | Aug-trained models · Locally scaled segmentation (ViT) | mIoU: 82.18 -> 79.37 | Hurt |
| input-level Canon on locally scaled ImageNet (ablation) | Base models · Locally scaled ImageNet (Swin) | Top-1 accuracy: 77.94 -> 74.67 | Hurt |
| add monotone scale equivariance loss (AugL) on top of Aug (ablation) | ViT segmentation · Locally scaled segmentation | mIoU: 82.18 -> 82.35 | No clear effect |
| add DEC on top of Aug for MNIST classification | Swin (MNIST) · Locally scaled MNIST overall | Accuracy: 93.93 -> 96.53 | Improved |
| add DEC to pretrained models for locally scaled ImageNet | ViT, DeiT, Swin, BEiT · Locally scaled ImageNet | Top-1 accuracy: 80.15 -> 80.36 (ViT); 77.94 -> 78.32 (Swin) | Improved |
| add an invariance loss (InvL) to Aug on ImageNet (ablation) | ViT, Swin · Locally scaled ImageNet | Top-1 accuracy: 80.15 -> 80.16 | No clear effect |
| DEC amortized fixed-point solver instead of differentiable optimization canonicalization (ablation) | DEC · Training batch 10, 224x224 | GPU memory: 43.30 GB -> 5.75 GB | Improved |
| apply several DEC modules throughout the network instead of only at the input (ablation) | DEC · Scale-MNIST | Accuracy | Improved |
| add DEC to hierarchical models | HRViT, ResFormer · Scale-MNIST | Accuracy: ResFormer Inv-loss 94.92 -> 96.91 | Improved |
Closest papers by meaning
- Truly Scale-Equivariant Deep Nets with Fourier Layers (NeurIPS 2023 · similarity 0.88)
- Equivariance with Learned Canonicalization Functions (ICML 2023 · similarity 0.87)
- Relaxing Equivariance Constraints with Non-stationary Continuous Filters (NeurIPS 2022 · similarity 0.87)
- A Probabilistic Approach to Learning the Degree of Equivariance in Steerable CNNs (ICML 2024 · similarity 0.85)
- Learning (Approximately) Equivariant Networks via Constrained Optimization (NeurIPS 2025 · similarity 0.85)
- A Canonicalization Perspective on Invariant and Equivariant Learning (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
