Research map / Label-efficient and robust learning / Self-supervised and contrastive
Maximizing Incremental Information Entropy for Contrastive Learning
Jiansong Zhang, Zhuoqin Yang, Xu Wu, Xiaoling Luo, Peizhong Liu, Linlin Shen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SAIB entropy-generation module (ablation) | MoCo-v2 ResNet-18 · ImageNet-1k | top-1 accuracy (%): 52.50 -> 58.80 | Improved |
| add KL regularization (ablation) | MoCo-v2 + SAIB · ImageNet-1k | top-1 accuracy (%): 58.80 -> 59.15 | Improved |
| add spectral-norm encoder regularizer (ablation) | MoCo-v2 + SAIB + KL · ImageNet-1k | top-1 accuracy (%): 59.15 -> 59.41 | Improved |
| cascade 2 or 3 SAIB modules instead of 1 (ablation) | IE-CL · ImageNet-1k | top-1 accuracy (%): 59.41 -> 58.62 (2x), 58.71 (3x) | Hurt |
| add SAIB to MoCo-v2 (ablation) | MoCo-v2 · ImageNet-100 | top-1 accuracy (%): 66.29 -> 67.54 | Improved |
| add SAIB to BYOL (ablation) | BYOL · ImageNet-100 | top-1 accuracy (%): 67.95 -> 68.76 | Improved |
| add SAIB to SimCLR (ablation) | SimCLR · ImageNet-100 | top-1 accuracy (%): 63.34 -> 64.02 | Improved |
| add SAIB to SimSiam (ablation) | SimSiam · ImageNet-100 | top-1 accuracy (%): 66.25 -> 66.97 | Improved |
| use IE-CL instead of Matrix-SSL at 100 epochs | ResNet-50 pretraining · ImageNet linear eval | top-1 accuracy (%): 69.2 -> 68.3 | Hurt |
| use IE-CL for 800 epochs | ResNet-50 pretraining · ImageNet linear eval | top-1 accuracy (%): 71.9 -> 73.2 | Improved |
Closest papers by meaning
- Contrastive Learning via Variational Information Bottleneck (TPAMI 2025 · similarity 0.88)
- Your contrastive learning problem is secretly a distribution alignment problem (NeurIPS 2024 · similarity 0.88)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.88)
- A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics (NeurIPS 2025 · similarity 0.88)
- Separating common from salient patterns with Contrastive Representation Learning (ICLR 2024 · similarity 0.87)
- Multi-modal contrastive learning adapts to intrinsic dimensions of shared latent variables (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
