Research map / Label-efficient and robust learning / Self-supervised and contrastive
CompRess: Self-Supervised Learning by Compressing Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compress via similarity distribution (Ours-1q) instead of Reg-BN baseline | MoCo R50 to ResNet-18 · ImageNet linear eval | top-1: 58.2 -> 62.6 | Helped |
| use ResNet-50x4 SimCLR teacher compressed to AlexNet | AlexNet · ImageNet linear eval | top-1: 56.5 -> 59.0 | Helped |
| remove SoftMax, temperature and memory bank (ablation) | CompRess MoCo R50 to ResNet-18 · ImageNet linear eval | top-1: 53.6 -> 61.7 | Helped |
| use temperature 0.04 instead of tau=1 (ablation) | CompRess Ours-2q · 25% ImageNet, R18 | NN accuracy | Helped |
| larger memory bank of anchor points (ablation) | CompRess Ours-2q · 25% ImageNet, R18 | NN accuracy | Helped |
| vary momentum from 0.999 to 0 for key encoder (ablation) | Ours-2q · 25% ImageNet, R18 | NN accuracy: 47.35% -> 47.34% | No clear effect |
| cache teacher features (no teacher augmentation) (ablation) | R50 to ResNet-18 · ImageNet | NN accuracy: 53.4% -> 53.0% | Hurt |
| use teacher's memory bank for student (1q vs 2q) (ablation) | CompRess · ImageNet linear eval ResNet-18 | top-1: 61.7 -> 62.6 | Helped |
| use separate student memory bank (2q vs 1q) (ablation) | CompRess · CUB200 retrieval R50x4 teacher | R@1: 21.9 -> 26.5 | Helped |
| use 2q instead of 1q (ablation) | CompRess · ImageNet cluster alignment AlexNet | CA: 27.0 -> 33.3 | Helped |
Closest papers by meaning
- Self-Supervised Generative Adversarial Compression (NeurIPS 2020 · similarity 0.89)
- On Compressing Deep Models by Low Rank and Sparse Decomposition (CVPR 2017 · similarity 0.88)
- SEED: Self-supervised Distillation For Visual Representation (ICLR 2021 · similarity 0.88)
- EPSD: Early Pruning with Self-Distillation for Efficient Model Compression (AAAI 2024 · similarity 0.87)
- Language Modeling Is Compression (ICLR 2024 · similarity 0.86)
- Boost Self-Supervised Dataset Distillation via Parameterization, Predefined Augmentation, and Approximation (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.