Research map / Architectures and efficiency / Attention and transformers
TransCL: Transformer Makes Strong and Flexible Compressive Learning
Chong Mou, Jian Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use single-stage compressive learning instead of reconstruct-then-infer (ablation) | TransCL-32 · ImageNet-1k val (CS ratio 1%) | top-1 accuracy: 55.57 -> 78.00 | Improved |
| use single-stage compressive learning instead of reconstruct-then-infer (ablation) | TransCL-32 · ImageNet-1k val (CS ratio 10%) | top-1 accuracy: 80.65 -> 81.82 | Improved |
| learn the binary sampling matrix instead of float-point (ablation) | TransCL-32 · ImageNet-1k val (CS ratio 1%) | top-1 accuracy: 78.00 -> 76.12 | Hurt |
| learn the binary sampling matrix instead of float-point (ablation) | TransCL-32 · ImageNet-1k val (CS ratio 10%) | top-1 accuracy: 81.82 -> 80.91 | No clear effect |
| learn the binary sampling matrix instead of float-point (ablation) | TransCL-16 · PASCAL Context (CS ratio 1%) | mIoU: 50.83 -> 49.80 | Hurt |
| learn the binary sampling matrix instead of float-point (ablation) | TransCL-16 · PASCAL Context (CS ratio 10%) | mIoU: 55.52 -> 55.47 | No clear effect |
| replace CS measurement with bicubic downsampling and re-train ViT-B-32 (ablation) | ViT-B-32 · ImageNet-1k val (CS ratio 1%) | top-1 accuracy: 78.00 -> 73.91 | Hurt |
| use 32x32 blocks instead of 16x16 (ablation) | TransCL · ImageNet-1k val (CS ratio 1%) | top-1 accuracy: 78.86 -> 78.00 | Mixed |
| use transformer backbone instead of ResNet-152 (ResCL) (ablation) | TransCL · ImageNet-1k val (CS ratio 10%) | top-1 accuracy: 75.22 -> 81.82 | Improved |
| add knowledge distillation data-efficient training (ImageNet-1K only) (ablation) | TransCL-16-1 · ImageNet-1k val (CS ratio 1%) | top-1 accuracy: 78.86 -> 78.58 | No clear effect |
| reduce CS ratio from 10% to 1% (ablation) | TransCL-16 · ImageNet-1k val | top-1 accuracy: 83.86 -> 78.86 | Hurt |
| reduce CS ratio from 10% to 1% (ablation) | TransCL-16 · ADE20K | mIoU: 50.00 -> 46.57 | Hurt |
Closest papers by meaning
- ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing (CVPR 2018 · similarity 0.87)
- CPP-Net: Embracing Multi-Scale Feature Fusion into Deep Unfolding CP-PPA Network for Compressive Sensing (CVPR 2024 · similarity 0.84)
- Multi-Cross Sampling and Frequency-Division Reconstruction for Image Compressed Sensing (AAAI 2024 · similarity 0.84)
- Block Image Compressive Sensing with Local and Global Information Interaction (AAAI 2024 · similarity 0.84)
- Patch Rebirth: Fast and Transferable Model Inversion of Vision Transformers (NeurIPS 2026 · similarity 0.84)
- SparseFormer: Sparse Visual Recognition via Limited Latent Tokens (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
