Research map / Architectures and efficiency / CNN design
FTL: A universal framework for training low-bit DNNs via Feature Transfer
Kunyuan Du, Ya Zhang, Haibing Guan, Qi Tian, Shenggan Cheng, James Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use centered cosine distance in continuous space (ablation) | FTL (BNN) · CIFAR-10 (BNN) | accuracy gain: +0.84% | Improved |
| use L1 norm in continuous space instead of centered cosine distance (ablation) | FTL (BNN) · CIFAR-10 (BNN) | accuracy gain: +0.45% | Improved |
| apply L1 norm in discrete space (quantize 32-bit features first) (ablation) | FTL (BNN) · CIFAR-10 (BNN) | accuracy gain: -0.07% | No clear effect |
| gradient rescaling with intermediate w (0.002) instead of teacher-student w=1e-5 (ablation) | FTL (BNN) · CIFAR-10 (VGG-small 0.5x) | accuracy gain: -0.04% -> +0.29% | Improved |
| mutual learning mode (w=0.5) instead of gradient rescaling with small w (ablation) | FTL (BNN) · CIFAR-10 (VGG-small 0.5x) | accuracy gain: +0.29% -> -0.61% | Hurt |
| add FTL to fine-tuning from pretrained 32-bit DNN (ablation) | FTL (BNN) · CIFAR-10 (VGG-small) | Validation accuracy: 89.31 -> 89.62 | Improved |
| add FTL to logits transfer (ablation) | FTL (BNN) · CIFAR-10 (VGG-small) | Validation accuracy: 89.49 -> 89.96 | Improved |
| replace JGT feature guidance with FTL | FTL (BNN) · CIFAR-10 (BNN ResNet-small) | Top-1 accuracy: 88.89 -> 90.88 | Improved |
| replace JGT feature guidance with FTL | FTL (BNN) · CIFAR-10 (BNN VGG-small) | Top-1 accuracy: 89.10 -> 89.82 | Improved |
| replace JGT* with FTL | DoReFa-Net AlexNet · ImageNet (DoReFa AlexNet) | Top-1 accuracy: 51.6 -> 54.0 | Improved |
| add FTL with 4-bit quantized gradients | DoReFa-Net AlexNet · ImageNet (DoReFa AlexNet 1/2/4) | Top-1 accuracy: 41.5 -> 44.9 | Improved |
| add FTL to LQ-Net | LQ-Net ResNet-50 · ImageNet (ResNet-50 1/2/32) | Top-1 accuracy: 68.7 -> 69.6 | Improved |
Closest papers by meaning
- Explicit Loss-Error-Aware Quantization for Low-Bit Deep Neural Networks (CVPR 2018 · similarity 0.85)
- Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks (ICML 2020 · similarity 0.85)
- Big Transfer (BiT): General Visual Representation Learning (ECCV 2020 · similarity 0.84)
- Effective Training of Convolutional Neural Networks With Low-Bitwidth Weights and Activations (TPAMI 2021 · similarity 0.84)
- ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization (NeurIPS 2025 · similarity 0.83)
- Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
