Research map / Vision-language and multimodal / Multimodal LLMs
Self-Adapting Large Visual-Language Models to Edge Devices across Visual Modalities
Kaiwen Cai, ZheKai Duan, Gaowen Liu, Charles Fleming, Chris Xiaoxuan Lu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add contrastive learning to quantization-aware training (stage-2) instead of plain QAT (ablation) | EdgeVL DAT-T · ScanNet | avg top-1 accuracy (%): 40.3 -> 50.0 | Improved |
| apply post-training quantization to the stage-1 student (ablation) | EdgeVL ViT-S · ScanNet | avg top-1 accuracy (%): 39.3 -> 25.3 | Hurt |
| train a dual-modality student instead of single-modality CMKD (non-RGB) (ablation) | EdgeVL stage-1 · ScanNet | avg top-1 accuracy (%): 24.6 -> 39.6 | Improved |
| use RGB images as extra training signal for the non-RGB branch (ablation) | EdgeVL stage-1 · ScanNet | non-RGB accuracy (%): 37.8 -> 38.6 | Improved |
| set automatic curation threshold tau_c to 0.25 instead of 0.10 (ablation) | EdgeVL Swin-T · EuroSAT | avg top-1 accuracy (%): 51.4 -> 64.2 | Improved |
| raise tau_c from 0.25 to 0.50 (ablation) | EdgeVL Swin-T · EuroSAT | avg top-1 accuracy (%): 64.2 -> 62.0 | Hurt |
| use semi-hard instead of hard triplet sampling (ablation) | EdgeVL DAT-T · EuroSAT | avg top-1 accuracy (%): 62.7 -> 63.3 | Improved |
| use two-stage training instead of one-stage with PTQ (ablation) | EdgeVL DAT-T · ScanNet | avg top-1 accuracy (%): 30.0 -> 49.9 | Improved |
| use EdgeVL ViT-S Int8 instead of CLIP-B on Jetson AGX | edge inference · Jetson AGX Orin | latency (ms): 9.5 -> 4.6 | Improved |
| adapt on ScanNet and test on NYU2 RGB images vs pretrained CLIP-G | EdgeVL DAT-T · NYU2 | RGB accuracy (%): 69.7 -> 54.3 | Hurt |
| adapt on ScanNet and test depth images on NYU2 vs pretrained CLIP-G | EdgeVL DAT-T · NYU2 | non-RGB accuracy (%): 25.7 -> 51.1 | Improved |
Closest papers by meaning
- From Pixels to Words -- Towards Native Vision-Language Primitives at Scale (ICLR 2026 · similarity 0.89)
- CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.89)
- InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks (CVPR 2024 · similarity 0.88)
- Towards Better Vision-Inspired Vision-Language Models (CVPR 2024 · similarity 0.88)
- LION: Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge (CVPR 2024 · similarity 0.88)
- VladVA: Discriminative Fine-tuning of LVLMs (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
