Research map / Architectures and efficiency / Quantization
LLaVA-FA: Learning Fourier Approximation for Compressing Large Multimodal Models
Pengcheng Zheng, Chaoning Zhang, Jiarong Mo, GuoHui Li, Jiaquan Zhang, Jiahao Zhang, Sihan Cao, Sheng Zheng, Caiyan Qin, Guoqing Wang, Yang Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| raise LoRA rank from 64 to 256 (ablation) | LLaVA-FA-2B · Multimodal suite average | average accuracy: 58.9 -> 60.8 (+1.9%) | Improved |
| raise LoRA rank from 64 to 256 (ablation) | LLaVA-FA-1B · Multimodal suite average | average accuracy: 55.3 -> 56.6 (+1.3%) | Improved |
| raise amplitude and phase bit-widths from 2 to 4 (ablation) | LLaVA-FA-2B · Multimodal suite average | average accuracy: 56.3 -> 60.9 | Improved |
| raise amplitude and phase bit-widths from 2 to 4 (ablation) | LLaVA-FA-2B · Object HalBench | hallucination rate: 18.7 -> 11.2 | Improved |
| raise bit-widths from 4 to 6 each (ablation) | LLaVA-FA-2B · Multimodal suite average | average accuracy: 60.9 -> 61.2 | No clear effect |
| raise bit-widths from 4 to 6 each (compression cost) (ablation) | LLaVA-FA-2B · compression ratio | compression ratio: 11.2 -> 8.9 | Hurt |
| use Fourier-domain complex low-rank approximation instead of spatial real (ablation) | Qwen-2.5-7B query layers · Spectrum-decay experiment | Frobenius error at rank 256: 0.031 -> 0.018 | Improved |
| use Fourier-domain complex low-rank approximation instead of spatial real (ablation) | Qwen-2.5-7B query layers · Spectrum-decay experiment | min rank for 1% error: 312 -> 217 | Improved |
| compress with Fourier approximation versus Imp-2B, FLOPs and TTFT | Imp-2B · 576 image tokens | FLOPs (T): 2.14 -> 1.93 | Improved |
| compress with Fourier approximation versus Imp-2B, latency | Imp-2B · 576 image tokens | latency (ms): 36.2 -> 30.6 | Improved |
| compare LLaVA-FA-2B with the larger LLaVA-1.5-7B on average accuracy | LLaVA-FA-2B · Multimodal suite average | average accuracy: 62.1 vs 60.8 | Mixed |
| compare LLaVA-FA-1B with SPHINX-Tiny using 5M instead of 15M samples | SPHINX-Tiny · Multimodal suite average | average accuracy: 49.4 vs 49.2 | No clear effect |
Closest papers by meaning
- LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token (ICLR 2025 · similarity 0.89)
- CASP: Compression of Large Multimodal Models Based on Attention Sparsity (CVPR 2025 · similarity 0.88)
- iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Models (ICLR 2026 · similarity 0.86)
- Compressing Large Language Models using Low Rank and Low Precision Decomposition (NeurIPS 2024 · similarity 0.86)
- LLaVA-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models (ICLR 2025 · similarity 0.86)
- With Limited Data for Multimodal Alignment, Let the STRUCTURE Guide You (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
