Research map / Vision-language and multimodal / CLIP and zero-shot
GaussianVision: Vision-Language Alignment from Compressed Image Representations using 2D Gaussian Splatting
Yasmine Omri, Connor Ding, Tsachy Weissman, Thierry Tambe
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use structured grid initialization instead of random for 2DGS fitting (4900 Gaussians) (ablation) | 2DGS fitting · 3000 iterations | PSNR: 28.24 -> 35.25 | Improved |
| use structured initialization instead of random (400 Gaussians) (ablation) | 2DGS fitting · 3000 iterations | PSNR: 17.77 -> 22.04 | Improved |
| batch-aware CUDA kernels | 2DGS fitting · batch 4096, 400 Gaussians | fitting speedup: 90.3X | Improved |
| reduce Gaussian budget from 3136 to 400 splats | GS CLIP encoder · CLIP benchmark 38 datasets | relative zero-shot accuracy: 0.98 -> 0.91 | Hurt |
| halve tokens from 196 to 98 for GS encoders (ablation) | GS CLIP encoder · CLIP benchmark 38 datasets | relative accuracy: 0.98 -> 0.96 (3136 splats) | No clear effect |
| halve tokens from 196 to 98 for RGB by averaging patch pairs | RGB ViT-B/16 · CLIP benchmark 38 datasets | relative accuracy: 1.00 -> 0.87 | Hurt |
| replace RGB encoder with GS-3136 in LLaVA VLM | LLaVA Vicuna-7B · six VQA benchmarks | VQA score: MM-Vet 16.60 -> 19.40; POPE 77.83 -> 77.80 | Mixed |
| replace RGB encoder with GS-400 in LLaVA VLM | LLaVA Vicuna-7B · six VQA benchmarks | VQA score: SQA-IMG 63.81 -> 64.80; GQA 51.42 -> 50.21 | Mixed |
| use prune with larger initial Gaussian budget (ablation) | 2DGS pruning · Mini-ImageNet 224x224 | PSNR after pruning | Improved |
| train CLIP from raw GS inputs directly instead of two-stage distillation (ablation) | GS CLIP encoder · CLIP training | convergence | Hurt |
Closest papers by meaning
- Kernel-based Unsupervised Embedding Alignment for Enhanced Visual Representation in Vision-language Models (ICML 2025 · similarity 0.89)
- Tackling View-Dependent Semantics in 3D Language Gaussian Splatting (ICML 2025 · similarity 0.89)
- DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment (CVPR 2025 · similarity 0.88)
- Perception Encoder: The best visual embeddings are not at the output of the network (NeurIPS 2025 · similarity 0.88)
- Unifying 2D and 3D Vision-Language Understanding (ICML 2025 · similarity 0.88)
- Harnessing Frozen Unimodal Encoders for Flexible Multimodal Alignment (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
