Research map / Vision-language and multimodal / Human and embodied modeling
Learning Visual Prior via Generative Pre-Training
Jinheng Xie, Kai Ye, Yudong Li, Yuexiang Li, Kevin Qinghong Lin, Yefeng Zheng, Linlin Shen, Mike Zheng Shou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add special words to coordinate tokens (ablation) | VISORGPT · COCO | KL divergence: 1.720 -> 1.647 | Improved |
| add special words to coordinate tokens (ablation) | VISORGPT · Open Images | KL divergence: 1.950 -> 1.895 | Improved |
| initialize from pretrained GPT-2 text weights (ablation) | VISORGPT · COCO | KL divergence: 1.720 -> 1.959 | Hurt |
| initialize from pretrained GPT-2 text weights (ablation) | VISORGPT · Open Images | KL divergence: 1.950 -> 2.240 | Hurt |
| increase sampled sequences from ~40 to ~120 (ablation) | VISORGPT · COCO | KL divergence: 1.958 -> 0.930 | Improved |
| increase sampled sequences from ~40 to ~120 (ablation) | VISORGPT · Open Images | KL divergence: 3.247 -> 2.144 | Improved |
| scale model from 117M to 762M parameters (ablation) | VISORGPT · COCO (box) | KL divergence: 0.850 -> 0.798 | Improved |
| scale model from 117M to 345M parameters (ablation) | VISORGPT · COCO (box) | KL divergence: 0.850 -> 0.836 | Improved |
| use both prompt templates Ta+Tb instead of Ta (full corpus) (ablation) | VISORGPT · COCO | KL divergence location: 1.212 -> 1.583 | Hurt |
| use both prompt templates Ta+Tb instead of Ta (full corpus) (ablation) | VISORGPT · COCO | KL divergence shape: 1.813 -> 1.710 | Improved |
| use both prompt templates Ta+Tb instead of Ta (small corpus) (ablation) | VISORGPT-dagger · COCO | KL divergence location: 1.133 -> 1.032 | Improved |
| use both prompt templates Ta+Tb instead of Ta (small corpus) (ablation) | VISORGPT-dagger · COCO | KL divergence relation: 0.452 -> 0.445 | Improved |
Closest papers by meaning
- Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training (ICLR 2026 · similarity 0.90)
- In-Context Learning Can Help Vision Language Models Overcome Training Prior (NeurIPS 2026 · similarity 0.88)
- Revisiting the Role of Language Priors in Vision-Language Models (ICML 2024 · similarity 0.87)
- Learning Visual Generative Priors without Text (CVPR 2025 · similarity 0.87)
- VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models (CVPR 2026 · similarity 0.87)
- Make Prompts Adaptable: Bayesian Modeling for Vision-Language Prompt Learning with Data-Dependent Prior (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
