Research map / Image generation / Autoregressive and latent generation
The Prism Hypothesis: Harmonizing Semantic and Pixel Representations via Unified Autoencoding
Weichen Fan, Haiwen Diao, Quan Wang, Dahua Lin, Ziwei Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add semantic alignment loss on 25% of low-frequency tokens (ablation) | UAE (DINOv2) · ImageNet-1K linear probing | accuracy: 60.1 -> 83.0 | Improved |
| add semantic alignment loss on 25% of low-frequency tokens (ablation) | UAE (DINOv2) · ImageNet-1K reconstruction | PSNR: 34.3 -> 32.2 | Hurt |
| increase semantic-loss fraction K_base from 25% to 75% (ablation) | UAE (DINOv2) · ImageNet-1K reconstruction | PSNR: 32.2 -> 20.1 | Hurt |
| mask 85% instead of 25% of high-frequency tokens (ablation) | UAE · latent generation | gFID@20 epoch: 33.1 -> 9.7 | Improved |
| mask 15% instead of 25% of high-frequency tokens (ablation) | UAE · latent generation | gFID@20 epoch: 33.1 -> 12.2 | Improved |
| keep only 64 low-frequency tokens instead of 121 (ablation) | UAE latent diffusion · latent generation at 20 epochs | FID: 40.5 -> 9.7 | Improved |
| keep only 64 low-frequency tokens when training from scratch (ablation) | encoder-decoder from scratch · linear probing | accuracy: 20.7 -> 60.1 | Improved |
| apply unified training to RAE-style DINOv2-B tokenizer | DINOv2-B tokenizer · ImageNet-1K reconstruction | PSNR: 18.05 -> 31.00 | Improved |
| apply unified training to RAE-style DINOv2-B tokenizer | DINOv2-B tokenizer · MS-COCO 2017 reconstruction | rFID: 6.01 -> 2.01 | Improved |
| apply UAE unified training to JIT | JIT 700M, 80 epochs · pixel-space ImageNet generation | FID: 16.06 -> 14.55 | Improved |
| apply UAE unified training to JIT | JIT 700M, 40 epochs · pixel-space ImageNet generation | IS: 60.13 -> 72.82 | Improved |
| generate with 64 UAE low-frequency tokens instead of RAE latents in SiT | SiT · inference cost | GFLOPs: 313.84 -> 78.80 | Improved |
Closest papers by meaning
- Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders (NeurIPS 2026 · similarity 0.87)
- From Pixels to Words -- Towards Native Vision-Language Primitives at Scale (ICLR 2026 · similarity 0.86)
- Large Vision-Language Models Get Lost in Attention (ICML 2026 · similarity 0.86)
- Do Vision and Language Encoders Represent the World Similarly? (CVPR 2024 · similarity 0.86)
- Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models (CVPR 2025 · similarity 0.86)
- UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
