Research map / Image generation / Autoregressive and latent generation
Twins: Learn to Predict Unified Representations with Focal Loss
Kaixiong Gong, Xin Cai, Bin Lin, Hao Wang, Yunlong Lin, Mingzhe Zheng, Bohao Li, Jian-Wei Zhang, Miles Yang, Zhao Zhong, Liefeng Bo, Xiangyu Yue
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace MSE with focal-weighted flow matching loss on VAE channels (ablation) | Twins DiT, VAE-decoded · ImageNet 256 gFID w/o guidance (80 epochs) | gFID: 14.41 -> 3.84 | Improved |
| replace MSE with focal loss (ablation) | Twins DiT, VAE-decoded · ImageNet 256 gFID w/o guidance (20 epochs) | gFID: 23.69 -> 7.38 | Improved |
| replace MSE with focal loss (ablation) | Twins DiT, SigLIP2-decoded · ImageNet 256 gFID w/o guidance (80 epochs) | gFID: 2.86 -> 2.50 | Improved |
| replace MSE with focal loss (ablation) | Twins DiT · ImageNet 512 gFID w/o guidance (80 epochs) | gFID: 6.80 -> 3.78 | Improved |
| jointly model ViT and VAE channels with plain MSE instead of VAE only (ablation) | DiT on Twins · ImageNet 256 gFID w/o guidance (80 epochs) | gFID: 3.99 -> 14.41 | Hurt |
| concatenate Flux.2 VAE latents to SigLIP2 features for the VLM encoder (ablation) | Qwen2.5-7B VLM · GQA, TQA, MME-S | accuracy / score: GQA 64.54 -> 64.93; TQA 56.92 -> 58.89; MME-S 1826.8 -> 1971.0 | Improved |
| concatenate Flux.2 VAE latents to SigLIP2 features for the VLM encoder (ablation) | Qwen2.5-7B VLM · MMBench | accuracy: 77.00 -> 77.00 | No clear effect |
| concatenate SigLIP2 semantic features with Flux.2 VAE latents instead of semantic features alone | unified tokenizer · ImageNet-1K val reconstruction | PSNR: 18.83 -> 31.46 | Improved |
| use features with higher intrinsic dimension (Flux.2 vs DINOv2-B) as the diffusion target (ablation) | DiT latent diffusion · ImageNet 256 FID vs intrinsic dimension | FID | Hurt |
Closest papers by meaning
- DuoGen: Towards Autonomous Interleaved Multimodal Generation (CVPR 2026 · similarity 0.89)
- Dual Diffusion for Unified Image Generation and Understanding (CVPR 2025 · similarity 0.88)
- Reconstruction Alignment Improves Unified Multimodal Models (ICLR 2026 · similarity 0.88)
- Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models (CVPR 2025 · similarity 0.88)
- G$^2$TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models (NeurIPS 2026 · similarity 0.88)
- MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
