Research map / Vision-language and multimodal / Human and embodied modeling
MAGVLT: Masked Generative Vision-and-Language Transformer
Sungwoong Kim, Daejin Jo, Donghoon Lee, Jongmin Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace autoregressive decoding with masked iterative refinement | VLT · MS-COCO validation, zero-shot T2I | FID: 16.93 -> 12.08 | Improved |
| Replace autoregressive decoding with masked iterative refinement | VLT · MS-COCO validation, zero-shot T2I | speed: $1.00\times$ -> $8.12\times$ | Improved |
| Add joint image-and-text mask prediction (IT2IT) (ablation) | MAGVLT · MS-COCO, I2T | CIDEr: 53.5 -> 60.4 | Improved |
| Add joint image-and-text mask prediction (IT2IT) (ablation) | MAGVLT · MS-COCO, T2I | FID: 10.74 -> 12.08 | Hurt |
| Add UnrollMask (ablation) | MAGVLT · MS-COCO, T2I | FID: 14.12 -> 13.26 | Improved |
| Add UnrollMask (ablation) | MAGVLT · MS-COCO, I2T | CIDEr: 53.8 -> 56.5 | Improved |
| Add MixSel (ablation) | MAGVLT · MS-COCO, I2T | CIDEr: 56.5 -> 60.4 | Improved |
| Add MixSel (ablation) | MAGVLT · MS-COCO, T2I | FID: 12.08 -> 12.07 | No clear effect |
| Replace MixSel with MixRandom (mixed context without indicating the selected context) (ablation) | MAGVLT · MS-COCO, I2T | CIDEr: 60.4 -> 57.9 | Hurt |
| Replace MixSel with MixRandom (mixed context without indicating the selected context) (ablation) | MAGVLT · MS-COCO, T2I | FID: 12.08 -> 13.43 | Hurt |
| Increase the share of T2I training relative to I2T and IT2IT (ablation) | MAGVLT · MS-COCO, T2I | FID: 15.17 -> 12.08 | Improved |
| Increase the share of T2I training relative to I2T and IT2IT (ablation) | MAGVLT · MS-COCO, I2T | CIDEr: 61.7 -> 60.4 | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 0 of 1 on most shared results.
- SimVLM huge — SimVLM: Simple Visual Language Model Pretraining with Weak Supervision (ICLR 2022) · better on 0 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- JetFormer: An autoregressive generative model of raw images and text (ICLR 2025 · similarity 0.91)
- Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency (ICLR 2024 · similarity 0.89)
- LMFusion: Adapting Pretrained Language Models for Multimodal Generation (NeurIPS 2025 · similarity 0.89)
- GMAIL: Generative Modality Alignment for generated Image Learning (ICML 2025 · similarity 0.89)
- Unified Generative and Discriminative Training for Multi-modal Large Language Models (NeurIPS 2024 · similarity 0.88)
- Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
