AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

FUSE: Fine-Grained and Semantic-Aware Learning for Unified Image Understanding and Generation

Peng Zhang, Wanggui He, Mushui Liu, Wenyi Xiao, Siyu Zou, Yuan Li, Xingjian Wang, Guanghao Zhang, Yanpeng Liu, Weilong Dai, Jinlong Liu, Shuyi Ying and 5 more

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FUSE, a unified image understanding and generation model that combines fine-grained and semantic-aware learning to improve fine-grained tasks such as image editing and virtual try-on beyond MLLM-only semantic encoding.

Problem

Unified models that rely heavily on MLLM high-level semantic encoding for diffusion-based image generation struggle with fine-grained visual tasks like image editing and virtual try-on.

Approach

Augments MLLM semantic conditioning with fine-grained visual learning so that the diffusion generator receives both high-level semantic and low-level detail signals; the abstract is truncated so the exact mechanism is not stated.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
inject VAE features via SD-Connector (ablation)FUSE · any2any-add subset (zero-shot editing)editing consistencyImproved
remove VAE features at diffusion stage (ablation)FUSE · any2any-add subsetsemantic-level editingNo clear effect
use step-wise credit assignment instead of uniform reward weighting (ablation)Adaptive-GRPO · held-out test setsemantic rewardImproved
use step-wise credit assignment instead of uniform reward weighting (ablation)Adaptive-GRPO · held-out test setfine-grained rewardImproved
use FUSE unified model instead of BAGELBAGEL · GenEvaloverall score: 0.88 -> 0.89Improved
use FUSE instead of FLUX.1-devFLUX.1-dev · MJHQ-30KFID: 6.6 -> 5.6Improved
use FUSE instead of FLUX.1-devFLUX.1-dev · WISEscore: 0.50 -> 0.65Improved
use FUSE instead of UniWorld-V1UniWorld-V1 · ImgEdit-Benchoverall: 3.37 -> 3.88Improved
use FUSE instead of best baselines on position promptsJanus-Pro-7B · GenEval Positionscore: 0.79 -> 0.77Hurt
use FUSE instead of Step1X-Edit on style editsStep1X-Edit · ImgEdit-Bench Stylescore: 4.44 -> 4.20Hurt

Limitations

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy