Research map / Vision-language and multimodal / Multimodal LLMs

EffectMaker: Unifying Reasoning and Generation for Customized Visual Effect Creation

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EffectMaker, a reference-based VFX customization framework that pairs an MLLM (Qwen3-VL-8B) for semantic reasoning with a Wan2.2-TI2V-5B video DiT using in-context visual conditioning, plus a 130k-video synthetic dataset (EffectData) across 3k effect classes.

Problem

Customized visual effect (VFX) generation for video is hard due to scarce effect-specific data and the difficulty of modeling stylized or supernatural effects; existing methods need per-effect LoRA fine-tuning or fail to generalize to unseen effects.

Approach

Combine semantic guidance from an MLLM's understanding and reasoning features (injected via decoupled cross-attention) with visual in-context guidance from a reference video (dual-stream attention and biased RoPE in a video DiT), enabling per-effect-free transfer of a reference effect onto a user image.

Key results

• On OpenVFX (14 classes), EffectMaker achieves average VQ 2.84 vs 2.27 for Omni-Effects and 2.06 for VFXCreator, and CAS 4.63 vs 4.40 for Omni-Effects.
• Ablation: semantic+visual conditioning gives VQ 2.92, TA 1.24, CAS 4.40, RAS 4.16, outperforming either semantic-only or visual-only conditioning.
• Scaling effect classes from 100 to 1000 improves open-set CAS from 3.76 to 4.22 and RAS from 3.22 to 4.04.

Surprising findings

• Visual-only in-context conditioning performs worse than semantic-only conditioning on TA and CAS, suggesting low-level cue copying alone does not capture effect structure.

Limitations

• Struggles with extremely complex effects involving rapid or large motion, limited by base model capacity.
• Reliance on synthetic training data may introduce bias and miss real-world VFX diversity.
• Fails when the user's first-frame image is semantically incompatible with the reference effect.

Experimental setup

Datasets: EffectData OpenVFX Higgsfield

Benchmarks: OpenVFX

Metrics: VQ MQ TA CAS RAS user preference

Baselines: VFXCreator Omni-Effects Wan2.2-FT Video-as-Prompt VFXMaster

Builds on: Wan2.2 Qwen3-VL VFXCreator Omni-Effects Stable Diffusion 3

Closest papers by meaning

• VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models (CVPR 2024 · similarity 0.85)
• DuoGen: Towards Autonomous Interleaved Multimodal Generation (CVPR 2026 · similarity 0.84)
• VSTAR: Generative Temporal Nursing for Longer Dynamic Video Synthesis (ICLR 2025 · similarity 0.84)
• CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilities (AAAI 2025 · similarity 0.84)
• PromptCoT: Align Prompt Distribution via Adapted Chain-of-Thought (CVPR 2024 · similarity 0.84)
• VIGC: Visual Instruction Generation and Correction (AAAI 2024 · similarity 0.84)