Research map / Vision-language and multimodal / Multimodal LLMs
Rendering-Aware Reinforcement Learning for Vector Graphics Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RLRF, online GRPO-based reinforcement learning that rewards rendered SVG outputs (pixel, DreamSim, length rewards) after SFT, improving SVG generation in Qwen2.5-VL and StarVector models.
Problem
Autoregressive VLMs trained with token-level SFT for SVG generation never observe rendered outputs, leading to poor visual fidelity, looping, and weak out-of-distribution generalization; rendering is non-differentiable for discrete token generation.
Approach
Treat rendering as a black-box evaluator: sample multiple SVG rollouts per image, render with CairoSVG, and optimize a composite reward (normalized L2, Canny-L2, DreamSim, SVG length deviation) with GRPO and no KL term, after an SVG-SFT stage.
Key results
- On SVG-Stack-Hard, Qwen2.5-VL-7B with SVG-SFT then RLRF reduces MSE from 8.60 to 1.03 and raises DINO from 93.00 to 98.70 (vs SVG-SFT).
- StarVector-1B with RLRF improves MSE from 4.60 to 3.46 and DINO from 96.00 to 98.00 and code efficiency from -800 to -127.
- Qwen3-8B trained with RLRF on captions only (no SVG supervision) improves CLIP on Flickr30k from 22.50 to 24.42 and Accurate from 2.74 to 3.65; OmniSVG remains best on MM-Icon/MM-Illustration.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add RLRF after SVG-SFT | Qwen2.5-VL-7B · SVG-Stack-Hard | MSE: 8.60 -> 1.03 | Helped |
| Add RLRF after SVG-SFT | Qwen2.5-VL-7B · SVG-Stack-Hard | SSIM: 79.40 -> 95.10 | Helped |
| Add RLRF after SVG-SFT | Qwen2.5-VL-7B · SVG-Stack-Hard | Code Efficiency: -2.8k -> -334 | Helped |
| Add RLRF after SVG-SFT | Qwen2.5-VL-7B · SVG-Emoji | MSE: 6.39 -> 4.93 | Helped |
| Add RLRF after SVG-SFT | Qwen2.5-VL-7B · SVG-Icons | MSE: 6.16 -> 7.38 | Hurt |
| Apply RLRF directly without SVG-SFT (ablation) | Qwen2.5-VL-7B-Instruct · SVG-Stack-Hard | MSE: 23.10 -> 14.23 | Hurt |
| Add SVG-SFT before RLRF (ablation) | Qwen2.5-VL-7B-Instruct · SVG-Stack-Hard | MSE: 14.23 -> 4.42 | Helped |
| Remove the KL divergence penalty (ablation) | RLRF · Im2SVG | Reward progression | Helped |
| Increase roll-outs per input from 8 to 64 (ablation) | RLRF · Im2SVG | MSE: 8 to 64 | Helped |
| Increase sampling temperature up to 1.2 (ablation) | RLRF · Im2SVG | Reward: 1.2 | Helped |
| Increase sampling temperature beyond 1.2 (ablation) | RLRF · Im2SVG | Reward: beyond 1.2 | Hurt |
| Combine L2 and L2-Canny rewards (ablation) | RLRF · Im2SVG | Convergence | Helped |
Surprising findings
- RLRF applied directly to instruction-tuned Qwen2.5-VL-7B without SVG-SFT performs poorly (MSE 14.23 vs 4.42 with SFT first).
- Removing the KL penalty improved reward progression and avoided early saturation.
- Text2SVG RLRF on Qwen3-8B works with text captions only, though the model never sees images during training.
Limitations
- Model tends to give up on very complex inputs, diverging and looping.
- Text2SVG results still trail OmniSVG on MM-Icon and MM-Illustration; relies on basic primitives and imprecise Bezier curves.
- Standard metrics like CLIP and Aesthetic are biased toward natural images and poorly suited to SVG outputs; SVG-Icons MSE/SSIM slightly worse than SFT.
Experimental setup
Datasets: SVG-Stack Flickr30k MM-Icons
Benchmarks: SVG-Stack-Hard SVG-Emoji SVG-Fonts SVG-Icons Flickr30k MM-Icon MM-Illustration
Metrics: MSE SSIM DINO score LPIPS code efficiency (token count difference) CLIP score Accurate (LLM judge) Aesthetic
Baselines: StarVector-1B SVG-SFT VTracer Potrace PyAutoTrace LIVE DiffVG GPT-4o Claude 3.7 Sonnet Gemini 1.5 Pro OmniSVG Qwen2.5-VL-72B
Builds on: GRPO PPO DreamSim StarVector DiffVG
Closest papers by meaning
- Code as Reward: Empowering Reinforcement Learning with VLMs (ICML 2024 · similarity 0.89)
- Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation (ICLR 2026 · similarity 0.88)
- COVR: Collaborative Optimization of VLMs and RL Agent for Visual-Based Control (AAAI 2026 · similarity 0.88)
- VLMs-Guided Representation Distillation for Efficient Vision-Based Reinforcement Learning (CVPR 2025 · similarity 0.88)
- Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models (CVPR 2025 · similarity 0.87)
- StarVector: Generating Scalable Vector Graphics Code from Images and Text (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.