Research map / Large language models / Agents, code and math
E-comIQ-ZH: A Human-Aligned Dataset and Benchmark for Fine-Grained Evaluation of E-commerce Posters with Chain-of-Thought
Meiqi Sun, Mingyu Li, Junxiong Zhu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add domain-specific SFT on E-comIQ-18k training set (ablation) | Qwen2.5-VL-7B · E-comIQ-18k test (overall) | SRCC: 0.119 -> 0.346 | Improved |
| add GRPO stage with accuracy and distribution rewards after SFT (ablation) | Qwen2.5-VL-7B+SFT · E-comIQ-18k test (overall) | SRCC: 0.346 -> 0.433 | Improved |
| add GRPO stage with accuracy and distribution rewards after SFT (ablation) | Qwen2.5-VL-7B+SFT · E-comIQ-18k test (background) | SRCC: 0.530 -> 0.520 | No clear effect |
| add SFT before GRPO instead of GRPO only (ablation) | Qwen2.5-VL-7B · E-comIQ-18k test (overall) | SRCC: 0.154 -> 0.346 | Improved |
| add accuracy-only GRPO after SFT (ablation) | Qwen2.5-VL-7B+SFT · E-comIQ-18k test (overall) | SRCC: 0.346 -> 0.360 | Improved |
| add distribution reward term to the accuracy reward (ablation) | SFT+GRPO · E-comIQ-18k test (overall) | SRCC: 0.360 -> 0.410 | Improved |
| apply original Q-Insight GRPO reward (no CoT) on the dataset | Q-Insight · E-comIQ-18k test (overall) | SRCC: 0.152 -> 0.235 | Improved |
| use E-comIQ-M as automatic evaluator on out-of-domain benchmark vs human scores | E-comIQ-M · E-comIQ-Bench | PLCC/SRCC with human: around 0.34 | Mixed |
| use OCR-based text metrics to evaluate Chinese text rendering | E-comIQ-Bench · E-comIQ-Bench | Phrase F1 vs human text score: GPT-4o phrase F1 0.86, human text 2.83 | Hurt |
Closest papers by meaning
- P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark (ICLR 2026 · similarity 0.84)
- AI-generated Image Quality Assessment in Visual Communication (AAAI 2025 · similarity 0.81)
- Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers (NeurIPS 2025 · similarity 0.81)
- GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Models (AAAI 2025 · similarity 0.81)
- Can Vision–Language Models Assess Graphic Design Aesthetics? A Benchmark, Evaluation, and Dataset Perspective. (ICLR 2026 · similarity 0.81)
- An Evaluation Framework for Product Images Background Inpainting Based on Human Feedback and Product Consistency (AAAI 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
