Research map / Vision-language and multimodal / Multimodal LLMs

Unlocking the Essence of Beauty: Advanced Aesthetic Reasoning with Relative-Absolute Policy Optimization

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Aes-R1, a two-stage framework for image aesthetic assessment: an automatically built aesthetic chain-of-thought dataset (AesCoT) for SFT cold start, followed by RAPO, an RL algorithm with combined relative-rank and absolute-error rewards on Qwen2.5-VL-7B.

Problem

MLLMs for image aesthetic assessment lack aesthetic reasoning data and interpretable rationales, and RL for IAA fails to activate reasoning and struggles to design a reward that calibrates absolute scores while preserving relative ranking.

Approach

Combines a GPT-4.1-generated, score-masked aesthetic critique pipeline (AesCoT) for cold-start SFT with RAPO, a GRPO-style RL algorithm whose reward sums a FRank-style pairwise rank reward and a Gaussian absolute-error reward, so both ordinal consistency and calibrated scores are optimized.

Key results

• Aes-R1 reaches average PLCC 0.6337 and SRCC 0.6186 across five benchmarks, the best among compared methods, versus Q-Insight's 0.5954/0.5813 under the same retraining.
• Reward ablation: combined error plus rank reward yields average PLCC/SRCC 0.6297/0.6102, versus 0.4255/0.4433 for binary reward alone.
• Initializing RAPO from 1 epoch SFT gives the best result; 10-epoch SFT lowers entropy (0.705) and reduces RL gains.

Surprising findings

• Excessive SFT (10 epochs) followed by RL performs worse than RL from no SFT on OOD datasets, as entropy collapse limits exploration.
• Rank-only reward yields high SRCC but depressed PLCC, showing ordinal ranking alone does not calibrate scores.

Limitations

• Cold-start data relies on GPT-4.1 generated critiques filtered with partly automatic checks and human audits, limiting scalability and fidelity to expert judgment.
• Absolute-error reward depends on a hyperparameter sigma and on the relative-rank term using estimated per-image mean and variance from sampled outputs.

Experimental setup

Datasets: AVA TAD66K FLICKR-AES PARA AADB

Benchmarks: TAD66K AVA FLICKR-AES PARA AADB

Metrics: PLCC SRCC

Baselines: Qwen2.5-VL-7B Qwen2.5-VL-32B GPT-4o GPT-4.1 Seed1.5-VL-thinking NIMA Q-Align DeQA-Score Q-Insight VisualQuality-R1

Builds on: GRPO DAPO DeepSeek-R1 VisualQuality-R1 Q-Insight

Closest papers by meaning

• Bridging Cognitive Gap: Hierarchical Description Learning for Artistic Image Aesthetics Assessment (AAAI 2026 · similarity 0.90)
• Advancing Comprehensive Aesthetic Insight with Multi-Scale Text-Guided Self-Supervised Learning (AAAI 2025 · similarity 0.89)
• Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms (NeurIPS 2024 · similarity 0.89)
• ArtiMuse: Fine-Grained Image Aesthetics Assessment with Joint Scoring and Expert-Level Understanding (CVPR 2026 · similarity 0.88)
• Can Vision–Language Models Assess Graphic Design Aesthetics? A Benchmark, Evaluation, and Dataset Perspective. (ICLR 2026 · similarity 0.87)
• Venus: Benchmarking and Empowering Multimodal Large Language Models for Aesthetic Guidance and Cropping (CVPR 2026 · similarity 0.87)