Research map / Vision-language and multimodal / Multimodal LLMs

Hierarchical Process Reward Models are Symbolic Vision Learners

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SymHPR, a hierarchical process reward model trained with GRPO on synthetic logic-form parsing data, and builds SymVAE, a self-supervised symbolic auto-encoder that reconstructs diagrams via a rendering engine, evaluated on diagram reconstruction, understanding, and geometry reasoning.

Problem

Pixel-based vision encoders capture textures and colors rather than the explicit geometric primitives and relations in semantically sparse diagrams, and existing process reward models rely on costly human annotation or unreliable LLM judges.

Approach

Parse diagrams into hierarchical symbolic logic forms (points, lines, shapes, indicators, relations) and train with rule-based, dependency-weighted process rewards (F1, L2 distance) on synthetic data; then use a rendering engine as decoder for self-supervised auto-encoding, stabilized by Gaussian-noise hard negatives and power reward normalization.

Key results

• SymVAE-7B reaches MSE 16.8 and SSIM 0.83 on Geo170K diagrams, versus 37.2 MSE and 0.73 SSIM for VQ-GAN.
• SymVAE-7B achieves 72.6% average accuracy on MathGlance diagram understanding versus 59.2% for Qwen2.5-VL-7B.
• SymVAE+CoTs-7B reaches 51.8% on MathVerse test-mini (vs 49.2% Qwen2.5-VL-7B) and 79.4% on GeoQA (vs 76.4%).

Surprising findings

• Flat (unweighted) averaging of reward components gives performance similar to the cold-start model, while hierarchical dependency weighting yields gains.
• Vanilla GRPO with visual rewards collapses due to low reward variance on sparse diagrams, with KL near zero.

Limitations

• Rule-based symbolic design limits flexibility for geometric concepts outside the predefined primitives.
• Fails to reconstruct domain-specific symbols such as resistors and ammeters, and dense diagrams.
• Neuro-symbolic pipeline with LLM is less flexible than direct visual-token integration.

Experimental setup

Datasets: synthetic diagram-logic form dataset PGDP Geo170K GeoPeP / MathGlance VisCodex MathV360K ChartMimic MMMU-Pro EEE-Bench MMChemOCR

Benchmarks: MathVerse GeoQA MathGlance ChartMimic Direct Mimic Geo170K test diagrams

Metrics: MSE LPIPS SSIM DINO similarity accuracy

Baselines: VAE VQ-GAN GPT-4o Qwen2.5-VL-7B PyhParser MAVIS-7B Primitive-7B VisCodex-8B

Builds on: GRPO Qwen2.5-VL AlphaGeometry VisCodex Verifiers framework

Closest papers by meaning

• Primitive Vision: Improving Diagram Understanding in MLLMs (ICML 2025 · similarity 0.86)
• Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation (ICLR 2026 · similarity 0.85)
• Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learning (CVPR 2026 · similarity 0.85)
• Can Large Language Models Understand Symbolic Graphics Programs? (ICLR 2025 · similarity 0.84)
• Synthesizing Visual Concepts as Vision-Language Programs (CVPR 2026 · similarity 0.84)
• CogFlow: Bridging Perception and Reasoning through Knowledge Internalization for Visual Mathematical Problem Solving (ICLR 2026 · similarity 0.84)