Research map / Vision-language and multimodal / Multimodal LLMs
ProJudge: A Multi-Modal Multi-Discipline Benchmark and Instruction-Tuning Dataset for MLLM-based Process Judges
Jiaxin Ai, Pengfei Zhou, Zhaopan Xu, Ming Li, Fanrui Zhang, Zizhen Li, Jianwen Sun, Yukang Feng, Baojin Huang, Zhongyuan Wang, Kaipeng Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale InternVL2.5 judge from 8B to 38B | InternVL2.5 · ProJudgeBench | step correctness accuracy: 25.58 -> 78.85 | Improved |
| fine-tune on ProJudge-173k with Dynamic Dual-Phase | Qwen2.5-VL-3B · ProJudgeBench | step correctness accuracy: 11.47 -> 81.29 | Improved |
| fine-tune on ProJudge-173k with Dynamic Dual-Phase | InternVL2.5-8B · ProJudgeBench | step correctness accuracy: 25.58 -> 84.50 | Improved |
| add Dynamic Dual-Phase to standard fine-tuning (ablation) | InternVL2.5-8B · ProJudgeBench (overall) | step correctness accuracy: 83.37 -> 84.50 | Improved |
| add Dynamic Dual-Phase to standard fine-tuning (ablation) | Qwen2.5-VL-3B · ProJudgeBench (overall) | step correctness accuracy: 80.57 -> 81.29 | Improved |
| add Dynamic Dual-Phase to standard fine-tuning (ablation) | Qwen2.5-VL-7B · ProJudgeBench (overall) | step correctness accuracy: 81.91 -> 83.72 | Improved |
| add Dynamic Dual-Phase to standard fine-tuning (ablation) | Qwen2.5-VL-7B · ProJudgeBench OlympicArena subset (unseen) | step correctness accuracy: 77.88 -> 82.67 | Improved |
| add Dynamic Dual-Phase to standard fine-tuning (ablation) | InternVL2.5-8B · ProJudgeBench OlympicArena subset (unseen) | step correctness accuracy: 81.71 -> 85.07 | Improved |
| add Dynamic Dual-Phase to standard fine-tuning (ablation) | Qwen2.5-VL-3B · ProJudgeBench OlympicArena subset (unseen) | step correctness accuracy: 80.38 -> 81.01 | Improved |
| increase problem difficulty from high school to OlympiadBench | Gemini-2.0 · ProJudgeBench | step correctness accuracy: 80.2 -> 56.2 | Hurt |
| judge solutions from a weak student model versus a strong one (GPT-4o judging MiniCPM-V-2_6 vs reverse) | GPT-4o / MiniCPM-V-2_6 as judges · ProJudgeBench cross-model evaluation | judge accuracy: 84.31 vs 15.69 | Mixed |
| evaluate biology versus physics problems | Qwen2.5-VL-72B · ProJudgeBench by discipline | step correctness accuracy: 84.13 (biology) vs 77.08 (physics) | Mixed |
Closest papers by meaning
- AlphaMath Almost Zero: Process Supervision without Process (NeurIPS 2024 · similarity 0.88)
- MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs (NeurIPS 2024 · similarity 0.87)
- PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization (ICLR 2024 · similarity 0.87)
- RBench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation (ICML 2025 · similarity 0.87)
- EvaLearn: Quantifying the Learning Capability and Efficiency of LLMs via Sequential Problem Solving (NeurIPS 2025 · similarity 0.87)
- Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
