Research map / Vision-language and multimodal / Multimodal LLMs

OralGPT-Plus: Learning to Use Visual Tools via Reinforcement Learning for Panoramic X-ray Analysis

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces OralGPT-Plus, an agentic VLM for panoramic dental X-ray analysis trained with expert tool-use trajectories (DentalProbe), SFT, and rubric/curiosity-based GRPO with zoom-in and mirror-in tools, evaluated on the new MMOral-X benchmark.

Problem

Single-pass vision-language models cannot revisit ambiguous regions or compare bilateral symmetry, limiting reliable multi-step diagnosis on panoramic dental radiographs.

Approach

An agentic thought-action-observation loop with a zoom-in tool and a symmetry-based mirror-in tool, trained by dentist-like SFT on expert-curated trajectories and then RL with a hybrid reward (rubric-based continuous score, confidence-conditioned curiosity bonus, format).

Key results

• OralGPT-Plus-7B reaches 43.16 / 20.60 / 24.96 on MMOral-X Simple/Moderate/Complex, above GPT-5's 32.90 / 8.30 / 9.30.
• On MMOral-OPG, OralGPT-Plus-7B obtains an overall open-ended VQA score of 45.35 versus 42.34 for GPT-5.
• Removing the Mirror-In tool drops Simple accuracy from 43.16 to 34.68 and Complex from 24.96 to 14.30.

Surprising findings

• Without dentist-like instruction tuning, models fail to activate tool-usage behaviors even after RL, so tool use does not emerge from reward optimization alone.
• Ablating the conditioned diagnostic reward induces reward hacking, where the agent invokes a single tool and answers without meaningful inspection.

Limitations

• Failure cases on complex radiographs with dense, overlapping lesions, where tool use can amplify hallucinations.
• Reliance on GPT-5-mini as judge and rubric evaluator, and GPT-5 for trajectory synthesis.
• Small 3B model fails to sustain multi-round tool use and gains little from RL.

Experimental setup

Datasets: DentalProbe Tufts Dental Database Children's Panoramic Dataset DENTEX Diagnostics TEAM MMOral-OPG

Benchmarks: MMOral-X MMOral-OPG

Metrics: Avg@5 score (0-1 judge scale) pass@1 standard deviation coefficient of variation

Baselines: GPT-5 Gemini-2.5-Flash Doubao-seed-1-6 Claude-sonnet-4-5 Qwen3-VL-235b-a22b-thinking Grok-4 GLM-4.5v LLaVA-Med MedDr HuatuoGPT-V HealthGPT MedVLM-R1

Builds on: GRPO Mini-o3 PixelReasoner Qwen2.5-VL LLaMA-Factory

Closest papers by meaning

• Towards Better Dental AI: A Multimodal Benchmark and Instruction Dataset for Panoramic X-ray Analysis (NeurIPS 2025 · similarity 0.91)
• OralGPT-Omni: A Versatile Dental Multimodal Large Language Model (CVPR 2026 · similarity 0.89)
• OralXrays-9: Towards Hospital-Scale Panoramic X-ray Anomaly Detection via Personalized Multi-Object Query-Aware Mining (CVPR 2025 · similarity 0.87)
• CariesXrays: Enhancing Caries Detection in Hospital-Scale Panoramic Dental X-rays via Feature Pyramid Contrastive Learning (AAAI 2024 · similarity 0.85)
• MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level Precision (AAAI 2026 · similarity 0.85)
• NeBLa: Neural Beer-Lambert for 3D Reconstruction of Oral Structures from Panoramic Radiographs (AAAI 2024 · similarity 0.85)