Research map / Vision-language and multimodal / Multimodal LLMs

MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Presents MiniCPM-V 4.5, an 8B MLLM that combines a unified 3D-Resampler for image/video token compression, a corruption-based unified objective for document knowledge and OCR, and hybrid short/long reasoning RL, reporting OpenCompass results that rival larger proprietary and open models at lower inference cost.

Problem

MLLM training and inference are bottlenecked by the large number of visual tokens (especially for video), by brittle external document parsers used for knowledge learning, and by verbose long-reasoning outputs even on simple tasks.

Approach

Extends the 2D resampler to a 3D-Resampler that jointly compresses spatial-temporal video frames into few tokens, unifies document knowledge learning and OCR by predicting original text from dynamically corrupted document image regions, and trains a single model with hybrid short/long reasoning modes alternated during GRPO rollouts.

Key results

• MiniCPM-V 4.5 (8B) reaches 77.0 OpenCompass average, above GPT-4o (75.4) and Qwen2.5-VL 72B (76.1) as reported in its table.
• On Video-MME with subtitles, it scores 73.5 using 0.26h inference time and 28G memory versus 3.00h and 60G for Qwen2.5-VL 7B.
• The 3D-Resampler uses 21.3 tokens per frame versus 64.0 for the 2D baseline while improving Video-MME (72.5 vs 71.5 w/o subs).

Surprising findings

• Hybrid RL reaches 77.1 OpenCompass with 3.1B training tokens, beating long-reasoning-only (77.0, 4.4B tokens) and short-only (76.0, 1.6B).
• Video OCR ability appears in the model even though no video OCR training data was specifically collected.

Limitations

• Many comparisons use the authors' own evaluation runs and reasoning-mode settings, so cross-model numbers may not be directly comparable.
• Several benchmarks show the model behind Qwen2.5-VL 72B or GLM-4.1V (e.g., MMBench, DocVQA, MathVerse), so it is not uniformly best.
• Pretraining and RL data details (exact sizes, filtering thresholds) are described only qualitatively.

Experimental setup

Datasets: LAION-2B COYO OmniCorpus MINT-1T WebVid Vript OpenVid OpenThoughts RLAIF-V VisualWebInstruct Llama-Nemotron-Post-Training Dataset MulDimIF

Benchmarks: OpenCompass MMVet MMStar MME MMBench V1.1 MMMU MathVista AI2D MathVerse LogicVista EMMA OCRBench

Metrics: accuracy average score CHAIR MMHal rate OmniDocBench edit distance inference time GPU memory

Baselines: Qwen2.5-VL 72B Qwen2.5-VL 7B InternVL3 8B GLM-4.1V 9B thinking GPT-4o MiMo-VL-7B-RL

Builds on: LLaVA-UHD RLPR RLAIF-V GRPO DPO Warmup-Stable-Decay learning rate schedule

Closest papers by meaning

• On Path to Multimodal Generalist: General-Level and General-Bench (ICML 2025 · similarity 0.88)
• SEED-Bench: Benchmarking Multimodal Large Language Models (CVPR 2024 · similarity 0.88)
• BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile Devices (CVPR 2025 · similarity 0.87)
• Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reduction (CVPR 2025 · similarity 0.87)
• Incentivizing Versatile Video Reasoning in MLLMs via Data-Efficient Reinforcement Learning (CVPR 2026 · similarity 0.87)
• Towards General Continuous Memory for Vision-Language Models (NeurIPS 2025 · similarity 0.87)