Research map / Vision-language and multimodal / Multimodal LLMs

Twin-T & TwintVQA: A Reliable Structure-Detail Separating VLM and a Comprehensive Benchmark for Chart and Table Tasks

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Twin-T, a 1B/7B chart-table VLM with a dual-head (structure + detail) image encoder fused via Schur-style gating and trained with MINT preference learning, plus the TwintVQA benchmark of 4,941 questions.

Problem

Current VLMs underuse the regular layout and explicit numbers of charts and tables, leading to value errors and unreliable chart-table analysis across image, LaTeX, and Python formats.

Approach

A dual-head image encoder processes the raw image and a Canny structure image, then a parameter-free Schur-style soft gate removes structural directions from the image embedding to separate detail features before fusion; MINT adds numeric-keyword contrastive preference, low-entropy number regularization, and text-vision evidence matching in preference learning.

Key results

• Twin-T-7B achieves overall 719.94, the highest among open-source models and above GLM-4.5V-106B (667.08), approaching GPT-4o (714.89) and Gemini-2.5-Pro (724.26).
• Twin-T-1B reaches overall 576.20, outperforming several 7-8B open models such as Qwen2.5-VL-7B's 631.09? no: it is below Qwen2.5-VL-7B; it does exceed LLaVA-Next-8B (429.92) and ChartVLM-8B (367.64).
• Ablation: removing the dual-head encoder drops 7B overall by about 32.8 points (675.05 to 642.28); removing Stage 2 drops 7B overall from 719.94 to 672.52.

Surprising findings

• Canny structure images on their own can reduce performance, which is why a structure-gating step is needed.

Limitations

• Falls short on OCR-heavy tasks and code reconstruction (Chart-to-Python), attributed to smaller model size and data scale.
• Benchmark scores rely on GPT-4o-Mini as judge on a 0-1 scale.

Experimental setup

Datasets: ChartQA AI2D CharXiv ChartMimic LogicVista OCRVQA SEEDBench2 TableVQA

Benchmarks: TwintVQA AI2D CharXiv ChartMimic ChartQA LogicVista OCRVQA SEEDBench2 TableVQA

Metrics: score (0-1 converted to percentage) overall sum

Baselines: GPT-4o Gemini-2.5-Pro GLM-4.5V-106B Qwen2.5-VL-7B Ovis2-8B ChartAst-13B ChartLlama-13B Kimi-VL-16B Gemma3-27B SynTab-7B

Builds on: LLaVA Ovis2 Qwen2.5-VL DPO

Closest papers by meaning

• ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding (CVPR 2026 · similarity 0.88)
• ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.88)
• EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understanding (AAAI 2025 · similarity 0.88)
• Same or Not? Enhancing Visual Perception in Vision-Language Models (CVPR 2026 · similarity 0.87)
• TABLET: A Large-Scale Dataset for Robust Visual Table Understanding (ICLR 2026 · similarity 0.87)
• VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning (ICML 2026 · similarity 0.86)