Research map / Vision-language and multimodal / Multimodal LLMs

RNED: Rotary Number Encoding and Decoding for Medical VLMs

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RNED, which encodes scalars as rotations of a dedicated [NUM] token embedding and decodes them via score matching, to improve numerical prediction in medical VLMs.

Problem

Medical VLMs perform poorly on quantitative prediction (e.g., lesion sizes, bounding-box coordinates) because LLM tokenizers split numbers into subword units and next-token objectives do not optimize numeric accuracy.

Approach

Encode a number as a norm-preserving multi-frequency rotation (RoPE-inspired) of a single [NUM] token embedding, so ordinal structure is kept and norms are unchanged; decode by searching for the candidate value whose precomputed weighted cosine score best matches the output embedding's score, trained with an added MSE score-matching loss.</natural_idea>

Key results

• On Opport-CT single-value measurement, RNED raises R2 from 0.338 (standard tokenization) to 0.568 and success rate from 55.7% to 81.8%.
• On CT-RATE, CT-CHAT + RNED improves single-value R2 from 0.370 to 0.592 and double-value R2 from 0.164 to 0.608.
• On MedSeq-Bench, MedSeq-Grounder + RNED beats the prior SOTA on 5 of 8 tasks, with average IoU 74.21 vs 72.55 (per the paper's table) and an average Acc@5 gain of 2.62 points.

Surprising findings

• Adding raw numeric values to the [NUM] embedding yields negative R2 (-2.025), and prompt-based number formats perform even worse than the standard tokenization baseline.
• On MedSeq-Bench the RNED variant is worse on the Image Difference Grounding tasks, where the baseline already performs well.

Limitations

• Evaluated only on 1D numeric values; multi-dimensional coordinates are left to future work.
• Opport-CT dataset is in-house and not released, limiting reproducibility.
• Decoding relies on a precomputed candidate grid over a bounded range (e.g., [0, 3000] with step 0.01), and the hyperparameter p needs tuning.

Experimental setup

Datasets: Opport-CT CT-RATE MedSeq-Bench

Benchmarks: Opport-CT CT-RATE MedSeq-Bench

Metrics: MAE R2 Success rate IoU Acc@5 BLEU-1 ROUGE-L METEOR GREEN

Baselines: xVal Learnable number embedding Abacus embedding p10 p100 Standard tokenization CT-CHAT MedSeq-Grounder Qwen2.5-VL

Builds on: RoPE xVal LLaVA LoRA

Closest papers by meaning

• How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images (ICLR 2026 · similarity 0.87)
• Toward a Vision-Language Foundation Model for Medical Data: Multimodal Dataset and Benchmarks for Vietnamese PET/CT Report Generation (NeurIPS 2025 · similarity 0.86)
• 3D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Tasks (NeurIPS 2025 · similarity 0.85)
• MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models (ICLR 2025 · similarity 0.85)
• Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding (ICLR 2025 · similarity 0.85)
• DrVD-Bench: Do Vision-Language Models Reason Like Human Doctors in Medical Image Diagnosis? (NeurIPS 2025 · similarity 0.85)