Research map / Vision-language and multimodal / Visual question answering
PDF-WuKong: A Large Multimodal Model for Efficient Long PDF Reading with End-to-End Sparse Sampling
Xudong Xie, Hao Yan, Liang Yin, Yang Liu, Jing Ding, Minghui Liao, Yuliang Liu, Wei Chen, Xiang Bai
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add sparse sampler (ablation) | PDF-WuKong · PaperPDF English (ablation) | ANLS: 11.1 -> 40.3 | Improved |
| add end-to-end joint training of sampler and MLLM (ablation) | PDF-WuKong · PaperPDF English (ablation) | ANLS: 40.3 -> 42.6 | Improved |
| increase training data from 100k to 1M (ablation) | PDF-WuKong · PaperPDF English (ablation) | ANLS: 38.7 -> 42.6 | Improved |
| increase training data from 500k to 1M (ablation) | PDF-WuKong · PaperPDF English (ablation) | ANLS: 41.6 -> 42.6 | Improved |
| add multi-evidence data to single-evidence data at equal scale (ablation) | PDF-WuKong · PaperPDF English (ablation) | ANLS: 39.2 -> 40.0 | Improved |
| increase sampled chunks from top 1 to top 5 (ablation) | PDF-WuKong · PaperPDF English (ablation) | ANLS: 39.20 -> 42.59 | Improved |
| increase sampled chunks from top 5 to top 20 (ablation) | PDF-WuKong · PaperPDF English (ablation) | ANLS: 42.59 -> 43.42 | No clear effect |
| use parsed input instead of page images | PDF-WuKong · PaperPDF single-evidence subset | ANLS: 36.6 -> 41.5 | Improved |
| use parsed interleaved text-image input instead of plain text (InternVL2) | InternVL2 · PaperPDF English test | GPT-Acc: 37.5 -> 54.3 | Improved |
| add text RAG (BGE-M3) to IXC2-VL | IXC2-VL · PaperPDF English test | GPT-Acc: 37.6 -> 48.4 | Improved |
| fine-tune BGE-M3 retriever on PaperPDF | InternVL2 + BGE-M3 · PaperPDF English test | ANLS: 29.8 -> 36.4 | Improved |
| use PDF-WuKong instead of InternVL2 with parsed input | PDF-WuKong · PaperPDF Chinese test | ANLS: 28.5 -> 40.9 | Improved |
Closest papers by meaning
- Docopilot: Improving Multimodal Models for Document-Level Understanding (CVPR 2025 · similarity 0.89)
- SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understanding (ICLR 2025 · similarity 0.88)
- Needle In A Multimodal Haystack (NeurIPS 2024 · similarity 0.88)
- MMLONGBENCH-DOC: Benchmarking Long-context Document Understanding with Visualizations (NeurIPS 2024 · similarity 0.88)
- DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understanding (CVPR 2025 · similarity 0.87)
- Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documents (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
