Research map / Vision-language and multimodal / Visual question answering
LaTr: Layout-Aware Transformer for Scene-Text VQA
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pre-train on IDL instead of no additional pre-training | LaTr-Base with Rosetta OCR · TextVQA validation, constrained | accuracy: 44.06% to 48.38% (+4.32%) | Helped |
| Use IDL pre-training | LaTr-Base with Amazon-OCR · TextVQA validation | accuracy: 52.29% to 58.03% (+3.32%) | Helped |
| Fine-tune on TextVQA and ST-VQA (ablation) | LaTr-Base · TextVQA validation | accuracy: 58.03 to 59.53 | Helped |
| Increase capacity from Base to Large | LaTr‡ · TextVQA test | accuracy: 59.55 to 61.60 | Helped |
| Add layout-aware pre-training on IDL (ablation) | LaTr-Base with ordered OCR and 2-D embeddings · TextVQA validation | accuracy: 51.22 to 57.38 (+7.01%) | Helped |
| Add 2-D layout embeddings without pre-training (ablation) | LaTr-Base with ordered OCR · TextVQA validation | accuracy: 50.37 to 51.22 (+0.85%) | Helped |
| Add ViT visual features without pre-training (ablation) | LaTr-Base with ordered OCR and 2-D embeddings · TextVQA validation | accuracy: 51.22 to 52.29 (+1.09%) | Helped |
| Add ViT visual features after layout-aware pre-training (ablation) | LaTr-Base · TextVQA validation | accuracy: 57.38 to 58.03 | Helped |
| Use FRCNN rather than no visual features (ablation) | LaTr‡-Base · TextVQA validation | accuracy: 58.92 to 58.45 | Hurt |
| Use ViT rather than FRCNN visual features (ablation) | LaTr‡-Base · TextVQA validation | accuracy: 58.45 to 59.53 | Helped |
| Increase IDL pre-training data from 1M to 64M (ablation) | LaTr-Base · TextVQA validation | accuracy: 55.12 to 58.03 | Helped |
| Use IDL rather than TextVQA for pre-training (ablation) | LaTr-Base · TextVQA validation | accuracy: 51.81 to 58.03 | Helped |
Closest papers by meaning
- Question Aware Vision Transformer for Multimodal Reasoning (CVPR 2024 · similarity 0.87)
- BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions (AAAI 2024 · similarity 0.87)
- Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understanding (CVPR 2025 · similarity 0.87)
- VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning (CVPR 2024 · similarity 0.87)
- Object Attribute Matters in Visual Question Answering (AAAI 2024 · similarity 0.86)
- Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documents (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.