Research map / Large language models / Language, safety and interpretability

SAIL: Sample-Centric In-Context Learning for Document Information Extraction

AAAI 2025 · Large language models › Language, safety and interpretability · approach: Data and evaluation methodology · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SAIL, a training-free sample-centric in-context learning method for document information extraction that selects per-sample layout-similar, entity-level text-similar, and document-level text-similar examples and fills a unified prompt template.

Problem

Training-free document information extraction from visually rich documents struggles with understanding layout-text relationships and with giving precise, model-agnostic guidance to LLMs via fixed in-context examples.

Approach

Select tailored demonstrations for each test sample using three similarity signals (document-level text via Sentence-BERT, entity-level text via nearest-neighbor entity search, and layout similarity via MSE on rendered box images), and place them in a unified ICL prompt template with an LLM-generated layout analysis step.

Key results

• SAIL with GPT-4 reaches 84.67 F1 on FUNSD, 96.41 on CORD, and 98.18 on SROIE, versus 78.94, 87.47, 89.23 for ICL-D3IE with GPT-4.
• SAIL with ChatGLM3 drops only about 12.7% on SROIE when switching from GPT-3.5 (85.03 F1), while ICL-D3IE drops 73.8% (18.83 F1).
• GPT-4o multimodal baseline achieves 50.72 F1 on FUNSD, 71.53 on CORD, and 47.49 on SROIE, below SAIL.

Measured effects reported in the paper

Surprising findings

• Document-level text similarity examples alone gave lower F1 than fixed random examples on the long-text FUNSD dataset (69.60 vs 69.87 with none).
• Pulling layout images into a 1D vector for cosine similarity markedly reduced F1 (94.73) compared to MSE or SSIM on layout images (95.80, 95.42).

Limitations

• Search process adds time cost (13.3% with GPT-4).
• Diverse per-sample examples increase total token count.
• Reported results depend on API LLMs and specific prompt budgets (e.g., 4 examples per type limited by token count).

Experimental setup

Datasets: FUNSD CORD SROIE

Benchmarks: FUNSD CORD SROIE

Metrics: entity-level F1 precision recall

Baselines: ICL-D3IE Standard ICL BERT LiLT BROS XYLayoutLM LayoutLM LayoutLMv2 LayoutLMv3 GPT-4o LLaVA-v1.5-7B

Builds on: ICL-D3IE Sentence-BERT in-context learning

Closest papers by meaning

• In-Context Pretraining: Language Modeling Beyond Document Boundaries (ICLR 2024 · similarity 0.86)
• Relation-Rich Visual Document Generator for Visual Information Extraction (CVPR 2025 · similarity 0.86)
• A Simple yet Effective Layout Token in Large Language Models for Document Understanding (CVPR 2025 · similarity 0.85)
• DocKS-RAG: Optimizing Document-Level Relation Extraction through LLM-Enhanced Hybrid Prompt Tuning (ICML 2025 · similarity 0.84)
• Adaptive Markup Language Generation for Contextually-Grounded Visual Document Understanding (CVPR 2025 · similarity 0.84)
• DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understanding (CVPR 2025 · similarity 0.84)