Research map / Large language models / Language, safety and interpretability

Adaptive Few-shot Prompting for Machine Translation with Pre-trained Language Models

AAAI 2025 · Large language models › Language, safety and interpretability · approach: Multilingual LLM data and culture · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AFSP, an adaptive few-shot prompting framework that retrieves translation demonstrations using the deployed LLM's own embeddings (hybrid dense/sparse/multi-vector), samples multiple candidates, and reranks them with a self-supervised BERT reranker, and releases a 5,528-sentence Diplomatic Chinese-English corpus.

Problem

LLM-based machine translation with in-context learning is prompt-sensitive, so a fixed set of demonstrations is suboptimal across inputs; the paper targets selecting input-specific demonstrations and improving semantic consistency of outputs.

Approach

Retrieve top-k demonstrations using a training-free hybrid score (dense, sparse, and multi-vector) computed from the deployed LLM's own embedding layer rather than third-party embedders, then sample multiple candidate translations and select one with a BERT reranker trained via self-supervised negative sampling with text degradation operations.

Key results

• On the Diplomatic corpus with Llama3-8B (Chinese-to-English), AFSP improves BLEU-4 from 25.32 (KNN few-shot) to 31.02 range-level gains reported in the tables, with the full AFSP achieving best scores across models.
• With ChatGLM3-6B on Diplomatic English-to-Chinese, AFSP reaches BLEU-4 27.36 versus 20.22 for KNN few-shot and 20.27 for few-shot.
• Replacing the deployed LLM's embeddings with third-party embedders (BGE-M3, E5-Large, BGE-large, BCE) yields lower scores in the ChatGLM3-6B ablation, supporting use of the LLM's own embeddings.

Measured effects reported in the paper

Surprising findings

• Reported test sets are only 500 pairs each, and the UN evaluation uses 120,000 randomly selected entries, which is inconsistent with the stated protocol.
• Some ablation tables report the same numbers for different settings (e.g., CHRF of 49.78 in the alpha ablation for English-to-Chinese versus 48.xx for other weights), suggesting possible table inconsistencies.

Limitations

• Few-shot shot count limited to k=3 by GPU memory on an RTX 3090.
• Top-30 sampling for open models and top-5 for ChatGPT due to cost; human evaluation uses only 5 examples per direction with 14 annotators.

Experimental setup

Datasets: UN Open Corpus v1.0 (Chinese-English) Diplomatic Chinese-English corpus

Benchmarks: Diplomatic Chinese-English corpus UN Open Corpus v1.0 Chinese-English subset

Metrics: BLEU-4 METEOR ROUGE-1 ROUGE-2 ROUGE-L CHRF COMET-Kiwi

Baselines: Zero-shot prompting Few-shot prompting KNN Few-shot BGE-M3 E5-Large BGE-large BCE

Builds on: M3-Embedding BERT in-context learning kNN few-shot prompting

Closest papers by meaning

• Large Language Models are Demonstration Pre-Selectors for Themselves (ICML 2025 · similarity 0.86)
• A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (ICLR 2024 · similarity 0.85)
• Model-Aware Tokenizer Transfer (NeurIPS 2026 · similarity 0.85)
• Comparative Analysis of Demonstration Selection Algorithms for In-Context Learning in Large Language Models (Student Abstract) (AAAI 2025 · similarity 0.85)
• The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generation (CVPR 2025 · similarity 0.84)
• Teaching Large Language Models to Translate with Comparison (AAAI 2024 · similarity 0.84)