Research map / Large language models / Language, safety and interpretability
CONSIDER: Commonalities and Specialties Driven Multilingual Code Retrieval Framework
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CONSIDER, a multilingual code retrieval framework that models pairwise and global commonalities across programming languages and uses a confusion-matrix-guided negative sampling scheme to capture language specialties, evaluated on CodeSearchNet.
Problem
Multilingual code retrieval struggles with data scarcity for some languages in specific domains and with identifying the intended programming language of a query when languages are interchangeable.
Approach
Combines pairwise and global commonality extraction modules (attention over language tokens plus mutual information maximization with JSDE) with a contrastive learning negative sampler that draws batches according to a confusion matrix computed on the validation set, plus language balancing.
Key results
- In the multilingual scenario, CONSIDER improves overall MRR over the multilingual baseline by +5.8 (RoBERTa(code), 62.6 vs 56.2) to +4.3 points (UniXCoder, 69.6 vs 65.3).
- With UniXCoder, CONSIDER reaches 69.6 overall MRR in the multilingual setting, versus 65.4 for the Distill method.
- In the monolingual scenario, CONSIDER gives consistent gains across all six languages over the monolingual models, per Figure 3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply CONSIDER instead of Distill | UniXCoder · Ruby | MRR: +3.4 | Helped |
| Apply CONSIDER instead of Distill | UniXCoder · JavaScript | MRR: +2.0 | Helped |
| Apply CONSIDER instead of Distill | UniXCoder · Go | MRR: +2.0 | Helped |
| Apply CONSIDER instead of Distill | UniXCoder · Python | MRR: +4.4 | Helped |
| Apply CONSIDER instead of Distill | UniXCoder · Java | MRR: +4.3 | Helped |
| Apply CONSIDER instead of Distill | UniXCoder · PHP | MRR: +4.7 | Helped |
| Apply CONSIDER instead of Distill | UniXCoder · Overall multilingual | MRR: +4.3 | Helped |
| Add the commonalities enhancement module (ablation) | CONSIDER with CodeBERT · Monolingual and multilingual scenarios | MRR | Helped |
| Add the confusion-matrix-guided sampling algorithm (ablation) | CONSIDER with CodeBERT · Monolingual and multilingual scenarios | MRR | Helped |
Surprising findings
- Plain multilingual training hurts high-resource languages in the monolingual setting, even though it helps low-resource ones.
Limitations
- Evaluated only on CodeSearchNet with six languages and a single retrieval metric (MRR).
- Confusion matrix updates require periodic validation retrieval, adding training overhead and hyperparameters (alpha, beta) tuned by grid search.
Experimental setup
Datasets: CodeSearchNet
Benchmarks: CodeSearchNet
Metrics: MRR
Baselines: Multilingual training Distill RoBERTa(code) CodeBERT GraphCodeBERT UniXCoder
Builds on: UniXCoder contrastive learning Jensen-Shannon mutual information estimator (Ou et al. 2021) knowledge distillation multilingual code retrieval (Li, Xu, and Chen 2022)
Closest papers by meaning
- CodeMMR: Bridging Natural Language, Code, and Image for Unified Retrieval (CVPR 2026 · similarity 0.87)
- CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking (ICLR 2025 · similarity 0.86)
- CPRet: A Dataset, Benchmark, and Model for Retrieval in Competitive Programming (NeurIPS 2025 · similarity 0.86)
- Towards A Generalist Code Embedding Model Based On Massive Data Synthesis (NeurIPS 2025 · similarity 0.85)
- PinNet: Pinpoint Instructive Information for Retrieval Augmented Code-to-Text Generation (ICML 2024 · similarity 0.85)
- McEval: Massively Multilingual Code Evaluation (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.