Research map / Large language models / Language, safety and interpretability
In-Context Learning as Rate–Distortion Optimization
Jiayu Zhang, Changbang Li, Canran Xiao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add marginal information gain scoring (ablation) | RDCO · 10-dataset ICL suite | overall average: 59.27 -> 61.76 | Improved |
| add length normalization (information density per token) (ablation) | RDCO · 10-dataset ICL suite | overall average: 60.46 -> 61.76 | Improved |
| add reliability weight wi (ablation) | RDCO · 10-dataset ICL suite | overall average: 60.98 -> 61.76 | Improved |
| add redundancy penalty (ablation) | RDCO · 10-dataset ICL suite | overall average: 60.63 -> 61.76 | Improved |
| add conflict penalty (ablation) | RDCO · generation suite | average exact match: 58.59 -> 60.26 | Improved |
| add KL-bounded compaction (ablation) | RDCO · generation suite | average exact match: 58.67 -> 60.26 | Improved |
| set redundancy penalty lambda near 0 (ablation) | RDCO · validation split, B=256 | overall score | Hurt |
| use overly large conflict penalty alpha (ablation) | RDCO · validation split, B=256 | overall score | Hurt |
| set compaction KL threshold epsilon too permissive (ablation) | RDCO · overall score vs token budget | overall score | Hurt |
| use RDCO instead of best baseline TTF for classification | TTF · classification suite | average accuracy: 61.06 -> 63.26 | Improved |
| use RDCO instead of best baseline EPR for generation | EPR · generation suite | average exact match: 58.00 -> 60.26 | Improved |
| use RDCO instead of TTF on SST-5 | TTF · SST-5 | accuracy: 42.14 -> 41.75 | Hurt |
Closest papers by meaning
- Technical Debt in In-Context Learning: Diminishing Efficiency in Long Context (NeurIPS 2025 · similarity 0.91)
- Capturing In-Context Learning Dynamics with Task Operators (NeurIPS 2026 · similarity 0.90)
- Implicit In-context Learning (ICLR 2025 · similarity 0.89)
- DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning (NeurIPS 2024 · similarity 0.89)
- In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering (ICML 2024 · similarity 0.89)
- Disentangling Latent Shifts of In-Context Learning with Weak Supervision (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
