Research map / Reinforcement learning / Policy gradient and actor-critic
Temporal-Difference Learning With Sampling Baseline for Image Captioning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with RL (CIDEr reward) instead of cross-entropy | FC captioner · MS COCO Karpathy test | CIDEr: 96.3 -> 109.8 | Helped |
| replace shared per-word reward with TD(lambda) per-word discounted values (ablation) | FC captioner, greedy baseline · MS COCO Karpathy test | CIDEr: 107.4 -> 108.7 | Helped |
| replace greedy-sampled baseline with multinomial-sampled baseline (ablation) | TD FC captioner · MS COCO Karpathy test | CIDEr: 108.7 -> 109.8 | Helped |
| use TD with multinomial baseline instead of shared reward with greedy baseline (ablation) | FC captioner · MS COCO Karpathy test | CIDEr: 107.4 -> 109.8 | Helped |
| replace greedy-sampled baseline with multinomial-sampled baseline (ablation) | TD FC captioner · MS COCO Karpathy test | BLEU-4: 33.1 -> 33.1 | No clear effect |
| replace greedy-sampled baseline with multinomial-sampled baseline (ablation) | TD FC captioner · MS COCO Karpathy test | METEOR: 25.7 -> 26.0 | Helped |
| use attention features instead of global FC features | TD-Multinomial · MS COCO Karpathy test | CIDEr: 109.8 -> 111.6 | Helped |
| compare TD-Multinomial-FC with SCST (FC-2K) | captioner · MS COCO Karpathy test | CIDEr: 106.3 -> 109.8 | Helped |
| increase beam size K from 1 to 10 (ablation) | XENT-FC · MS COCO Karpathy test | CIDEr | Mixed |
| increase beam size K from 1 to 10 (ablation) | TD-Multinomial-FC · MS COCO Karpathy test | CIDEr | No clear effect |
Closest papers by meaning
- CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning (ICLR 2026 · similarity 0.90)
- BalCapRL : A Balanced Framework for RL-Based MLLM Image Captioning (NeurIPS 2026 · similarity 0.88)
- Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards (CVPR 2025 · similarity 0.86)
- PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Models (CVPR 2024 · similarity 0.86)
- RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Models (NeurIPS 2025 · similarity 0.86)
- Dynamic Prompt Optimizing for Text-to-Image Generation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.