Research map / Reinforcement learning / Policy gradient and actor-critic
Self-Critical N-Step Training for Image Captioning
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with reinforcement learning that directly optimizes CIDEr rather than cross entropy | Att2in image captioner · MSCOCO Karpathy test split | CIDEr: 102.1 -> 115.1 | Helped |
| Use K Monte Carlo rollouts rather than max-probability rollout (ablation) | 1-step self-critical training · MSCOCO Karpathy test split | CIDEr: 115.1 -> 115.4 | Helped |
| Use a 1-2-2-step advantage schedule rather than 1-step max-probability training (ablation) | Self-critical max-probability rollout · MSCOCO Karpathy test split | CIDEr: 115.1 -> 115.2 | Helped |
| Use a 1-2-2-step schedule with Monte Carlo rollouts rather than 1-step (ablation) | Self-critical Monte Carlo rollout · MSCOCO Karpathy test split | CIDEr: 115.4 -> 114.9 | Hurt |
| Use 2-step max-probability advantage rather than 1-step advantage (ablation) | Self-critical max-probability rollout · MSCOCO Karpathy test split | CIDEr: 115.1 -> 114.6 | Hurt |
| Use 4-step max-probability advantage rather than 1-step advantage (ablation) | Self-critical max-probability rollout · MSCOCO Karpathy test split | CIDEr: 115.1 -> 114.5 | Hurt |
| Use a 1-2-4-T-step schedule rather than 1-step max-probability training (ablation) | Self-critical max-probability rollout · MSCOCO Karpathy test split | CIDEr: 115.1 -> 114.8 | Hurt |
| Use a T-4-2-1-step schedule rather than 1-step max-probability training (ablation) | Self-critical max-probability rollout · MSCOCO Karpathy test split | CIDEr: 115.1 -> 114.0 | Hurt |
| Use per-token self-critical advantage rather than sequence-level advantage | Self-critical image captioning · MSCOCO Karpathy test split | CIDEr: 112.7 -> 115.1 | Helped |
| Use non-parametric rollout value estimates rather than a parametrized estimator | Self-critical per-token training · MSCOCO Karpathy test split | CIDEr: 113.9 -> 115.4 | Helped |
| Ensemble four 1-2-2-step-maxpro models rather than use the single model (ablation) | 1-2-2-step-maxpro · MSCOCO official test server c5 | CIDEr: 111.3 -> 112.6 | Helped |
Closest papers by meaning
- Self-Critical Sequence Training for Image Captioning (CVPR 2017 · similarity 0.91)
- CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning (ICLR 2026 · similarity 0.88)
- BalCapRL : A Balanced Framework for RL-Based MLLM Image Captioning (NeurIPS 2026 · similarity 0.87)
- RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Models (NeurIPS 2025 · similarity 0.85)
- Learning by Correction: Efficient Tuning Task for Zero-Shot Generative Vision-Language Reasoning (CVPR 2024 · similarity 0.85)
- Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.