Research map / Reinforcement learning / Policy gradient and actor-critic
Temporal Difference Calibration in Sequential Tasks: Application to Vision-Language-Action Models
Shelly Francis-Meretzki, Mirco Mutti, Yaniv Romano, Aviv Tamar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace binary cross-entropy with TD loss (TDQC) (ablation) | black-box RNN predictor · OpenVLA WidowX, unseen tasks | sequential Brier score: .344 -> .192 | Improved |
| replace binary cross-entropy with TD loss (TDQC) (ablation) | black-box RNN predictor · UniVLA LIBERO, unseen tasks | ROC-AUC: 54.37 -> 70.17 | Improved |
| replace MC/BCE loss with TD loss (ablation) | SAFE-RNN · pi0 LIBERO, unseen tasks | sequential Brier score: .172 -> .097 | Improved |
| replace MC/BCE loss with TD loss (ablation) | SAFE-RNN · pi0-FAST Franka, unseen tasks | ROC-AUC: 56.69 -> 68.43 | Improved |
| replace MC/BCE loss with TD loss (ablation) | SAFE-RNN · OpenVLA LIBERO, unseen tasks | ROC-AUC: 69.04 -> 65.12 | Hurt |
| replace BCE with TD loss (ablation) | SAFE-MLP · OpenVLA LIBERO, unseen tasks | ROC-AUC: 64.99 -> 60.09 | Hurt |
| replace BCE with TD loss (ablation) | SAFE-MLP · pi0-FAST LIBERO, unseen tasks | sequential Brier score: .162 -> .150 | Improved |
| replace BCE with TD loss (ablation) | SAFE-MLP · OpenVLA LIBERO, unseen tasks | sequential Brier score: .231 -> .229 | No clear effect |
| train black-box predictor on action probabilities with TDQC instead of static heuristics | black-box RNN · OpenVLA LIBERO, unseen tasks | sequential Brier score: .313 -> .197 | Improved |
| use TD-calibrated action probabilities (black box) instead of white-box hidden states | RNN-TDQC vs SAFE-RNN · OpenVLA LIBERO, unseen tasks | sequential Brier score: .255 -> .197 | Improved |
| use TDQC-trained value predictor to guide action search among 10 sampled actions | OpenVLA · LIBERO-10, 3 held-out tasks | success rate: 13% over the regular baseline (55% average) | Improved |
| use BCE-trained value predictor for guided action search | OpenVLA · LIBERO-10, 3 held-out tasks | success rate: 50% | Improved |
Closest papers by meaning
- World–Value–Action Model: Implicit Planning for Vision–Language–Action Systems (NeurIPS 2026 · similarity 0.88)
- PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.88)
- ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models (NeurIPS 2026 · similarity 0.88)
- Vision Language Models are In-Context Value Learners (ICLR 2025 · similarity 0.88)
- FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies (NeurIPS 2026 · similarity 0.88)
- VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Model (ICML 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
