Research map / Reinforcement learning / Policy gradient and actor-critic

Efficient Adaptive Data Acquisition via Pretrained Belief Representations

NeurIPS 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Model-based RL with guarantees · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes POLAR, which uses hidden representations of pretrained tabular foundation models (TabICL, TabPFN) as belief-state encoders and trains a lightweight policy head with policy gradients for BED, BO, and active learning.

Problem

Adaptive data acquisition (Bayesian experimental design, Bayesian optimisation, active learning) requires mapping observation histories to decisions; posterior-based methods suffer from approximation error, while direct history-based policy learning is sample-inefficient.

Approach

Decouple representation from decision: a pretrained TFM's candidate-conditioned hidden representations serve as belief representations, and a small policy head trained by score-function policy gradient selects the next query; backbone is updated only by an auxiliary supervised prediction loss, not by policy gradients.

Key results

• On 2D location finding, POLAR slightly outperforms ALINE using about 100x fewer training samples and surpasses DAD and RL-BOED with about 1000x fewer samples.
• On CES at 1e6 training samples, POLAR reaches sPCE 13.06 ± 0.12 vs ALINE 9.11 ± 0.17 and DAD 11.46 ± 0.16.
• On HPO-B across six search spaces, POLAR achieves the lowest normalised regret and best average rank throughout the trajectory.

Surprising findings

• Coupling policy gradients into the backbone severely degrades performance, indicating decoupling is critical.
• A frozen backbone remains competitive with peak RL-BOED performance on location finding.

Limitations

• Policy acts over finite candidate pools; continuous designs need candidate generation.
• Local one-step credit assignment rather than long-horizon value estimation may miss delayed information gains.
• Deployment is slower than other amortised policies due to foundation-model forward passes.

Experimental setup

Datasets: location finding simulator CES simulator HPO-B Dockstring

Benchmarks: location finding (2D, 5D) CES HPO-B Dockstring

Metrics: sPCE EIG lower bound sNMC EIG upper bound normalised regret average rank deployment time

Baselines: Random DAD RL-BOED ALINE GP Meta-GP PFNs4BO NAP TabICL Random Forest

Builds on: DAD ALINE TabICL TabPFN sPCE

Closest papers by meaning

• ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisition (NeurIPS 2025 · similarity 0.87)
• The Wasserstein Believer: Learning Belief Updates for Partially Observable Environments through Reliable Latent Space Models (ICLR 2024 · similarity 0.86)
• Predictive Coding Enhances Meta-RL To Achieve Interpretable Bayes-Optimal Belief Representation Under Partial Observability (NeurIPS 2025 · similarity 0.86)
• Adaptive teachers for amortized samplers (ICLR 2025 · similarity 0.85)
• Information Templates: A New Paradigm for Intelligent Active Feature Acquisition (NeurIPS 2026 · similarity 0.85)
• ESCORT: Efficient Stein-variational and Sliced Consistency-Optimized Temporal Belief Representation for POMDPs (NeurIPS 2025 · similarity 0.85)