Research map / Reinforcement learning / Visual and representation RL

CIG: Exploration via Conditional Information Gain

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Novelty and impact-driven exploration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Derives CIG, a tractable per-step intrinsic reward from a log-determinant over an ensemble disagreement kernel (with trace reduction and Cholesky decomposition) that conditions on both replay buffer and rollout prefix, and evaluates it within DreamerV2 on 12 MiniGrid and OGBench tasks.

Problem

Intrinsic exploration rewards in model-based RL either condition only on the lifelong buffer or only on the within-rollout prefix; joint trajectory-level information gain is intractable for deep models, and prior hybrids use heuristic weights or Gaussian-process dynamics that do not scale.

Approach

A finite-ensemble, moment-matched Gaussian approximation of trajectory information gain yields a log-determinant over a trace-reduced ensemble disagreement kernel with an aleatoric ridge; its Cholesky factorization gives per-step rewards equal to lifelong disagreement minus the part explained by the rollout prefix.

Key results

• On MultiRoom-N7-S8 clean, CIG accumulates an order of magnitude more successes than P2E by end of training.
• CIG is the only method in the top tier on every task and has the highest aggregate normalised IQM with non-overlapping CIs against all baselines across 12 tasks.
• Under Noisy-TV distractors, RND, ICM, APT, E3B and E3B x P2E drop to near-zero successes on MiniGrid, while CIG and P2E retain non-trivial success with CIG outpacing P2E on every task.

Surprising findings

• APT roughly doubles CIG's successes on ObstructedMaze clean, and high visit-entropy alone does not predict downstream performance.
• Naively multiplying an episodic indicator with a lifelong bonus (E3B x P2E) trails standalone P2E on MultiRoom and KeyCorridor.

Limitations

• APT outperforms CIG on ObstructedMaze clean, and E3B x P2E surpasses CIG on Puzzle 3x3 Noisy-TV.
• Evaluated only in model-based RL with short imagined rollouts (T≈15); O(T^3) cost and kernel storage may be prohibitive for long horizons.
• Trace reduction discards directional structure of the epistemic covariance.

Experimental setup

Datasets: MiniGrid OGBench

Benchmarks: MiniGrid KeyCorridorS4R3 MiniGrid MultiRoom-N7-S8 MiniGrid ObstructedMaze-2Dlhb OGBench AntMaze OGBench Cube-Triple OGBench Puzzle-3x3 OGBench Scene

Metrics: cumulative successes coverage percentage inter-quartile mean probability of improvement

Baselines: Plan2Explore RND ICM APT E3B E3B x P2E

Builds on: DreamerV2 Plan2Explore TIP E3B RND

Closest papers by meaning

• Beyond Optimism: Exploration With Partially Observable Rewards (NeurIPS 2024 · similarity 0.88)
• MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization (ICLR 2025 · similarity 0.88)
• DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
• Causal Information Prioritization for Efficient Reinforcement Learning (ICLR 2025 · similarity 0.87)
• Effective Exploration Based on the Structural Information Principles (NeurIPS 2024 · similarity 0.87)
• Improving Intrinsic Exploration by Creating Stationary Objectives (ICLR 2024 · similarity 0.87)