Research map / Learning theory and algorithms / Prediction and decision losses

Time-Sensitive Anytime-Valid Testing

NeurIPS 2026 · Learning theory and algorithms › Prediction and decision losses · approach: Adaptive testing and experiments · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces time-sensitive testing-by-betting, maximizing expected time-dependent rejection rewards under a fixed alternative, proving a Markovian (wealth, time) reduction, e-process equivalence for simple-vs-simple, a finite-horizon Neyman-Pearson solution for hard deadlines, and an exponential-decay-optimal (EDO) criterion generalizing GRO.

Problem

x

Approach

Assign a non-increasing reward to the rejection time and choose betting strategies maximizing its expected value under a prescribed simple alternative; show Markov (wealth, time) policies suffice, reduce the simple-vs-simple case to e-processes, solve hard deadlines via Neyman-Pearson, and approximate exponential-decay rewards by a stationary constant bet (EDO) that recovers GRO as the time scale

Key results

• Markovian reduction: optimal value over all predictable policies equals optimum over policies depending only on current wealth and round.
• Simple-vs-simple: restricting to testing-by-betting loses no power relative to general e-processes; hard-deadline optimum is a Neyman-Pearson region at horizon T with Doob e-process.
• EDO criterion: stationary constant bet with eta* D_{1/(1-eta*)}(pi1||pi0)=1/T; reduces to GRO as T→infinity; in Gaussian case power one iff T ≥ 2σ²/μ².

Surprising findings

• For Gaussian hard-deadline testing, the optimal Doob e-process never rejects before the deadline, coinciding with the fixed-sample Neyman-Pearson test.
• With small time scales EDO may select an action with non-positive log-drift, so early rejection can come at the cost of losing power.

Limitations

• Only simple alternatives are treated; composite alternatives and composite nulls are left open.
• Rewards restricted to non-negative values, excluding loss-type objectives such as minimizing expected rejection time directly.
• Bellman recursion generally lacks closed-form solutions; EDO is an approximation and boundary effects are only heuristically handled.

Experimental setup

Metrics: expected reward of rejection time rejection probability within deadline

Baselines: GRO SPRT Neyman-Pearson fixed-horizon test

Builds on: GRO criterion Ville's inequality Neyman-Pearson lemma SPRT

Closest papers by meaning

• Peeking with PEAK: Sequential, Nonparametric Composite Hypothesis Tests for Means of Multiple Data Streams (ICML 2024 · similarity 0.85)
• Sequential Kernel Goodness-of-fit Testing (ICML 2024 · similarity 0.85)
• When to Trust the Cheap Check: Weak and Strong Verification for Reasoning (ICML 2026 · similarity 0.84)
• On the Limits of Test-Time Compute: Sequential Reward Filtering for Better Inference (ICML 2026 · similarity 0.83)
• Optimal Rates for Differentially Private Hypothesis Testing with E-values (NeurIPS 2026 · similarity 0.83)
• Strategic Scaling of Test-Time Compute: A Bandit Learning Approach (ICLR 2026 · similarity 0.83)