Research map / Large language models / Reasoning and chain of thought

Correctness-Optimized Residual Activation Lens (CORAL): Transferrable and Calibration-Aware Inference-Time Steering

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CORAL, an inference-time steering method that trains a weight-decay MLP probe on frozen residual activations to predict per-option residual correctness and uses it to recalibrate MCQA probabilities, improving accuracy and calibration and transferring to held-out benchmarks.

Problem

LLMs remain miscalibrated after instruction tuning and preference alignment, and existing inference-time steering methods optimize proxies for correctness (verbalized confidence, attention heads, perturbation stability) rather than correctness itself, and rarely show transfer to out-of-distribution

Approach

Train a regularized (weight-decay, output-penalized) MLP probe on mean-pooled answer-token residual-stream activations to predict the residual correctness r_j = (y_j - p_j) per option, then center the predictions and add them to the base option probabilities at inference to jointly improve accuracy and Brier-based calibration.

Key results

• On the transfer benchmarks, CORAL averages about 14% accuracy improvement and 49% ECE improvement over baselines across three 7B models (as stated by the authors).
• On Mistral-7B-Instruct-v0.3 ARC-Challenge, CORAL reaches 73.46% accuracy and 3.65 ECE versus 61.86% and 8.58 for few-shot prompting.
• On RACE with Deepseek-7B-Chat, CORAL reaches 73.06% accuracy and 2.62 ECE, versus 55.67% and 4.40 for the best baseline (few-shot).

Surprising findings

• Single-neuron SAE feature ablations have negligible effect on ECE (mean change 0.0125 percentage points), and SAE-based steering with top features degrades accuracy and calibration.
• Multi-layer probes over concatenated top layers gave no improvement over single-layer probes, and attention-head probes had low R^2 (mean 0.022).

Limitations

• Evaluation is limited to multiple-choice question answering; generalization to free-form generation is untested.
• Requires labeled correctness data to train probes and adds inference latency from per-option forward passes.
• Transfer gains shrink when source and target tasks have different reasoning patterns (e.g., HellaSwag longer options).

Experimental setup

Datasets: CommonsenseQA RACE MMLU ARC-Challenge HellaSwag OpenBookQA Math-MC

Benchmarks: MMLU RACE CommonsenseQA ARC-Challenge HellaSwag OpenBookQA Math-MC

Metrics: accuracy ECE cwECE Brier score NLL

Baselines: SteerConf Few-Shot Prompting (lm-eval harness) ITI CCPS

Builds on: Inference-Time Intervention CCPS SteerConf CorrSteer

Closest papers by meaning

• To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Models (ICML 2025 · similarity 0.88)
• Conditional Optimal Bridge for Riemannian Activation Steering (NeurIPS 2026 · similarity 0.88)
• Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior (NeurIPS 2026 · similarity 0.88)
• Steering When Necessary: Flexible Steering Large Language Models with Backtracking (NeurIPS 2025 · similarity 0.88)
• Focus On This, Not That! Steering LLMs with Adaptive Feature Specification (ICML 2025 · similarity 0.87)
• APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference (ICML 2024 · similarity 0.87)