Research map / Architectures and efficiency / RNNs and normalization

RAPTOR: Ridge-Adaptive Logistic Probes

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Layer-wise analysis of language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RAPTOR, a validation-tuned ridge logistic probe whose normalized weights serve as concept vectors for additive activation steering, evaluated against GCS and xRFM, with a CGMT-based high-dimensional analysis of ridge logistic regression.

Problem

Probe-then-steer pipelines need concept vectors that are accurate, directionally stable under data perturbations, and cheap to estimate, but existing estimators trade these off poorly in the high-dimensional few-shot regime of LLM probing.

Approach

Use a single-knob l2-regularized logistic probe with train-only standardization and validation-selected ridge strength lambda; fold the weights back to native space and normalize them as the steering direction. A CGMT analysis of a Gaussian teacher-student model explains how lambda controls signal alignment and orthogonal energy.

Key results

• Averaged over the 7x6 grid, RAPTOR reaches 0.874 best-layer accuracy vs 0.854 for GCS and 0.871 for xRFM.
• RAPTOR improves over GCS in all 42 settings for average accuracy and in 41/42 for best-layer accuracy.
• Adaptive steering achieves probe-target success of 1.000 across all eight dataset-direction settings, with intervention rates of roughly 0.54 to 0.83.

Surprising findings

• GCS remains the most stable method overall in directional robustness, slightly ahead of RAPTOR.
• Labeled hidden states are nearly linearly separable in most layers (152 of 168 layer-dataset pairs), motivating ridge regularization.

Limitations

• Directional stability is weaker than GCS in several settings.
• The Gaussian teacher-student theory is a stylized model and not an exact model of LLM representations.
• Robustness evaluation covers only six model-dataset settings due to resource constraints.

Experimental setup

Datasets: STSA Cities Common CounterFact HateXplain Sarcasm

Benchmarks: STSA Cities Common CounterFact HateXplain Sarcasm

Metrics: best-layer accuracy average-layer accuracy directional robustness (mean absolute pairwise cosine) wall-clock training time probe-target success rate intervention rate steering strength |alpha|

Baselines: GCS xRFM

Builds on: GCS xRFM GCAV TCAV CGMT

Closest papers by meaning

• ADAPT: Hybrid Prompt Optimization for LLM Feature Visualization (NeurIPS 2026 · similarity 0.84)
• Compress to Impress: Efficient LLM Adaptation Using a Single Gradient Step on 100 Samples (NeurIPS 2025 · similarity 0.84)
• Structured Model Probing: Empowering Efficient Transfer Learning by Structured Regularization (CVPR 2024 · similarity 0.83)
• Fantastic Features and Where to Find Them: A Probing Method to combine Features from Multiple Foundation Models (NeurIPS 2025 · similarity 0.83)
• Optimal Representation Size: High-Dimensional Analysis of Pretraining and Linear Probing (NeurIPS 2026 · similarity 0.83)
• When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Search (NeurIPS 2026 · similarity 0.83)