Research map / Large language models / Reasoning and chain of thought

Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberation

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Routing, pruning and distillation for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SpecBench, a 1,500-prompt benchmark of scenario-specific safety and behavioral specifications, and Align3, a lightweight test-time deliberation method with hierarchical reflection that improves specification alignment (SAR) with minimal token overhead.

Problem

LLMs must follow dynamic, scenario-specific behavioral and safety specifications, but there is no unified benchmark measuring this jointly, and training-based alignment is costly to adapt as specifications evolve.

Approach

Formalizes specification alignment as a constrained objective (maximize behavioral compliance subject to safety budget), defines the SAR metric that zeroes unsafe responses and scores safe ones by behavioral compliance, and proposes Align3, a three-stage single-pass thinking intervention (behavior optimization, safety-guided refinement at end-of-thinking, holistic audit).

Key results

• Align3 raises Qwen3-14B-thinking SAR from 57.32% (vanilla thinking) to 62.92% and Qwen3-14B non-thinking from 51.03% to 62.92%, with about 1,832 average tokens.
• GPT-5-chat achieves the highest SAR of 82.14% among evaluated models, ahead of GPT-4.1 at 69.20%.
• Multi-pass TTD methods such as TPO improve Qwen3-14B SAR to 58.76% but use about 21.6k completion tokens versus 946 for vanilla.

Surprising findings

• Self-Refine sharply degrades Llama-3.1-8B-Instruct SAR (44.54% to 35.16%), attributed to weaker generation quality and no external reward signal.
• Gemini-2.5-pro underperforms Gemini-2.5-flash-thinking, likely due to the constrained reasoning budget.

Limitations

• GPT-5 and OpenAI o-series models could not be evaluated because vendor safety guards blocked many prompts.
• Evaluation relies on LLM evaluators (GPT-4.1), with potential bias acknowledged by the authors.
• Benchmark covers only five scenarios with about 1,500 prompts and 103 specifications.

Experimental setup

Datasets: WMDP WildJailbreak BioprotocolBench ChildSafe DPO Code Vulnerability Mitre Attack Arena Hard BigCodeBench Fitness QA Medical Dialogue Traveler Trip

Benchmarks: SpecBench

Metrics: Specification Alignment Rate (SAR) safety score behavioral score

Baselines: Best-of-N Self-Refine TPO ZeroThink MoreThink RealSafe-R1-8B STAIR-Llama-3.1-8B-DPO-3 GPT-4.1 GPT-5-chat

Builds on: Best-of-N Self-Refine TPO MoreThink ZeroThink deliberative alignment

Closest papers by meaning

• RvLLM: LLM Runtime Verification with Domain Knowledge (NeurIPS 2025 · similarity 0.86)
• CoRe: Benchmarking LLMs’ Code Reasoning Capabilities through Static Analysis Tasks (NeurIPS 2025 · similarity 0.86)
• The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awareness (NeurIPS 2025 · similarity 0.86)
• Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements (ICLR 2025 · similarity 0.85)
• Unveiling the Magic of Code Reasoning through Hypothesis Decomposition and Amendment (ICLR 2025 · similarity 0.85)
• CodeCrash: Exposing LLM Fragility to Misleading Natural Language in Code Reasoning (NeurIPS 2025 · similarity 0.85)