Research map / Large language models / Instruction and fine-tuning

Stability Regimes for Framing-Sensitive Fine-Tuning in Language Models

NeurIPS 2026 · Large language models › Instruction and fine-tuning · approach: Fine-tuning for reasoning data

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that loss attenuation on framing-specific REFUTES training slices can shift false-claim acceptance in fine-tuned LLM personas while preserving aggregate accuracy, and introduces the FrameRef dataset and a sequential exposure simulation.

Problem

Fine-tuning LLMs to act as simulators of framing-sensitive judgment often causes broad behavioral drift (bias shifts, degenerate heuristics, lost task competence), undermining experimental validity; the challenge is to induce localized decision shifts while preserving aggregate behavior.

Approach

Attenuate the SFT loss on refuted claims in a target framing slice (weight alpha in (0,1)) so target errors are corrected less, and identify model-specific stability regimes over sample count and alpha where balanced accuracy is preserved while target-frame MSPR rises; then test the personas in a feedback-driven sequential exposure task.

Key results

• For Llama-3.1-8B at alpha=0.3 and 15k samples, max balanced accuracy deviation from baseline was 0.015 while target-frame MSPR shifted by up to 0.053.
• Under feedback, Llama-8B cumulative linear score dropped from 7.38 (baseline) to 2.69 (authoritative persona), while without feedback scores were comparable (about 16-18).
• Human validation (120 Prolific participants, 2,400 judgments) found framing affected false-claim acceptance (chi-square = 11.80, p = 0.038) but not true-claim acceptance.

Surprising findings

• Mistral-7B-Instruct-v0.3 showed MSPR decreasing as attenuation increased, so the stable regime does not transfer across models.
• Random attenuation of REFUTES examples raised MSPR across nearly all framings, indicating global credulity drift rather than localized effects.

Limitations

• Claims are limited to short, verifiable factual statements, not full real-world information ecosystems.
• Spillover across framings persists due to shared representations, so intervention is not tightly localized.
• Stability regimes are model-dependent; Mistral-7B was judged unsuitable for the persona simulations.

Experimental setup

Datasets: FEVER FEVEROUS FrameRef

Benchmarks: FrameRef test split

Metrics: balanced accuracy true negative rate percent SUPPORTS mean SUPPORT probability on REFUTES (MSPR) confidence on correct and incorrect answers cumulative score (linear and log)

Baselines: No SFT base model random attenuation control alpha=1 SFT baseline Qwen2.5-14B-Instruct Mistral-7B-Instruct-v0.3 Llama-3.3-70B-Instruct

Builds on: FEVER FEVEROUS LoRA Chain-of-thought prompting

Closest papers by meaning

• Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models (NeurIPS 2026 · similarity 0.87)
• The Lock-in Hypothesis: Stagnation by Algorithm (ICML 2025 · similarity 0.86)
• Reflection-Bench: Evaluating Epistemic Agency in Large Language Models (ICML 2025 · similarity 0.86)
• Alignment Dynamics in LLM Fine-Tuning (NeurIPS 2026 · similarity 0.85)
• DEBATE: A Large-Scale Benchmark for Evaluating Opinion Dynamics in Role-Playing LLM Agents (NeurIPS 2026 · similarity 0.85)
• On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs (ICML 2026 · similarity 0.85)