Research map / Large language models / Language, safety and interpretability

Mind the Quote: Enabling Quotation-Aware Dialogue in LLMs via Plug-and-Play Modules

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: Faithfulness of LLM explanations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Formalizes quotation-aware dialogue as span-conditioned generation, builds a synthetic data pipeline and five-scenario benchmark, and proposes QuAda, a per-head query/value bottleneck adapter that modulates attention to quoted token spans.

Problem

LLMs lack an explicit mechanism to locate and use quoted spans of earlier dialogue text referenced by users, and existing span-injection methods either inflate prompts or use static attention steering.

Approach

QuAda attaches two bottleneck MLPs per attention head: a query-side bias that shifts attention scores toward or away from quoted tokens and a value-side enrichment added to quoted-token values, training only these adapters on a synthetic quotation corpus while keeping the backbone frozen and the prompt unchanged.

Key results

• QuAda on Qwen2.5-3B-Instruct reaches 95.2% Base, 94.6% Multi-Span, 92.8% Exclude, 85.8% Info-Combine, and 90.2% Coref accuracy, outperforming trained PASTA, Marker, and Concat baselines.
• QuAda on Llama3.1-8B-Instruct reaches 96.0% Base, 98.2% Multi-Span, 93.2% Exclude, 82.6% Info-Combine, and 94.8% Coref accuracy.
• Untrained Vanilla Qwen2.5-3B scores 28.4% on Base and 29.4% on Coref, showing that the quotation task is not solved by the backbone alone.

Measured effects reported in the paper

Surprising findings

• Trained Concat-Repeat drops to 15.2%-39.8% range on Exclude and Coref for some settings, showing content duplication can hurt negative or positional reasoning.
• Value-only QuAda variant recovers most of the full method's performance, while query-only is much weaker.

Limitations

• Limited to monolingual English text spans and monomodal settings.
• Benchmark relies on LLM-generated data with LLM-based checks and LLM-based judging (GPT-4o-mini), with only 25 samples per scenario used for human agreement.

Experimental setup

Datasets: CoNLL-2012 synthetic quotation dialogues generated by LLMs

Benchmarks: Mind the Quote benchmark (Base, Multi-Span, Exclude, Info-Combine, Coref)

Metrics: Accuracy Consistency score (1-5)

Baselines: Vanilla PASTA Marker-Insertion Concat-Repeat

Builds on: PASTA Focus Directions Concat-Repeat Marker-Insertion

Closest papers by meaning

• StateTree: Enhancing Long-term Dialogue Reasoning via Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
• Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training (ICLR 2025 · similarity 0.86)
• Teaching Language Models to Evolve with Users: Dynamic Profile Modeling for Personalized Alignment (NeurIPS 2025 · similarity 0.86)
• Aligning Spoken Dialogue Models from User Interactions (ICML 2025 · similarity 0.86)
• Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents (ICLR 2026 · similarity 0.86)
• Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Models (NeurIPS 2024 · similarity 0.86)