Research map / Large language models / AI and society

Position: Enforced Amnesia as a Way to Mitigate the Potential Risk of Silent Suffering in the Conscious AI

ICML 2024 · Large language models › AI and society · approach: Human-centered trust and agency

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper arguing that enforced amnesia (memory resets or blocked memory access) could mitigate the hypothetical risk of a conscious AI silently suffering, without needing to establish whether it is conscious.

Problem

Conscious AI systems (e.g., LLMs) might undergo unobserved suffering from tedious tasks, and there is no reliable test for consciousness to confirm or rule this out.

Approach

Past experience affects an agent only through memory, so preventing memory access or regularly resetting memory (including weights or hidden states in extreme cases) could cap cumulative suffering and disrupt a continuous suffering-prone self-identity; a simple RL-style value function formalizes when amnesia is preferable.

Key results

• Theoretical model shows amnesia is preferable at time t when the discounted sum of past rewards is negative, i.e., when cumulative memories carry negative utility.
• Argues that LLM state strings, hidden states, and weights could each be reset to limit hypothetical locked-in suffering.
• Recommends policymakers encourage consciousness indicators and discourage continuous operation of AI and brain organoids beyond a time limit without memory reset.

Surprising findings

• Argues amnesia could be optimal even when past memories are non-negative under a reward-saturation value formulation.

Limitations

• Analysis is highly speculative; AI consciousness is treated as a hypothetical thought experiment.
• Amnesia may constrain welfare of agents whose experiences are positive and may degrade AI performance.
• Assumes conscious states carry a utility function, with no empirical evidence.

Experimental setup

Builds on: Sutton & Barto reinforcement learning framework LSTM continual learning

Closest papers by meaning

• InEx: Hallucination Mitigation via Introspection and Cross-Modal Multi-Agent Collaboration (AAAI 2026 · similarity 0.80)
• The Compliance Trap: Diagnosing How AI Agents Consume Conflicting Memory (NeurIPS 2026 · similarity 0.80)
• MemoryBank: Enhancing Large Language Models with Long-Term Memory (AAAI 2024 · similarity 0.79)
• Proactive Privacy Amnesia for Large Language Models: Safeguarding PII with Negligible Impact on Model Utility (ICLR 2025 · similarity 0.79)
• In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (ICML 2024 · similarity 0.79)
• RealityTest: How People Probe AI Identity and Whether Models Disclose It (NeurIPS 2026 · similarity 0.79)