Deployment-Time Memorization in Foundation-Model Agents: Privacy-Utility Tradeoffs in Persistent Memory Systems
By
[Submitted on 8 Jun 2026]
Summary
This paper introduces the concept of "deployment-time memorization" in foundation-model agents, where memory is an explicit function during deployment rather than just a property of model weights. The authors study how memory-design choices (summarization aggressiveness, retrieval breadth, and deletion mode) jointly affect personalization utility, extraction risk, and deletion fidelity. They propose metrics including Personalization Recall (PR), Adversarial Extraction Rate (AER), and Forgetting Residue Score (FRS). Key findings show that key-fact summarization reduces canary extraction by 76% on Gemma 3 12B and 64% on GPT-4o-mini while preserving personalization recall, but raw-only deletion leaves derived summary copies recoverable in ~20% of instances, requiring full-pipeline purge or tombstone redaction for complete erasure.
Source
bskyDeployment-Time Memorization in Foundation-Model Agents: Privacy-Utility Tradeoffs in Persistent Memory Systemsarxiv.orgKey quotes
· 5 pulledFoundation-model agents are increasingly long-lived systems that remember users across interactions, making memorization an explicit deployment-time function rather than solely a property of model weights.
Existing work addresses parametric memorization or audits fixed memory configurations, but does not characterize how memory-design choices jointly shape personalization utility, extraction risk, and deletion fidelity.
On LongMemEval, key-fact summarization reduces canary extraction by 76% on Gemma 3 12B and 64% on GPT-4o-mini while preserving nearly all personalization recall; critically, once content is compressed away, increasing k no longer restores leakage.
The same compression, however, induces a deletion-fidelity failure: raw-only deletion leaves derived summary copies recoverable in approximately 20% of instances, and only full-pipeline purge or tombstone redaction drives worst-tier residue to zero.
Together, these results establish that persistent agent memory must be evaluated as a first-class memorization mechanism -- assessed by what it helps agents recall, what it makes extractable, and what it can truly erase.
You might also wanna read
Systematic Study of Agent Memory Systems for LLMs Reveals No One-Size-Fits-All Architecture
This paper presents a systematic experimental study of agent memory systems for LLM agents from a data management perspective. It proposes a
arxiv.org·10h ago
Memori launches agent-native persistent memory infrastructure using structured knowledge graphs from agent trace data
Memori is a new agent-native memory infrastructure that enables AI agents to create structured, long-term persistent memory directly from ag
Product Hunt·1mo ago
Analyzing Memorization in Transformers Through Loss Landscape Curvature Decomposition
This research paper analyzes how memorization manifests in transformer models (both language models and vision transformers) through loss la
arxiv.org·7mo ago
Robot Memory System Enables AI Robots to Learn from Past Experiences
Robot Memory (robotmem) is a persistent memory system for AI robots that enables robots to learn from past experiences. The system stores ep
github.com·3mo agoJavelinGuard: Low-Cost Transformer Architectures for LLM Security
AgentMemory: Open-source persistent memory tool for AI coding agents
AgentMemory is an open-source tool that gives AI coding agents (like Claude Code, Codex, Cursor, etc.) persistent memory across sessions, so
Product Hunt·1mo agoComments
Sign in to join the conversation.
No comments yet. Be the first.