When Agents Learn to Act Like You: Privacy Leakage and Impersonation in Persona Skills

Introduction

Personalized agents are moving beyond remembering preferences. They are beginning to learn how a person communicates, makes choices, and approaches tasks. The paper “When Agents Learn to Be You” examines the safety implications of this shift, focusing on persona skills: portable, executable artifacts distilled from a user’s personal interaction history.

The promise is clear. A persona skill can make downstream agents more useful by carrying a user’s habits, task patterns, and interaction style across contexts. But the same mechanism also concentrates personal signals that were previously scattered across many conversations. Once these signals are packaged into a reusable skill, their potential impact is amplified.

Key points

• A new benchmark called AntiSkillBench: The authors propose an end-to-end benchmark for evaluating risks and defenses across the persona-skill pipeline.
• Dataset design: AntiSkillBench contains 7,500 persona-grounded dialogue traces built from 50 behaviorally rich profiles across diverse task scenarios.
• Multiple risk levels: The evaluation covers privacy leakage at the skill level, as well as attribute disclosure and behavioral impersonation at the agent level.
• Different distillation strategies: The benchmark tests risks across three persona-skill distillation approaches, rather than treating the pipeline as a single fixed implementation.
• Defense evaluation: Four defense configurations are examined, spanning online and post-hoc interventions, including active risk suppression and passive provenance protection.

The risk is broader than sensitive fields

One important message of the paper is that privacy risk in personalized agents is not limited to names, locations, or other explicit attributes. The experiments indicate that risks can extend to communication style, personality traits, and other behavioral cues. In practice, an agent may reveal who a person is not only by stating facts, but also by behaving in a recognizably similar way.

This is especially relevant because persona skills are designed for reuse. A user’s distilled preferences and behavior patterns may be transferred to different agents and task settings. That portability is useful, but it also means that a single unsafe skill can expose a stable personal pattern across many interactions.

Why it matters

The paper reports that persona-skill risks persist across three frontier agents and across different distillation protocols. Existing defenses show limited effectiveness, and their success depends on the distillation strategy. They also fail to generalize well across different kinds of risks.

This points to a gap in current AI safety practice. Defenses built for individual records or retrieval-based memory may not be enough when personal history is converted into an executable capability. Protecting future personalized agents may require mechanisms for privacy preservation, authenticity awareness, provenance tracking, and careful control over when and how persona skills are reused.

AntiSkillBench is valuable because it turns an emerging concern into a measurable evaluation problem. As agent systems become more personal, benchmarks like this can help developers test whether personalization is useful without becoming a channel for privacy leakage or behavioral impersonation.

Source: Hugging Face Daily Papers