Persona skills distill personal interaction histories into portable and executable artifacts for downstream agents. While enabling flexible personalization, this process concentrates fragmented personal signals, amplifies their impact through reuse, and challenges defenses designed for individual records or retrieval-based memory. To systematically investigate the safety of the persona-skill pipeline, we introduce AntiSkillBench, an end-to-end benchmark for evaluating risks and defenses across the persona-skill pipeline. It comprises: (i) a dataset of 7,500 persona-grounded dialogue traces, constructed from 50 behaviorally rich profiles spanning diverse task scenarios; (ii) an evaluation suite that measures skill-level privacy leakage and agent-level attribute disclosure and behavioral impersonation across three skill-distillation strategies; and (iii) a defense evaluation covering four configurations across online and post-hoc interventions, including active risk suppression and passive provenance protection. Experiments across three frontier agents show that persona-skill risks persist across agent backbones and distillation protocols, extending from explicit attributes to communication styles and personality traits. Existing defenses exhibit limited and distillation-dependent effectiveness, failing to generalize across risk and distillation strategies. These results highlight AntiSkillBench as a challenging benchmark for developing privacy-preserving and authenticity-aware persona skills.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a study titled "When Agents Learn to Be You." Sam, what is this research investigating?
Sam: So you know how AI assistants like chatbots get more useful the more you talk to them? They start to learn your preferences, your way of phrasing things, what topics matter to you. This paper looks at a new development where all of that learning gets compressed into a single portable file—the researchers call it a "persona skill." The idea is that you could hand this file to any new AI system, and it would instantly know how to act like you.
Alex: That sounds convenient. But I'm guessing there's a catch?
Sam: A significant one. The paper's central question is whether these files are actually safe. Because here's the thing—these files don't just store simple facts like your name or your birthday. They capture the subtle, almost invisible patterns of how you think: the kinds of arguments you make, the way you respond when you're frustrated, the topics you keep coming back to. In other words, you aren't just creating a helpful assistant. You are creating something much closer to a digital clone.
Alex: So the risk isn't just "the AI knows my password." It's that the AI has learned to be me.
Sam: Exactly. And that's a much harder problem to contain. To investigate it properly, the research team built a testing framework called AntiSkillBench. They created thousands of simulated conversations across fifty different user profiles, then used those conversations to train persona skills for each profile. The goal was to see whether an attacker could use those files to either extract private information or convincingly pretend to be the original user.
Alex: Walk me through what those two threats actually look like in practice.
Sam: Sure. The first threat is what they call "skill-level privacy leakage." Imagine someone gets hold of your persona file and just queries it directly—asking it questions to fish out sensitive details you mentioned in past conversations. The second threat is more subtle. It's called "agent-level impersonation." Here, the attacker doesn't just want your data—they want to become you. They deploy the persona skill and use it to interact with your contacts, your colleagues, maybe your bank, in a way that sounds convincingly like you wrote it.
Alex: How do you even measure whether an AI is successfully impersonating someone? That seems hard to quantify.
Sam: It's a genuinely tricky measurement problem. The researchers developed a metric they call VocabGain. Rather than just checking whether the AI can repeat facts about you, they put the agent into scenarios where you—the real you—would have a distinctive, recognizable way of responding. Then they measure how closely the agent's output matches your actual style. Think of it like a fingerprint check, but for your writing voice rather than your hand. The higher the VocabGain score, the more of your personal fingerprint has been successfully copied.
Alex: And what did they actually find when they ran these tests?
Sam: The results were notable, and somewhat sobering. The privacy risks turned out to be structural—meaning they aren't just a bug you can patch. They're a byproduct of how these persona skills are built in the first place. The researchers tested a defense called "Privacy Sanitization," which works by scrubbing the file of obvious identifiers: names, addresses, that kind of thing. It helped, but only partially.
Alex: So you clean out the obvious clues, and the clone still works?
Sam: That's the core finding. The sanitization could stop the AI from directly stating your home address. But it couldn't stop the AI from mimicking the way you construct an argument, or the particular phrases you reach for when you're making a point. Your identity isn't stored in the file like a label on a jar—it's woven into the entire pattern of how the AI learned to behave. You can't scrub it out without destroying the thing that makes the skill useful in the first place.
Alex: So is there anything that actually works as a defense?
Sam: The paper suggests one more promising approach: something called "Adversarial Obfuscation." Instead of just removing information, this method actively feeds the AI false or conflicting signals during training—essentially muddying the waters so the resulting persona is less of a clean copy. It's more effective than simple scrubbing, but the researchers are careful not to present it as a complete solution. The underlying tension remains. As long as we want AI to be deeply personalized, we are training it to be a mirror. And a mirror, by design, reflects everything it sees.
Alex: That's a genuinely uncomfortable trade-off to sit with. The more useful the tool, the more of yourself you've handed over to it.
Sam: That's precisely the dilemma the paper leaves us with. And it's worth noting that this isn't a distant hypothetical—persona skills are already being developed and deployed. The researchers are essentially saying: we need to think carefully about the privacy architecture of these systems before they become widespread, not after.
Alex: It's one of those cases where the technology moves faster than our understanding of its risks. Thanks for walking us through it, Sam. And thanks to everyone listening to ResearchPod.