Persona Vectors Reveal What Open-Weight LLMs Express, Suppress, and Resist

Lead

Prompting a language model can show what it is willing to say in a particular interaction, but it does not necessarily reveal how its behaviors are internally organized. The arXiv paper “What Models Express, Suppress, and Resist: Auditing Open-Weight LLMs with Persona Vectors” tackles this gap by using persona vectors—behavioral directions in activation space—to examine open-weight models more systematically.

The central idea is straightforward but important: post-training shapes what a model tends to do, avoid, or refuse, yet those tendencies may not be visible through ordinary prompts alone. By probing activation space, the authors attempt to distinguish traits that are genuinely default, traits that are present but subdued, and traits that remain resistant to standard extraction.

Key points

• A broader trait inventory: Instead of focusing on a handful of behaviors, the study compiles 53 traits across four behaviorally distinct domains and applies the framework to two open-weight models.
• Three behavioral categories: Each trait is labeled as natural, meaning expressed at baseline; steerable, meaning latent but amplifiable; or intractable, meaning resistant to standard extraction methods.
• Helpful defaults dominate: Both models default to helpful, task-oriented behavior. The paper reports that all nine agentic traits are natural under the study’s labeling scheme.
• Clinical behavior aligns with desirability judgments: In the clinician setting, the models’ default behavior matches an independent board-certified psychologist’s desirability judgments on 16 of 17 traits.
• Steering reveals suppressed tendencies: The largest gains from steering appear for traits excluded by the models’ defaults, including hyperbole, hallucination, and sycophancy. This suggests that some undesirable behaviors may be suppressed rather than absent.
• Trait composition is asymmetric: Across 171 generic-trait pairs, two steerable traits can collapse the composition, while pairs involving a default trait do not show the same effect.
• Intractable traits may still be recoverable: For a trait such as “evil,” standard extraction fails, but a vector transferred from a fine-tuned variant can still recover it. The remaining refusals appear inside the model’s chain-of-thought.

Why it matters

The paper’s contribution is not merely a new way to steer model outputs. Its stronger implication is methodological: persona vectors can act as probes for the hidden structure of model behavior. They help separate what a model naturally expresses from what it suppresses and what it resists.

For AI safety and evaluation, that distinction matters. A prompt-based benchmark may conclude that a model does not exhibit a behavior, when the behavior is actually latent and can be amplified through activation-level intervention. Conversely, some traits may be structurally harder to extract, indicating stronger resistance from the post-training process.

The authors therefore position persona vectors less as reliable control mechanisms and more as diagnostic instruments. They provide a way to draw a behavioral map of open-weight models: where helpfulness is the default, where risky tendencies remain accessible, and where refusal mechanisms still leave traces. As open-weight systems become more widely used, this kind of internal behavioral audit could become an important complement to conventional evaluation.

Source: arXiv