Linear activation steering has gained popularity as a simple and empirically effective way to control language model behavior. More recently, spherical steering paradigms have been proposed to address limitations of additive interventions, often motivated by the assumption that hidden-state norm does not carry concept-relevant information. In this work, we revisit this assumption through a controlled empirical study designed to disentangle the roles of angular and radial components. We show that steering methods differ mainly in how they couple two geometric effects: changing a token's angular alignment with a concept direction and changing its hidden-state norm. Across seven language models, we find that concepts are represented primarily in angular structure, supporting the motivation for spherical methods, but that norm remains important for the stability and downstream effects of steering. Our results explain why interventions with similar concept-level effects can behave differently, and suggest that activation steering should be parameterized by interpretable angular and radial components of the intervention, rather than by a single additive coefficient that entangles these two effects.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at how researchers are learning to steer the behavior of large language models—and why getting that steering wrong can cause more problems than it solves.
Sam: The technique at the center of this paper is called "activation steering." To understand it, think about what's happening inside an AI language model. As it processes text, it generates streams of internal signals—numerical patterns that represent concepts, tone, and meaning. Activation steering means reaching into those signals while the model is running and nudging them in a new direction. The goal is to shift the model's behavior—make it more cautious, more helpful, or focused on a specific topic—without retraining it from scratch.
Alex: So it's like adjusting the controls on a plane mid-flight rather than rebuilding the engine on the ground?
Sam: That's a useful way to put it. And the problem is that most current methods do this adjustment blindly. They push the signal in a new direction without accounting for the full geometry of what they're changing. The result is often a model that starts producing garbled, repetitive, or incoherent text—it loses what researchers call "fluency."
Alex: So the paper is asking: is there a more precise way to steer these models without accidentally breaking them?
Sam: Exactly. And the researchers argue that the reason previous methods fail is that they treat steering as a one-dimensional problem—just push the signal this way or that way. But the paper suggests it's actually a two-part geometric problem. Imagine a compass needle. The direction it points tells you the concept—what the model is "thinking about." But the needle also has a length, which represents the strength or energy of that signal. Previous methods only changed the direction. They ignored the length entirely.
Alex: And that's what caused the fluency problems?
Sam: That's what the evidence suggests. When you force the needle to point in a new direction but keep its length exactly the same, you're essentially cramming a new concept into a space that was calibrated for the old one. The model gets confused—like trying to pour a new ingredient into a recipe without adjusting the other quantities. The researchers found that when they allowed the signal strength to increase slightly alongside the direction change, the model's fluency held up significantly better.
Alex: So the model needs a little extra "room" to accommodate the new instruction. The length of the signal has to grow to match the new direction.
Sam: Right. In the paper's language, the direction is called the "angular component" and the strength is the "radial component." Those are just geometric terms for the two independent parts of any signal vector—think of it like describing a point on a map using both a compass bearing and a distance. The core finding is that you have to tune both, separately, to get reliable results.
Alex: How did they actually test this? What did the experiments look like?
Sam: They ran six different steering methods across seven different language models, measuring both how well the steering worked and how much the model's output quality degraded. The consistent pattern was this: methods that fixed the signal strength while changing the direction caused the most damage to fluency. Methods that allowed the strength to scale appropriately caused far less. It wasn't a marginal difference—the gap between stable and unstable outputs was meaningful, and it widened for larger models.
Alex: That last point is worth pausing on. Why would larger models be more sensitive to this?
Sam: The paper doesn't fully resolve that question, but the implication is that larger models have more finely calibrated internal signal strengths. Their signals are tuned more precisely to their training, so forcing a mismatch between direction and strength throws them off more severely. It's a bit like a more complex instrument being harder to play out of tune.
Alex: So the practical upshot is: don't just add a fixed value to the signal. Treat the direction and the strength as two separate dials that need to be adjusted together.
Sam: That's the key insight. The authors frame it as moving from a one-size-fits-all additive change to a two-parameter process. You choose your direction to encode the desired behavior, then you calibrate the strength to keep the model stable. The important thing is that stability isn't a lucky side effect—it's something you can actively control.
Alex: Is there a formula for how much to adjust the strength? Or is it still trial and error?
Sam: That's the central practical limitation the paper acknowledges. The optimal strength adjustment depends on the task and the specific model—there isn't yet a universal rule that works everywhere. It's currently an empirical process, meaning researchers have to run tests to find the right balance for each situation. The authors flag this as the logical next step: developing a way to predict the right adjustment automatically, without needing to run dozens of experiments each time.
Alex: So the paper's contribution isn't a finished tool—it's more of a framework. A clearer way of understanding why previous methods failed and what a better approach should look like.
Sam: That's a fair characterization. By separating the geometry—direction versus strength—the research gives the field a more principled foundation to build on. It reframes the problem from "how hard do we push the signal?" to "in which direction, and with how much energy?" Those sound similar, but treating them as independent variables opens up a much more precise approach to model control.
Alex: And that kind of conceptual clarity tends to be what moves a field forward. Thanks for walking through the mechanics of this, Sam. That's it for this look at activation steering and the geometry behind it. Thanks for listening to ResearchPod.