Matan Avitan, Yoav Goldberg, Yanai Elazar
5 min
Abstract
Concept erasure aims to remove a target concept from a representation while preserving the other information encoded in it. This is difficult because representations encode many concepts that are often correlated with the erasure target, so removing the target risks damaging them. We propose the Manifold Constraint Hypothesis (MCH): if natural representations concentrate on a structured, lower-dimensional manifold, then interventions should be constrained to that manifold and better preserve other information encoded in the representation during interventions. We instantiate MCH in a new concept erasure method: MANifold aware Concept Erasure (MANCE). MANCE performs iterative updates to the representations using signals from a classifier that predicts a target concept. We estimate the manifold using representations obtained from natural inputs, and then we project the concept removal update to the estimated manifold. We perform extensive evaluation on 119 settings spanning text and vision, including 13 language models, three NLP concepts, and 40 CelebA-CLIP attributes. Employing MANCE on top of previous methods shows consistent improved leakage results. We also introduce MANCE+ and MANCE++, which prepend a closed-form erasure algorithm before employing MANCE, achieving better leakage--surgicality tradeoffs relative to matched full-space updates. MANCE++, our best method, achieves state-of-the-art results on nonlinear concept erasure. These results support MCH in the erasure setting: interventions should be constrained to the natural representation manifold.
Sam: But how do you know the erasure actually worked? How do you confirm the concept is really gone and not just hiding somewhere?
Alex: They use what they call leakage testing. After the erasure, they train a separate, small neural network — essentially a detective — whose only job is to search the model's internal representations and see if it can still find traces of the erased concept. If the detective comes up empty, the erasure held. If it finds something, the concept has leaked back through.
Sam: That's a clever check. And what's the "surgicality budget" they mention?
Alex: Think of it as a damage limit. Before you run the erasure, you set a ceiling on how much the model's other abilities are allowed to degrade. It's the paper's way of saying: you can remove this bias, but only if you don't break the model's ability to do its actual job. It forces a trade-off to be explicit rather than hidden.
Sam: So you're not just measuring whether the concept is gone — you're also measuring what you broke in the process.
Alex: Precisely. And that dual measurement is what makes the evaluation rigorous. A method that erases perfectly but destroys everything else isn't useful.
Sam: You mentioned MANCE++. What does that add?
Alex: It's a two-stage approach. The first stage uses a fast mathematical shortcut to strip away the obvious, surface-level signals of the unwanted concept — the parts that are easy to find and easy to remove. Think of it like skimming the visible oil off the top of water before you do the deeper cleaning. Once the surface is clear, the main MANCE loop focuses its energy on the subtle, entangled pockets of information that the quick pass couldn't reach.
Sam: So the first stage handles the easy work, and the second stage handles the stubborn residue.
Alex: That's the logic. The paper describes the first stage as targeting what they call second-moment signals — essentially, the subtle patterns in how data clusters together, which simpler methods tend to miss entirely. By clearing those out first, the geometric editing in the second stage becomes more precise and more efficient.
Sam: It sounds like a well-designed pipeline. What are the limitations the authors actually admit to?
Alex: They're candid about two things. First, estimating the manifold — mapping that mountain road — becomes less reliable when data is sparse or when the road curves very sharply. In those cases, the guardrail is less accurate, and edits can still drift slightly off course. Second, if the internal dimension of the model is very high, the advantage of staying on the manifold shrinks, because the "road" takes up a larger fraction of the total space anyway.
Sam: So the method is only as good as the map. If the road is too twisty or the landscape is too complex, you might still lose your footing.
Alex: That's a fair summary of the authors' own caution. MANCE represents a meaningful, geometry-aware step forward in concept erasure, but they're clear that building better manifold estimators is an open problem for future work. The shape of the data matters — and learning to read that shape more accurately is where the field needs to go next.
Sam: It's a bit like the difference between a blunt instrument and a precise tool. The tool is better, but it still depends on the skill of the person using it — or in this case, the quality of the underlying map.
Alex: Well put. And that tension — between precision and the difficulty of achieving it — is probably the most honest thing the paper has to say. Thanks for listening to ResearchPod.