Study guide · Safety & security · 2025

The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence

Tom Wollschläger et al. · ICML 2025

A 2024 paper found that a chat model's refusals hinge on a single direction inside the network. This ICML 2025 paper asks whether that is the whole story, and finds it is not: by searching with gradient descent instead of taking an average, the authors find many directions that each control refusal, whole multi-dimensional "cones" of them, and several that work independently of one another.

The paper in four sentences

Earlier work found one "refusal direction" by subtracting the model's average internal state on harmless requests from its average on harmful ones.

This paper instead searches for refusal directions with an optimiser, asking for directions that switch refusal on when added and off when removed, while leaving everything else alone.

It finds not one direction but whole regions, called concept cones, of up to five dimensions, in which every direction controls refusal; larger models have roomier cones.

Being at right angles is not enough to make two directions truly separate, so the authors define representational independence and show there are several refusal directions that do not influence each other at all.

Idea · §1–2

Where we started: one arrow

A language model keeps, for each word it reads, a long list of numbers that can be drawn as an arrow in a space with thousands of dimensions. Many ideas correspond to directions in that space. Arditi et al. (2024) found a direction tied to refusal: remove it and the model stops refusing harmful requests; add it and the model refuses harmless ones.

Their direction came from simple averaging: the average arrow on harmful requests minus the average arrow on harmless ones. That is a quick estimate, but it answers a narrow question ("how do these two groups differ on average?") rather than the one we care about ("which directions actually control refusal?"). It might pick up other differences between the groups too, and it can only ever give one answer.

Everyday pictureTo find what makes a dish spicy, you could compare the average recipe of spicy and mild dishes. You would get one answer, mostly chilli, but also some garlic that happens to co-occur. Testing ingredients one by one would reveal that chilli, black pepper and wasabi each make it hot, in different ways.

TakeawayThe single refusal direction came from averaging. That is a good first guess but cannot reveal whether other directions also control refusal.

Idea · §4 · Def. 4.1

Searching instead of averaging

The authors' method, Refusal Direction Optimization (RDO), treats the direction as something to be learned. A candidate direction is scored on three things at once, and nudged step by step (gradient descent, the same process used to train networks) to improve the score:

  • When the direction is removed, the model should answer harmful requests rather than refuse.
  • When the direction is added, the model should refuse harmless requests.
  • When it is removed, the model's behaviour on harmless requests should not change. This is the "do no collateral damage" term.

The resulting direction controls refusal about as well as the averaged one, and causes less collateral damage: on TruthfulQA, a test of not repeating common misconceptions, models edited with the learned direction lost less accuracy than models edited with the averaged one.

TakeawayAsking directly for a direction that causes refusal, and nothing else, gives a cleaner direction than averaging.

Check yourself
Why is the "no change on harmless requests" term needed?
Without it, the optimiser could find a direction whose removal simply breaks the model, which also stops refusals. The term forces the direction to be specific to refusal.

Idea · §5 · Figs. 4, 17, 19

Not a line but a cone

Once you can search, you can ask for more: not one direction but a whole region in which every direction controls refusal. The authors look for a cone: pick a few arrows at right angles to each other and consider every mixture of them with non-negative amounts. Each arrow in that wedge-shaped region should, by itself, switch refusal on and off.

They find such cones with up to five dimensions in every model they tested (Qwen 2.5, Gemma 2 and Llama-3, from 1.5 to 14 billion parameters). The cones work less well as they get bigger, and they degrade faster in small models. The authors' explanation: a bigger model's internal arrows have more numbers (5,120 for Qwen 2.5 14B against 1,536 for the 1.5B), so there is more room for distinct refusal directions.

Remove direction:

Illustrative toy, not the paper's model. Here the "harmful" signal is carried by five separate directions with different strengths, and the toy refuses according to their total. Removing the strongest one (as the 2024 single-direction edit would) cuts refusal a lot; removing more cuts it further, with diminishing returns. The paper reports the same qualitative pattern for real models when several independent directions are removed together (Fig. 8).
Everyday pictureA building with several fire exits. Blocking the main exit (the old single direction) slows people down a lot, but they still get out through the side doors. To understand the building's safety you need the floor plan, not just the main door.

TakeawayRefusal is controlled by a multi-dimensional region of directions, not a single line, and bigger models have room for larger regions.

Check yourself
Why does the cone only allow non-negative mixtures of its arrows?
So every direction in it pushes towards refusal. Allowing negative amounts would include directions that point the opposite way, which would suppress refusal instead.

Idea · §6.1 · Def. 6.1 · Figs. 6–8

At right angles is not the same as independent

Two directions at right angles ("orthogonal") look unrelated on paper. But a network has many layers with non-linear steps in between. Removing one direction at an early layer can change what later layers compute, including how strongly they express the other direction. The authors show exactly this: removing a learned direction that is at right angles to the averaged one still weakens the averaged direction in later layers.

So they define a stricter test, representational independence: removing direction A must leave the model's expression of direction B unchanged at every layer, and vice versa. They then search for refusal directions that pass this test with respect to the averaged direction and to each other, and find several. Each one still controls refusal on its own, yet none affects the others.

Illustrative toy. A and B are at right angles. Each later layer adds a little B on top of whatever is there, plus an amount proportional to the (non-negative part of) A, controlled by the slider. Remove A and watch B shrink at layers 2 and 3 even though A and B are orthogonal. Set the slider to 0 and the two become representationally independent: removing A leaves B untouched.

TakeawayBecause layers interact non-linearly, directions can be at right angles yet still influence each other; the paper finds refusal directions that are truly independent.

Check yourself
In the toy, why does removing A shrink B at later layers?
Layer 2 computes part of B from A. Removing A at layer 1 removes that input, so less B gets written later, despite A and B being orthogonal at every layer.

Idea · §6.2, §7 · Fig. 9

Why it matters

For people trying to understand models, the lesson is that a single direction was a useful simplification but not the full mechanism. Refusal is spread over several directions that can each do the job, which suggests that an attack need not go through the original direction.

The authors then optimise adversarial "gibberish suffixes" with GCG, adding an extra term to the optimisation that rewards suppressing one of the newly found independent directions. These suffixes jailbreak the model 36% of the time, about as often as removing that direction does, so an attack written into the prompt can work through a direction other than the original one. (The suppression was designed in, not discovered: the point is that it is achievable from the input.)

For people trying to make models safer, the result cuts both ways: removing one direction is not the end of refusal, but a determined editor with the weights can find and remove several. Later work, such as training models to explain their refusals, tries to make refusal harder to remove.

Later refinementThis paper refines Arditi et al. (2024): "refusal is one direction" becomes "refusal is a cone, with a dominant direction and several independent ones". The authors note their directions were all optimised against the same target answers, so even more varied mechanisms may exist.

TakeawayRefusal has more dimensions than first thought, which changes how we should picture both attacks and defences.

Follow the idea

Built on: Abliteration (2024)