The paper in four sentences
Earlier work found one "refusal direction" by subtracting the model's average internal state on harmless requests from its average on harmful ones.
This paper instead searches for refusal directions with an optimiser, asking for directions that switch refusal on when added and off when removed, while leaving everything else alone.
It finds not one direction but whole regions, called concept cones, of up to five dimensions, in which every direction controls refusal; larger models have roomier cones.
Being at right angles is not enough to make two directions truly separate, so the authors define representational independence and show there are several refusal directions that do not influence each other at all.
Idea · §1–2
Where we started: one arrow
A language model keeps, for each word it reads, a long list of numbers that can be drawn as an arrow in a space with thousands of dimensions. Many ideas correspond to directions in that space. Arditi et al. (2024) found a direction tied to refusal: remove it and the model stops refusing harmful requests; add it and the model refuses harmless ones.
Their direction came from simple averaging: the average arrow on harmful requests minus the average arrow on harmless ones. That is a quick estimate, but it answers a narrow question ("how do these two groups differ on average?") rather than the one we care about ("which directions actually control refusal?"). It might pick up other differences between the groups too, and it can only ever give one answer.
TakeawayThe single refusal direction came from averaging. That is a good first guess but cannot reveal whether other directions also control refusal.
Idea · §4 · Def. 4.1
Searching instead of averaging
The authors' method, Refusal Direction Optimization (RDO), treats the direction as something to be learned. A candidate direction is scored on three things at once, and nudged step by step (gradient descent, the same process used to train networks) to improve the score:
- When the direction is removed, the model should answer harmful requests rather than refuse.
- When the direction is added, the model should refuse harmless requests.
- When it is removed, the model's behaviour on harmless requests should not change. This is the "do no collateral damage" term.
The resulting direction controls refusal about as well as the averaged one, and causes less collateral damage: on TruthfulQA, a test of not repeating common misconceptions, models edited with the learned direction lost less accuracy than models edited with the averaged one.
TakeawayAsking directly for a direction that causes refusal, and nothing else, gives a cleaner direction than averaging.
Check yourself
- Why is the "no change on harmless requests" term needed?
- Without it, the optimiser could find a direction whose removal simply breaks the model, which also stops refusals. The term forces the direction to be specific to refusal.
Idea · §5 · Figs. 4, 17, 19
Not a line but a cone
Once you can search, you can ask for more: not one direction but a whole region in which every direction controls refusal. The authors look for a cone: pick a few arrows at right angles to each other and consider every mixture of them with non-negative amounts. Each arrow in that wedge-shaped region should, by itself, switch refusal on and off.
They find such cones with up to five dimensions in every model they tested (Qwen 2.5, Gemma 2 and Llama-3, from 1.5 to 14 billion parameters). The cones work less well as they get bigger, and they degrade faster in small models. The authors' explanation: a bigger model's internal arrows have more numbers (5,120 for Qwen 2.5 14B against 1,536 for the 1.5B), so there is more room for distinct refusal directions.
TakeawayRefusal is controlled by a multi-dimensional region of directions, not a single line, and bigger models have room for larger regions.
Check yourself
- Why does the cone only allow non-negative mixtures of its arrows?
- So every direction in it pushes towards refusal. Allowing negative amounts would include directions that point the opposite way, which would suppress refusal instead.
Idea · §6.1 · Def. 6.1 · Figs. 6–8
At right angles is not the same as independent
Two directions at right angles ("orthogonal") look unrelated on paper. But a network has many layers with non-linear steps in between. Removing one direction at an early layer can change what later layers compute, including how strongly they express the other direction. The authors show exactly this: removing a learned direction that is at right angles to the averaged one still weakens the averaged direction in later layers.
So they define a stricter test, representational independence: removing direction A must leave the model's expression of direction B unchanged at every layer, and vice versa. They then search for refusal directions that pass this test with respect to the averaged direction and to each other, and find several. Each one still controls refusal on its own, yet none affects the others.
TakeawayBecause layers interact non-linearly, directions can be at right angles yet still influence each other; the paper finds refusal directions that are truly independent.
Check yourself
- In the toy, why does removing A shrink B at later layers?
- Layer 2 computes part of B from A. Removing A at layer 1 removes that input, so less B gets written later, despite A and B being orthogonal at every layer.
Idea · §6.2, §7 · Fig. 9
Why it matters
For people trying to understand models, the lesson is that a single direction was a useful simplification but not the full mechanism. Refusal is spread over several directions that can each do the job, which suggests that an attack need not go through the original direction.
The authors then optimise adversarial "gibberish suffixes" with GCG, adding an extra term to the optimisation that rewards suppressing one of the newly found independent directions. These suffixes jailbreak the model 36% of the time, about as often as removing that direction does, so an attack written into the prompt can work through a direction other than the original one. (The suppression was designed in, not discovered: the point is that it is achievable from the input.)
For people trying to make models safer, the result cuts both ways: removing one direction is not the end of refusal, but a determined editor with the weights can find and remove several. Later work, such as training models to explain their refusals, tries to make refusal harder to remove.
TakeawayRefusal has more dimensions than first thought, which changes how we should picture both attacks and defences.
Abstract, restated
Adversarial inputs can bypass LLM safety alignment, and prior work attributed refusal to a single direction in activation space. The authors propose a gradient-based representation-engineering method, Refusal Direction Optimization, and use it to find multiple refusal directions and multi-dimensional concept cones that mediate refusal. Since orthogonal directions can still interact under intervention through non-linear effects, they introduce representational independence, which accounts for both linear and non-linear interactions, and find mechanistically independent refusal directions. Refusal is thus governed by complex spatial structures rather than a single direction. Wollschläger, Elstner, Geisler, Cohen-Addad, Günnemann & Gasteiger, ICML 2025 (PMLR 267; arXiv:2502.17420).
Section · §2 · Eqs. 1–2 · App. A
Setup and baseline
Residual-stream activations x(l) at layer l. The baseline is the difference-in-means (DIM) direction of Arditi et al., selected across layers and post-instruction positions. Two interventions with a unit direction r̂:
Models. Qwen 2.5 Instruct 1.5B/3B/7B/14B, Gemma 2 IT 2B/9B, Llama-3 Instruct 8B. Data. SaladBench harmful prompts (1,184 train, 128 validation) and Alpaca harmless prompts. Evaluation. Attack success rate (ASR) on JailbreakBench, StrongREJECT and SORRY-Bench, judged by the fine-tuned StrongREJECT judge; side effects on TruthfulQA, ARC-Challenge, GSM8K and MMLU.
Section · §4 · Def. 4.1 · Alg. 1 · Tables 1, 3, 5
Refusal Direction Optimization (RDO)
Definition 4.1 asks a refusal direction for two properties: monotonic scaling (adding or subtracting more of it monotonically increases or decreases refusal) and surgical ablation (ablating it makes harmful prompts be answered while harmless behaviour is preserved). RDO turns these into a loss over a direction r, optimised with AdamW:
- ℒablation: cross-entropy of the ablated model on harmful prompts toward a target completion that answers (tanswer).
- ℒaddition: cross-entropy on harmless prompts, with αr̂ added at layer ladd, toward a refusal target (trefusal).
- ℒretain: KL divergence between ablated and unmodified model on harmless prompts, over tretain, limiting side effects.
Weights λabl = 1.0, λadd = 0.2, λret = 1.0 (Table 3). The targets are generated in advance (Alg. 1), so RDO needs only forward and backward passes through the frozen model, optimising a single dmodel-vector.
Results. For ablation, RDO is competitive with DIM on ASR; for activation subtraction it is better on average (Table 5). Side effects are lower: on TruthfulQA, the paper reports roughly a 40% average reduction of the accuracy lost relative to DIM (Table 1). The figure below plots the values for two models.
Section · §5 · Alg. 2 · Figs. 4, 17, 19
Refusal Cone Optimization (RCO)
An N-dimensional cone with orthonormal basis B = [b1, …, bN]:
The aim is a concept cone: every direction in ℛN should satisfy Def. 4.1. Non-negativity keeps all members pointing toward refusal; orthonormality avoids degenerate, co-linear bases. Each step samples directions uniformly on the unit sphere within the cone (rather than only the basis vectors), evaluates the RDO loss on them, takes a gradient step on B, and re-orthonormalises with Gram–Schmidt (projected gradient descent, Alg. 2).
Findings. Refusal-mediating cones exist up to dimension 5 in all tested models. Performance of sampled directions (a lower bound on cone quality) declines with dimension; for Gemma 2 2B it degrades noticeably at 5 (Fig. 17), and best-of-N sampled ASR plateaus at 4 dimensions (Fig. 19). Across Qwen 2.5 sizes, the decline with dimension is steeper for smaller models (Fig. 4), which the authors attribute to residual width (1,536 for 1.5B vs 5,120 for 14B) allowing more near-orthogonal refusal directions.
Check yourself
- Why sample directions inside the cone rather than just optimise the basis vectors?
- The claim is that every direction in the cone mediates refusal. Training only on bi could leave mixtures that fail; uniform sampling optimises the whole region and avoids bias toward the basis.
Section · §6 · Def. 6.1 · Figs. 6–9
Representational independence
Motivation (Fig. 6). An RDO direction constrained to be orthogonal to DIM (RDO⊥) still reduces DIM's expression in later layers when ablated: orthogonality in one layer's basis does not prevent causal interaction through attention and MLP non-linearities.
Definition 6.1. Directions λ and μ are representationally independent under directional ablation if for every layer l in a set 𝕃:
where the ablated activations are computed by ablating at every layer, so effects accumulate through the network. The RepInd loss adds to the RDO loss the layer-averaged squared differences of these cosines (weight 200, over the first 90% of layers). Directions are found sequentially, each required to be RepInd from DIM and from all earlier ones.
Findings. Such directions exist and each still mediates refusal (Fig. 7). Ablating the top-k RepInd directions together increases ASR monotonically in k, exceeds the DIM baseline for k ≥ 4, and shows diminishing returns (Fig. 8); ablating several DIM directions from different layers does not give this gain. In §6.2 (Fig. 9) GCG suffixes are optimised with an additional loss term rewarding ablation-like suppression of the first RepInd direction; the resulting suffixes jailbreak 36% of the time, comparable to that direction's own ablation ASR. This shows an input-space attack can operate through a direction other than DIM; it does not show that ordinary GCG suffixes do so.
Section · §7 · Lineage
Limitations and lineage
- Shared targets. All directions are optimised toward the same answer and refusal targets, which may limit how distinct the found mechanisms can be; the authors suggest diverse targets or RL with a judge reward.
- Lower bounds. Cone dimensions and the number of independent directions are what the optimiser found, not upper limits; other, undiscovered regions may mediate refusal too.
- Scale. Models up to 14B; the trend with width suggests larger models may have larger cones, but this is extrapolation.
- Independence is defined for ablation. RepInd as defined concerns ablation interventions and the measured cosine; other interventions could reveal interactions it misses.
Reference
Glossary
- DIM
- Difference-in-means direction, the 2024 baseline.
- RDO
- Refusal Direction Optimization: gradient-based search for a single direction.
- RCO
- Refusal Cone Optimization: learns an orthonormal basis of a refusal cone.
- Concept cone
- Set of non-negative combinations of basis vectors, all mediating the concept.
- RepInd
- Representational independence: ablating one direction leaves the other's cosine unchanged at every layer.
- ASR
- Attack success rate, here judged by the StrongREJECT judge.
- Gram–Schmidt
- Procedure turning a set of vectors into an orthonormal basis of the same span.