The paper in four sentences
Abliteration works because standard refusals are short and nearly identical ("I'm sorry, but I can't help with that"), so the model's decision to refuse sits in one tidy direction that can be found and deleted.
The authors fine-tune models on 4,289 harmful prompts answered with extended refusals: a neutral overview of the topic, an explicit refusal, and a short ethical reason.
After abliteration, ordinary versions of three models fall to between 14% and 21% refusal on harmful questions, while the extended-refusal versions keep 91% to 97%.
The defence costs a little general ability, and more for the smallest model after it is attacked, but it shows that how a model refuses changes how easily the refusal can be removed.
Idea · §1, §3.1
The attack being defended against
A chat model is a large network whose internal state, for each word, is a long list of numbers that can be pictured as an arrow. In 2024, Arditi and colleagues found that the decision to refuse a harmful request is largely carried by one direction among those arrows. If that direction is projected out of the model's weights, a cheap one-off edit known as abliteration, the model stops refusing while staying fluent.
Anyone who can download a model's weights can perform this edit, so it is a serious problem for openly released models. Earlier defences mostly targeted attacks through the prompt, such as cleverly worded jailbreaks. This paper targets the weight edit itself.
TakeawayAbliteration removes refusals by deleting one internal direction; the question is how to make a model whose safety does not hang on one direction.
Idea · §3.2 · Figs. 1–2
Why a short "no" is fragile
The authors' key observation, in their words: "Abliteration exploits the fact that standard refusals are brief and stylistically uniform." When a model has been trained to answer every harmful request with near-identical boilerplate, the internal difference between "about to refuse" and "about to help" is concentrated: one tidy direction captures nearly all of it.
Their fix is to change the shape of the refusal. Each harmful request in their training data is answered in three parts:
- a neutral overview of the topic, saying what the thing is without giving any harmful detail;
- an explicit refusal;
- a short ethical rationale explaining why.
In the paper's own example (Fig. 1), a request about phishing gets a short description of what email spoofing is, then "I'm sorry, but I cannot provide techniques for spoofing email addresses or conducting phishing attacks", then a reason. The hypothesis is that producing this richer answer involves the model's understanding of the topic, of harm and of ethics, so the refusal gets tied to many internal features rather than to one switch.
TakeawayUniform one-line refusals concentrate the refusal decision in one direction; richer, explained refusals are hypothesised to spread it across many.
Check yourself
- Why does the overview in an extended refusal have to be "neutral"?
- It must engage with the topic without giving harmful detail; otherwise the "refusal" would itself leak what it refuses to provide.
- In the toy, why does spreading help even though the total signal is the same?
- Abliteration only removes one direction. The more evenly the signal is shared, the smaller the share that one direction carries.
Idea · §4.1–4.2 · Table 1
The result: refusals survive the cut
They fine-tune three open models (Llama-2-7B-Chat, Qwen2.5-3B-Instruct and Qwen2.5-1.5B-Instruct) on 10,000 examples: the 4,289 harmful prompts with extended refusals, plus 5,711 ordinary helpful examples so the models stay useful. Then they abliterate both the original and the fine-tuned version of each model and test on 550 harmful questions (CatQA), with another model judging whether each answer is a refusal.
The ordinary models lose 70 to 80 percentage points of refusal. The extended-refusal models lose at most about 9 points.
TakeawayAfter abliteration, standard models refuse 14–21% of harmful questions; extended-refusal models still refuse 91–97%.
Check yourself
- Before any attack, the Qwen extended-refusal models refuse 100% while the originals refuse about 93–94%. Why does that matter when reading the result?
- Part of the gap after the attack comes from a higher starting point. The fairer comparison is the size of the drop: about 3 to 9 points versus about 78 to 80.
Idea · §4.3, Limitations
What it costs
Nothing here is free. Fine-tuning on extended refusals slightly lowers general knowledge scores (MMLU) before any attack, by up to about a point. After abliteration, the defended models also lose more general ability than the undefended ones: the defended Llama-2 drops to 41.8 on MMLU from 46.4, and the defended Qwen 1.5B's perplexity (how surprised it is by ordinary text; lower is better) roughly doubles. In a sense the defence works because the attacker can no longer remove refusal cleanly: to strip it, they would have to damage the model.
The authors also list practical limits: extended refusals are wordier, which some users will find annoying; the method is not claimed to stop other jailbreaks, such as manipulating the context of a conversation; and it is meant to be combined with other safeguards, such as external moderation.
TakeawayThe defence costs a little capability, and abliterating a defended model costs the attacker more capability, which is the point.
Idea · §4.4 · Context
Where it fits
This paper turns an interpretability finding into a design principle: if safety behaviour is concentrated in one place, it can be cut out, so train it to be spread out. It fits with Wollschläger et al. (2025), who showed that refusal can already involve several directions; extended refusals seem to make use of more of them.
It is a first step rather than a complete answer. The updated version of the paper also tries seven ready-made prompt jailbreak collections, and the extended-refusal models resisted all of them better than the originals. A further test found that training on the refusal sentence alone gives no protection: the explanation and justification do the work. But only three fairly small models were tested, and a more determined attacker could remove several directions at once or simply fine-tune the safety away. Still, it shows that the style of a model's refusals is not cosmetic: it shapes how robust they are.
TakeawayHow a model is taught to say no affects how hard that "no" is to remove.
Abstract, restated
LLMs are aligned to refuse harmful instructions through safety fine-tuning. Abliteration isolates and suppresses the single latent direction most responsible for refusals, letting the model generate harmful content. The authors propose a defence that changes how models produce refusals: an extended-refusal dataset in which responses to harmful prompts give detailed justifications, distributing the refusal signal across multiple token positions and representational dimensions. Fine-tuning Llama-2-7B-Chat and Qwen2.5-Instruct (1.5B and 3B) on it, refusal rates under abliteration drop by at most 10%, versus 70–80% for the base models, and general performance is largely preserved. Abu Shairah, Hammoud, Ghanem & Turkiyyah (KAUST), arXiv:2505.19056 (v1 May 2025; this guide follows v2, 7 October 2025).
Section · §3.1 · Eqs. 1–5
Threat model: refusal-direction abliteration
The attack follows Arditi et al. (2024). For each layer l and post-instruction position i, compute mean residual activations on harmful (μ) and harmless (ν) prompts; each difference r = μ − ν is a candidate (Eqs. 1–2). The candidate that most reduces refusal is selected (Eq. 3), normalised to r̂, and projected out of each output matrix:
A detail that matters for the defence: the authors consider only directions whose abliterated model keeps judged coherence of at least 70% (§4.1, §4.3; v2 Figs. 2–3). Without such a constraint, an "attack" could reach zero refusal simply by destroying the model, which would not be a meaningful break.
Check yourself
- Why does the coherence threshold make the defence's numbers more meaningful?
- It forces the attacker to choose among directions that leave a usable model. A defended model then counts as robust if no usable abliteration removes its refusals, rather than merely because every edit breaks it.
Section · §3.2, §4.1 · Figs. 1–3
Extended-refusal data and evaluation
Data. 4,289 harmful prompts merged from BeaverTails, AdvBench, StrongREJECT and TDC-2023. Responses generated with GPT-4o in a fixed three-part structure: neutral topic overview, explicit refusal, ethical rationale. To preserve utility, 5,711 benign instruction–response pairs from Alpaca-GPT4-en are added, giving 𝒟FT of 10,000 examples (§3.2; §4.1 of v2 gives 4,268 and 5,732 instead, with the same total). Fine-tuning: 3 epochs at learning rate 1×10−6 (v2 §3.2).
Models. Llama-2-7B-Chat, Qwen2.5-3B-Instruct, Qwen2.5-1.5B-Instruct; each with an extended-refusal (ER) variant.
Metrics. Refusal score on CatQA (550 harmful questions) judged by Qwen-2.5-14B, cross-checked with Llama-Guard-3-8B (App. A). Utility: MMLU accuracy, perplexity on 1,000 C4 passages, and judged coherence.
Section · §4.2–4.3 · Table 1 · App. A
Results and their cost
Table 1 reports each model before and after abliteration. Undefended models fall by 78.0 to 80.2 points of refusal (the text summarises this as 70–80). ER models fall by 3.3 to 9.1 points. Coherence of abliterated ER models is 89.1–98.7%, against 100% for abliterated base models, a sign that the attacker must accept some damage. The Llama-Guard-3-8B cross-check (App. A) differs from the LLM judge by about 2 to 7 points: 90.4, 94.6 and 89.3 versus 92.7, 90.9 and 96.7.
The utility picture is less rosy than the paper's conclusion ("minimal" impact) suggests. Before attack, ER fine-tuning costs 0.6–1.3 MMLU points and raises perplexity by 2–2.6. After attack, the ER Llama-2 loses 4.6 MMLU points (46.4 → 41.8) and the ER Qwen 1.5B's perplexity jumps from 26.7 to 49.5, whereas abliteration moves the undefended models' MMLU by under 2 points (their perplexity does rise, most for Llama-2, from 14.1 to 18.5). Read charitably, this is the defence working: refusal is now entangled with general capability, so it cannot be removed surgically.
Section · §4.4 · Fig. 4 (v2)
Feature-space analysis
To test the "distributed signal" hypothesis, the authors take final hidden states for harmful and benign prompts, reduce them with PCA, and measure the Euclidean distance between the two class centroids, before and after abliteration. A large drop means abliteration has collapsed the distinction the model uses to decide to refuse.
Abliteration shrinks the separation by 28.7–33.9 units in standard models but only 7.7–13.7 in ER models; for Qwen 1.5B, 37.8% versus 13.4%. The authors read this as abliteration failing to collapse a representation that ER training has spread across many dimensions. Note that ER models start with a smaller separation for Qwen 3B and Llama-2 (79.6 vs 100.9, 74.6 vs 112.0), so the claim rests on robustness of the separation, not its size.
Check yourself
- Does this analysis prove the refusal signal is spread over many directions?
- No. It shows the class separation survives removing one direction, which is consistent with a distributed signal. A direct test would, for example, count how many directions must be removed to suppress refusal, as in the concept-cones paper.
Section · §4.5–4.6 · Tables 2–3 (v2)
Other jailbreaks, and which part of the refusal matters
Version 2 of the paper adds two experiments. More jailbreaks (§4.5, Table 2). Base and ER models are attacked with seven prompt-level jailbreak suites (DAN, HarmBench, three WildGuard test splits, WildJailbreak and TrustLLM's jailbreak set), with success judged by WildGuard. ER models have lower attack success on every suite; the gains are largest for the Qwen models, whose baselines were weakest (Qwen2.5-3B on WildJailbreak: 93.8% to 41%).
Component ablation (§4.6, Table 3). Qwen2.5-3B is fine-tuned on each part of the extended refusal alone and then abliterated. Training on the bare refusal sentence leaves only 17.8% refusal after abliteration, 73.1 points below the full ER model's 90.9; the explanation (topic overview) alone keeps 81.1% and the justification alone 86.2%. The protection comes from the explanatory parts, not from refusing as such, which supports the paper's account.
Check yourself
- What does the 17.8% result for the refusal-only model tell us?
- That fine-tuning on refusals alone, without explanation, leaves refusal just as removable as in the original model; the defence depends on the model producing explanatory content.
Section · Limitations · Lineage
Limitations and open questions
- Attack coverage. The weight-level attack is single-direction abliteration only; v2 adds seven prompt-level jailbreak suites (Table 2). Multi-direction ablation (e.g. removing several representationally independent directions, as in Wollschläger et al.), adaptive selection of directions on the defended model, or ordinary fine-tuning attacks are not tested.
- Scale and breadth. Three models of 1.5B–7B parameters; the component ablation (Table 3) is run on Qwen2.5-3B only.
- Reporting gaps. No comparison against other alignment-hardening defences, inconsistent dataset counts between §3.2 and §4.1 of v2, and a minor mismatch between "at most 10%" (abstract) and the 9.1-point maximum in the results.
- User experience. The authors note longer refusals may be unwelcome and suggest work on conciseness and combining with external moderation.
Reference
Glossary
- Abliteration
- Projecting a refusal direction out of a model's weights.
- Extended refusal
- Overview + explicit refusal + ethical rationale.
- CatQA
- 550 harmful questions across categories, used for the refusal score.
- Perplexity
- Exponentiated average negative log-likelihood on text; lower means the model predicts ordinary text better.
- Coherence
- Share of outputs judged coherent by an LLM judge.
- Centroid distance
- Distance between mean harmful and mean benign hidden states after PCA.