Inside RANA: refusal ablation and its tradeoffs

RANA is our edited version of Qwen3.8-27B, released with open weights as Qwen3.8-27B-RANA-abliterated for red-teaming and refusal-behavior research. We changed the weights themselves, instead of asking the model to behave differently through a jailbreak prompt.

Getting a model to stop refusing is only part of the job. The harder question is what the edit costs, whether in reasoning and coding capability, vision, tool use, or answers that run out of room before they finish. To measure those tradeoffs, we evaluated RANA against the base model and the orcarouter/Qwen3.8-27B-Uncensored reference, which we call orca below.

Compared with orca, RANA answered more of the held-out requests under both judges, ran out of tokens less often at the same budget, and stayed closer to the base model. Below we explain how we measured that, what the edit changed and cost, and which build to run. The BF16, FP8, GGUF, and MLX weights are available under Apache-2.0.

How RANA compares with base and orca

We ran all three models through the same harness with thinking enabled. For refusal testing we used 200 HarmBench behaviors and 240 held-out harmful prompts, kept apart from the prompts we used to build and choose the edit. Every prompt ran with three sampling seeds and a 16,384-token generation budget. Two LLM judges, gpt-oss-safeguard-20b and Mistral-Small-24B-Instruct-2501, read and labeled each response.

"Answered" means the judge found the harmful benchmark request substantively fulfilled. It is not an accuracy score. The judges disagree most on hedged and partial answers (soft refusals), so we report their results separately. Both favor RANA on the held-out set, while on HarmBench they disagree about which abliteration answers more often.

KL divergence measures how far the model's first-token probabilities moved from the base model's, on 100 harmless prompts. Lower means closer to the base model under that protocol, but closer is not automatically better on every task. The budget row counts responses that reached the token limit before they finished. The base model's rate is low because it usually refuses these harmful requests early.

Both abliterations recorded zero hard refusals (outright refusals) on HarmBench under both judges, after the same mechanical re-judging procedure. Each model had two raw gpt-oss hard-refusal labels across the three seeds, and a second pass with a fixed auditor reminder relabeled them. The raw results and the procedure are published together. That zero comes from these prompts and this judging policy, so it isn't a guarantee for every possible prompt.

What changed in the weights

RANA stands for Reasoning-Anchored, Norm-preserving Ablation. It removes a direction associated with refusal from the weights that write to the residual stream, the running representation that every layer reads from and writes to. It then rescales each edited weight vector back to its original length, or norm, which is what norm-preserving means here. The approach builds on refusal-direction research by Arditi and colleagues and on norm-preserving abliteration by Jim Lai, also known as grimjim.

In the BF16 release we changed 131 weight matrices in the attention outputs, MLP outputs, and token embeddings. That count includes two corresponding matrices in the multi-token prediction (MTP) head, which drafts tokens ahead so the server can check several at once (speculative decoding). All 333 vision-tower tensors remain byte-identical to the base model.

We ran functional checks on the published BF16 shards, and vision input, three-turn tool calling, and MTP speculative decoding all passed. These checks show that those paths work. Unchanged vision weights alone would not prove unchanged behavior on every image task.

The weights and evaluation records are public, but the exact intervention recipe and code are not. That makes this an open-weight release rather than a fully reproducible training or editing pipeline.

What the edit cost

We measured capability on MMLU-Pro, GSM8K, IFEval, TruthfulQA, and HumanEval, scoring TruthfulQA's MC1 and MC2 together as one task. The average in the table above is the mean absolute movement from base, so a gain adds to it just as a loss does. The number after each ± sign below is the standard error, the noise in that measurement.

RANA's clearest loss against base was about 1.5 percentage points on TruthfulQA. Its HumanEval change was -0.4 ± 0.9 points, within the measured uncertainty. Against orca, RANA was worse at instruction following, scoring 2.2 ± 0.9 points lower on IFEval. We ran MMLU-Pro on a 700-question subset of the full benchmark.

The release also had rejection criteria. An earlier candidate lost 2.24 points on HumanEval, and we did not release it. The final build passed the published gates, which include a mean absolute capability change of at most one point, no task below minus two points, and checks on the edited tensors and runtime functions.

Why the token budget matters

When a response stops at the token limit, that alone doesn't tell you whether the model refused, looped, or simply needed more room. Most of RANA's HarmBench budget hits were long reasoning traces, with no more repetition than normal output.

In a follow-up on the FP8 build, we re-ran all 59 prompts that hit the 16k budget in the first seed, this time allowing 64k output tokens. Of those, 57 finished, and the gpt-oss judge marked 55 as answering the request.

Our standard evaluation used a 20,480-token serving limit. The BF16 configuration declares 262,144 positions, but those evaluations don't validate the full context window. For long technical requests, adjust both context capacity and output budget as the relevant model card describes.

Choosing a build

The sizes are for the weight files alone. When you plan RAM or VRAM, allow for runtime memory and any separate vision or MTP files.

Evaluation coverage differs by build. FP8 has three-seed refusal testing with both judges. For GGUF, the functional and refusal checks cover Q4_K_M only, with one refusal-testing seed. We ran MLX's recorded checks on Linux with the CUDA backend rather than on a Mac, and they did not re-measure refusal behavior.

For MLX, download one precision folder and load it by local path. If you pass the whole repository ID to mlx-vlm, it can download every precision, roughly 160 GB. Each model card has the commands and file layout for its build.

Using RANA and checking results

RANA is a safety-alignment-removed research model, so if you put it in front of end users, moderation needs to live outside the model. Reduced refusal does not establish greater accuracy or safety, and the Apache-2.0 terms inherited from Qwen still apply.

To check our work, start with the model directory, the full model card, and the recorded results. Beyond the headline numbers, compare the released weights on your own workload.

Abliterant Chat, our hosted chat, is a separate product. These benchmarks describe the released weights, not the replies you get in Chat.