p-e-w/heretic

Heretic is worth a look if you already run open-weight transformers locally and want a reproducible way to strip refusal behavior instead of hand-tuning abliteration scripts. It is not a magic filter: the output model still needs to be evaluated, and the method works on specific transformer component types.

The problem

Someone wants a downloaded instruct model to answer prompts its alignment layer refuses. The usual route is to find an abliteration script, adapt its layer selection, pick a harmful/harmless prompt pair, and iterate manually until refusals drop without wrecking general answers. That loop is fiddly and hard to compare across runs. Heretic changes the middle of that workflow: you name a Hugging Face model, it computes residual directions from contrast prompts, and it optimizes ablation parameters toward a stated refusal/quality objective. The result is a modified checkpoint rather than a patch script you maintain.

What's different

Heretic implements a parametrized variant of directional ablation. For attention out-projection and MLP down-projection matrices, it orthogonalizes weights against a per-layer residual direction computed as a difference-of-means between first-token residuals for "harmful" and "harmless" prompts. Several ablation parameters are optimizable rather than fixed. That is the real change: the hard search is automated, and the run starts by benchmarking the system to choose batch size, so the same procedure can be repeated on different hardware. The README describes the process as fully automatic with no configuration required, while exposing a config file and CLI options for control.

The trade-off is conceptual and practical. The harmful/harmless contrast set is a proxy for refusal behavior, not a definition of what should be safe. Public discussion of Heretic's prompt source showed that the "harmful" examples include genuinely severe material alongside tamer hacking and propaganda requests comment on the harmful-behaviors dataset. Tweaking abliteration to defeat refusal on such prompts does not imply the resulting model discriminates good from bad requests. One commenter also linked a post-abliteration chat with a small model that they described as "utterly broken" commenter's example; that is a single unverified report, but it is the kind of regression you should actively check for.

Who should try it

Good fit: an engineer or researcher with a supported GPU and a specific open model in mind, who is comfortable running Python tooling and evaluating outputs. If you are comparing abliteration techniques, Heretic's parameter search and automatic setup reduce the setup overhead per run. Poor fit: anyone expecting a drop-in safety guarantee, or someone who wants to use it on proprietary API models. This operates on model weights you can load locally; it is not an API-level workaround. If you mainly need a cleaner model for a narrow domain, a fine-tune may be more predictable than modifying refusal behavior globally.

Requirements and limitations

You need Python 3.10+ and PyTorch 2.2+, with a PyTorch build appropriate to your hardware; some models need newer PyTorch, such as MXFP4-quantized gpt-oss requiring torch.accelerator from 2.6. The README points to uv and a pinned uv.lock for dependency consistency, which is useful because transformer internals and quantization support are sensitive to version drift. Expect GPU memory to be the binding constraint: you are loading and modifying the full model, not running a tiny adapter. The README's RTX 3090 timing example is cut off, so do not assume a fixed duration for your model or card.

The method targets transformer component types (attention out-projection and MLP down-projection), so architecture coverage is not universal. AGPL-3.0 also matters if you plan to redistribute a modified pipeline or build a service around it. Finally, there is no supplied independent benchmark of how much general capability survives; the optimization objective and your own evaluation set are where that question gets answered.

HotRepo's take

HotRepo's tracked daily-board history is too sparse to judge persistence: four observed days between 2026-08-30 and 2026-10-28, including 28,982 stars on 2026-08-30 and 36,889 on 2026-10-28. The 2025-11-16 Hacker News thread drew 745 points and 380 comments, which shows discussion volume, not consensus. Before trusting a run, check how the project's refusal/quality metrics are configured and verify that your target model architecture is supported by the ablation component list in the project README.