nace-ai/kapitsa

Kapitsa is worth a look if you want a capable coding model answering locally on a Mac and want the strongest decode speed the authors could measure. It is not a general-purpose framework: the model catalog and the hardware target are narrow, and that is the point.

The problem

A developer running a coding agent such as opencode, Claude Code, Codex or pi normally either pays a cloud provider or loads local weights through a generic engine that treats the Mac as one of many targets. The friction is choosing weights, getting them onto the machine, then pointing the agent at a server that speaks the right API. Kapitsa collapses that into "start the agent on a local model": it pulls the catalog model, serves OpenAI and Anthropic-compatible endpoints at a local address, and launches the agent against it. The result is an agent session where tokens are generated on the laptop instead of a remote GPU.

What's different

The technical story is KFlash, a speculative decoding variant. Instead of the drafter proposing one linear chain and the big model accepting it up to the first disagreement, KFlash has the model check a best-first tree of the drafter's top candidates in a single pass and keeps the longest branch it agrees with. The author's illustration claims identical output to plain decoding and 8-17% more tokens per accepted pass on short prompts, with tree size tuned per Mac.

The engine is hand-written Metal with Neural Engine help for long-prompt reading on some models, and keeps up to four conversations resident. For Flint on 16-24 GB machines, the unusual move is keeping routed experts at their stored 2-bit precision while requantizing the rest to 3-6 bits, which the README says gets 61 tok/s decode in 11.6 GB on an M1 Pro. These are the authors' self-reported, self-designed benchmarks; treat the headline tables as a starting hypothesis to reproduce with kapitsa bench compare, not as independent proof.

The trade-off is breadth. A Qwen and Nemotron plus Flint catalog is far smaller than what llama.cpp or Ollama run, and the optimizations are tied to specific weight layouts and Apple GPUs.

Who should try it

Good fit: a Mac-based developer who mostly uses one of the named coding agents, has a supported model sized to their RAM (the README suggests 9b or 4b on 16-24 GB), and wants local inference without configuring a server by hand. The kapitsa doctor command, which reports GPU, memory and which models fit, is a sensible first check for that user. Poor fit: someone who needs arbitrary GGUF or non-Qwen/Nemotron models, or who runs on Linux, Windows or NVIDIA hardware, because the entire design is Apple-silicon-only and the catalog is curated. A user who needs a model family outside the catalog should look at a broader engine first.

Requirements and limitations

Hardware is not negotiable: an M1-or-newer Mac running macOS 14 Sonoma or later, installed via Homebrew or the install script. Memory decides the model: 16-24 GB machines are steered to 9b or 4b, and the Flint numbers depend on 16-24 GB. Model choice is effectively a whitelist, so a team standardized on other weights cannot just point Kapitsa at them. Because the engine is new and highly optimized, keeping it current matters more than with a mature general runner; the catalog and per-Mac tree sizes may change between releases. The project offers release notifications via GitHub watch, which is the practical way to track that upkeep.

HotRepo's take

HotRepo recorded Kapitsa at #12 on the NEW-REPOSITORY board (cpp) with 427 total stars on 2026-10-25, across a two-date sample starting 2026-10-24. That is a snapshot of a 9-day-old repo, not a growth series; whether interest persists cannot be judged from this coverage. Before adopting, verify that your exact Mac and RAM tier appear in the per-model benchmark tables and that the weights you want are in the catalog.