[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"$f3ithj61ttcpol":3},{"code":4,"message":5,"data":6,"timestamp":32},10000,"success",{"full_name":7,"position":8,"description":9,"language":10,"stars":11,"forks":12,"stars_gained":13,"age_days":14,"hn_threads":15,"one_liner":22,"summary_md":23,"topic":24,"github_topics":27,"license":28,"homepage":29,"updated_at":30,"history":31},"nace-ai\u002Fkapitsa",12,"Blazing fast Apple Silicon optimized inference engine built for Qwen & Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai","C++",439,63,47,9,[16],{"id":17,"title":18,"points":19,"comments":20,"posted_at":21},50218893,"Apple Silicon Inference Engine: Kapitsa",1,0,"2026-10-24","A local inference engine for running Qwen, Nemotron and Flint models on Apple silicon Macs, aimed at coding agents.","Kapitsa is worth a look if you want a capable coding model answering locally on a Mac and want the strongest decode speed the authors could measure. It is not a general-purpose framework: the model catalog and the hardware target are narrow, and that is the point.\n\n## The problem\n\nA developer running a coding agent such as opencode, Claude Code, Codex or pi normally either pays a cloud provider or loads local weights through a generic engine that treats the Mac as one of many targets. The friction is choosing weights, getting them onto the machine, then pointing the agent at a server that speaks the right API. Kapitsa collapses that into \"start the agent on a local model\": it pulls the catalog model, serves OpenAI and Anthropic-compatible endpoints at a local address, and launches the agent against it. The result is an agent session where tokens are generated on the laptop instead of a remote GPU.\n\n## What's different\n\nThe technical story is [KFlash](https:\u002F\u002Fgithub.com\u002Fnace-ai\u002Fkapitsa\u002Fblob\u002Fmain\u002Fdocs\u002Fkflash.md), a speculative decoding variant. Instead of the drafter proposing one linear chain and the big model accepting it up to the first disagreement, KFlash has the model check a best-first tree of the drafter's top candidates in a single pass and keeps the longest branch it agrees with. The author's illustration claims identical output to plain decoding and 8-17% more tokens per accepted pass on short prompts, with tree size tuned per Mac.\n\nThe engine is hand-written Metal with Neural Engine help for long-prompt reading on some models, and keeps up to four conversations resident. For Flint on 16-24 GB machines, the unusual move is keeping routed experts at their stored 2-bit precision while requantizing the rest to 3-6 bits, which the README says gets 61 tok\u002Fs decode in 11.6 GB on an M1 Pro. These are the authors' self-reported, self-designed benchmarks; treat the headline tables as a starting hypothesis to reproduce with `kapitsa bench compare`, not as independent proof.\n\nThe trade-off is breadth. A Qwen and Nemotron plus Flint catalog is far smaller than what llama.cpp or Ollama run, and the optimizations are tied to specific weight layouts and Apple GPUs.\n\n## Who should try it\n\nGood fit: a Mac-based developer who mostly uses one of the named coding agents, has a supported model sized to their RAM (the README suggests 9b or 4b on 16-24 GB), and wants local inference without configuring a server by hand. The `kapitsa doctor` command, which reports GPU, memory and which models fit, is a sensible first check for that user. Poor fit: someone who needs arbitrary GGUF or non-Qwen\u002FNemotron models, or who runs on Linux, Windows or NVIDIA hardware, because the entire design is Apple-silicon-only and the catalog is curated. A user who needs a model family outside the catalog should look at a broader engine first.\n\n## Requirements and limitations\n\nHardware is not negotiable: an M1-or-newer Mac running macOS 14 Sonoma or later, installed via Homebrew or the install script. Memory decides the model: 16-24 GB machines are steered to 9b or 4b, and the Flint numbers depend on 16-24 GB. Model choice is effectively a whitelist, so a team standardized on other weights cannot just point Kapitsa at them. Because the engine is new and highly optimized, keeping it current matters more than with a mature general runner; the catalog and per-Mac tree sizes may change between releases. The project offers release notifications via GitHub watch, which is the practical way to track that upkeep.\n\n## HotRepo's take\n\nHotRepo recorded Kapitsa at #12 on the NEW-REPOSITORY board (cpp) with 427 total stars on 2026-10-25, across a two-date sample starting 2026-10-24. That is a snapshot of a 9-day-old repo, not a growth series; whether interest persists cannot be judged from this coverage. Before adopting, verify that your exact Mac and RAM tier appear in the per-model benchmark tables and that the weights you want are in the catalog.",{"slug":25,"name":26},"machine-learning","Machine Learning",[],"Apache-2.0","https:\u002F\u002Fx.com\u002Fkapitsaserve","2026-10-25T02:59:58Z",[],1792918411]