AlexsJones/llmfit

What it does

llmfit answers a narrow question before you download anything: which open-weight models will run comfortably on the machine in front of you. It probes CPU cores, system RAM, available GPUs and VRAM, and the accelerator backend (NVIDIA CUDA, Apple Silicon unified memory, AMD ROCm, Intel OneAPI), then scores every model in its catalog on memory fit, estimated speed, quality and context length README.

The tool ships in two interface shapes: an interactive TUI, opened by running llmfit with no arguments, and a classic CLI for scripting. There is also a headless web dashboard plus a REST API (/api/v1/system, /api/v1/models) for wiring recommendations into deployment pipelines.

Speed figures come from a memory-bandwidth model calibrated with runtime sampling and community benchmarks. llmfit info "<model>" prints the inputs behind an estimate so you can check the assumption rather than trust the number How it works.

Install and first run

Prebuilt packages exist for the main platforms. On macOS or Linux:

brew install AlexsJones/llmfit/llmfit

On Windows, use winget install --exact --id AlexsJones.llmfit or scoop install llmfit. There is also a Python distribution (uv tool install -U llmfit), a Docker image at ghcr.io/alexsjones/llmfit, and release binaries Install section. The convenience script curl -fsSL https://llmfit.axjns.dev/install.sh | sh pipes a shell script straight into a shell; reading it first, or using a package manager, is the more conservative route.

Launch the TUI:

llmfit

Your hardware appears at the top, models below ranked by score. / filters by name, family or quantization, and b opens community benchmark data. For scriptable output, use the non-interactive commands:

llmfit recommend --json   # top picks as JSON
llmfit fit                # full ranked table
llmfit doctor             # hardware detection report

Build from source with git clone, cd llmfit, then cargo build --release; the binary lands at target/release/llmfit.

Constraints and caveats

Recommendation quality is bounded by hardware detection and the catalog. One Hacker News reader with a 128 GB M4 MacBook Pro reported the tool proposing Qwen 2.5 and StarCoder 2 as top matches, which is a reminder that catalog currency matters for newer models HN comment. Estimates are memory-bandwidth approximations, not measurements; real throughput depends on context length, KV cache behavior and backend, so llmfit bench against a running provider is the way to replace an estimate with a number.

Cluster mode (Ray / multi-node) is explicitly marked preview and had only been exercised against a mock Ray dashboard at the time of writing Cluster guide. The tool reads configuration and hardware state; it contacts external services only for features you invoke, such as model downloads or the community leaderboard. License is MIT.

HotRepo's take

On the all-language daily list the project moved from roughly 30k stars in late July 2026 to 38,870 by 10 November, with daily gains in the 60–260 range across the recorded snapshots, and it reached 11th place on the Rust daily list that November day. The March 2026 Hacker News thread drew 301 points and 70 comments, and the discussion splits usefully: one reader wanted a website instead of a binary, and the reply pointed out that hardware probing is the whole reason it cannot be a pure web page thread. A separate comment lays out the arithmetic tok_per_s = memory_bandwidth / llm_size and notes MoE speed depends on active parameters, not total size — the same reasoning the estimator encodes. The value is in doing that arithmetic automatically across hundreds of models and quantizations, which is why the benchmark-sharing loop, letting measured numbers replace estimates for identical hardware, is the most interesting part of the design.