shanselman/gputaskman

What it does

gputaskman is a terminal UI ("an htop for what's inside your GPU") that goes beyond the single "15 GB used" figure in Task Manager. It ties per-process GPU memory and engine activity to loaded runtimes, mapped model files, and, for LLMs, estimated weights / KV-cache / compute buffers. It targets local GPU workloads—decision models, vision, speech, LLM inference, training, graphics, and video—on Windows (NVIDIA/AMD), native Linux and WSL (NVIDIA), and Apple Silicon macOS. Cloud, CPU-only, and NPU-only workloads are out of scope.

The README is explicit about evidence quality: device and process counters are measurements; model identification and GGUF-derived breakdowns are estimates. A loaded DLL or mapped file does not prove a model is resident on the GPU or being executed. Device view shows VRAM, GPU/memory activity, PCIe, temperature, power, and history charts; process view shows per-process memory, spill (GPU-accessible system RAM), an estimated paged figure, and detected runtimes; the model inspector names models and breaks down weights and KV cost when it has enough evidence.

Quick start

On 64-bit Windows 10/11 with an NVIDIA or AMD GPU and a current WDDM driver, the fastest path is the portable build from the Releases page: download gputaskman-VERSION-windows-x64.zip (or the arm64 build), extract the whole ZIP keeping _internal beside gputaskman.exe, then run in that folder:

.\gputaskman.exe
.\gputaskman.exe --once
.\gputaskman.exe --json

The portable build needs no Python, uv, or CUDA Toolkit; builds are currently unsigned and ship with a .zip.sha256 companion for integrity checks.

To run from source instead, install Git for Windows and uv, open a new PowerShell window, then:

winget install --id Git.Git --exact
winget install --id astral-sh.uv --exact
git clone https://github.com/shanselman/gputaskman.git
cd gputaskman
uv run --locked gputaskman

uv fetches the pinned Python version, creates .venv, and installs locked dependencies; the first run needs internet. The TUI runs after the first sample. Native Linux and WSL use the same uv run --locked gputaskman flow (or the Linux tarballs, which need glibc 2.35+); macOS on Apple Silicon currently runs from a clone only.

Key operations

Controls: arrows select a process, s cycles sort (VRAM, GPU%, spill, paged, name), c cycles chart views, a toggles top/all processes, p pauses collection, ? explains counters, q quits. A wide terminal (about 140 columns) makes the full table readable, and model discovery can lag a newly loaded model by roughly 15 seconds.

For scripted use, --json emits a snapshot and diff compares two snapshots:

uv run --locked gputaskman --json > before.json
# load the model or change the context length, then send one request
uv run --locked gputaskman --json > after.json
uv run --locked gputaskman diff before.json after.json

Take the "after" snapshot only after the model has answered a request; allocators cache freed memory, so shrinkage can appear late. --list-gpus and --gpu N select an adapter, and --no-host disables the Windows host counter stream in WSL.

Constraints and behavior differences

• Windows: per-process memory comes from WDDM counters; under GeForce WDDM, NVML may report per-process memory as N/A, which is the case the tool is designed for. AMD device telemetry depends on the driver's ADL, with AMD SMI as an optional fallback that requires a separate Python install.
• Native Linux: reads NVML process lists and utilization samples; Spill and Paged are blank because Linux has no WDDM equivalent. Other users' processes require sudo.
• WSL: Linux cannot see how much VRAM its processes use, so gputaskman streams Windows host counters through interop. vmwp.exe gives an exact VM total, but per-process attribution is inferred (*) or guessed (?). With interop off or --no-host, Linux rows show n/a.
• macOS: Apple GPUs share memory with the CPU, so there is no VRAM split; other users' processes show GPU% with memory n/a unless run with sudo. Metal gives one GPU time per process with no engine split.
• Estimates vs measurements: KV and weight figures come from GGUF metadata, launch arguments, runtime logs, and local server APIs; they are never added to measured totals. Missing counters are N/A in text and null in JSON, not zero.
• Privacy: the inspector and JSON can contain paths, command-line flags, model names, and local ports; the README advises redacting captures before sharing. No telemetry is sent.

Project status

Marked early and experimental. The README lists validated live checks (RTX 4080 SUPER, an RTX Spark N1X on Windows ARM64, an M4 Mac mini, a Vast.ai container on an RTX 3060 Ti) alongside untested areas: bare-metal Linux hardware, multi-GPU and mixed-vendor setups, AMD SMI telemetry, Vulkan/ROCm workloads, and WinML/Studio Effects on Copilot+ PCs. The MIT license permits commercial use and modification.

HotRepo's take

The project is very new—created 2026-11-03—and appeared on the Python "new" list the next day, gaining 5 stars over a roughly two-hour window that ended at 2026-11-04 22:45 UTC, with 105 stars and 6 forks by that point. That is a small sample; nothing here establishes a trend. What stands out is the documentation's insistence on separating measured counters from inferred ownership and estimates, including a WSL context-matching scheme and spot checks against llama.cpp logs. For readers tuning local LLM memory, that discipline is the main reason to try it, and the main thing to verify on their own hardware.