T-Crypt/berth

If one machine runs llama-swap, ComfyUI, Ollama and a game on the same card, an out-of-memory crash is usually a matter of time. Berth is a small local GPU broker that makes those processes take turns, and it is worth a look for anyone who already manages local inference by hand.

The problem

Before Berth, you start a chat engine and it quietly reserves most of the card. Later you launch an image pipeline or a game, and one of them dies with an out-of-memory error, or you spend the evening checking nvidia-smi and stopping processes before starting the next one. With Berth, each process asks a daemon for the card, waits until the previous holder has finished its reply or job, and then starts. The practical difference is that the card passes between engines without you babysitting memory; the harbor UI shows which lease is active and which processes hold VRAM without one.

What's different

The mechanism is a priority queue plus a deliberate drain. Requests are ordered as hold, foreground, interactive or batch; the holder is asked to unload, then terminated, and the next engine is only started after the driver reports the memory as free. Berth also watches processes it did not start and reports their PID and VRAM as a "stowaway" without killing them. That last choice is the important trade-off: unmanaged holders are visible but not removed, so a forgotten background process can still block a lease.

It is not a replacement for llama-swap; the README positions them as complementary, with llama-swap changing models within its config while Berth governs the whole card, including ComfyUI, vLLM, games and VMs passed through to a second card. The cost is coordination: engines must be launched through berth run, the gateway, or a supported claim path rather than started directly. A process started outside those paths will not queue, which is exactly why the stowaway report exists.

Who should try it

A good fit is a developer or self-hoster with one NVIDIA card who moves between a local chat model, an image workflow and occasionally a game, and who is tired of manually freeing VRAM. It also suits someone running several engines that are individually model-specific, where switching configs by hand has become the normal workflow. A poor fit is a multi-tenant inference server with strict SLOs: per-card daemons and an alpha status mean the project is aimed at a workstation pattern, not a scheduler for shared production capacity. Intel and Apple hardware are out for now, so laptop Mac users should not start here.

Requirements and limitations

Linux with NVIDIA is the supported path; Windows 10/11 with NVIDIA is experimental, with leases and process attribution working but no packaged installer or locked-down control pipe. AMD on Linux is experimental, with discrete RDNA3/RDNA4 and Phoenix APUs partially validated. Multiple GPUs are experimental, requiring one daemon per card identified by PCI address, with a single daemon for all cards and cross-card leases planned for v0.4. Installation is a Go install or a release binary, followed by user-service setup (install guide); ports are configured in berth.toml and require a service restart.

The ongoing upkeep is the real constraint. Because the project is at alpha (v0.3.6) and moves quickly, expect config and flag changes between releases; uninstall and service handling are documented, but you should be comfortable reading release notes and the harbor documentation rather than treating it as a set-and-forget appliance.

HotRepo's take

HotRepo first tracked the repository on 2026-10-24; on 2026-10-25 it appeared at #19 on the NEW-REPOSITORY board (新项目榜, not the overall daily Trending board) with 159 total stars, the only recorded observation. That is a single dated snapshot, so persistence of interest cannot be judged. Before adopting, check the support matrix against your exact GPU and platform rather than assuming "Linux + NVIDIA" covers a laptop or a VM.