MakazhanAlpamys/Soup

What Soup does

Soup is a command-line tool for fine-tuning and post-training large language models: SFT, preference optimization (DPO, GRPO, PPO, KTO, ORPO, SimPO, IPO, BCO), tool-calling, pretraining, distillation and more. A single soup.yaml describes the base model, dataset, training parameters and output; Soup handles GPU detection, quantization, batch sizing and LoRA setup. It runs locally with QLoRA and ships an OpenAI-compatible server, evaluation, merge/export and a local web UI.

Its headlining feature is layer streaming (opt-in via stream_layers: true, still beta): the frozen base model stays in host RAM and is fed to the GPU one decoder layer at a time. The project reports training Llama-3.1-8B-Instruct with NF4 on a 4 GB RTX 3050 Laptop at 119.6 tok/s and a 3.32 GB peak on v0.72.2, with a correctness protocol checking the streamed run against a resident one. A newer run from a contributor reads 208.6 tok/s at a higher clock in a 2.40 GB peak, but the project itself notes the raw gain there comes from the clock, not the algorithm.

It targets people who have a GPU and a dataset but do not want to assemble the training stack or manage remote machines.

Quick start

Install in an isolated environment from PyPI. The light install has the CLI and data tools; soup train needs the [train] extra.

pipx install "soup-cli[train]"
# or: uv tool install "soup-cli[train]"
# or inside a venv/Colab/Docker: pip install "soup-cli[train]"

Use double quotes around the extra so it works in cmd.exe, PowerShell, bash and zsh. Then create a config and train:

soup init --template chat        # interactive wizard or a template
soup train --config soup.yaml    # LoRA, quantization, batching handled

After training, try the model or ship it:

soup chat  --model ./output
soup export --model ./output --format gguf --quant q4_k_m

A minimal config looks like this (full schema reference in config/schema.py):

base: meta-llama/Llama-3.1-8B-Instruct
task: sft
data:
  train: ./data/train.jsonl
  format: alpaca
  val_split: 0.1
training:
  epochs: 3
  lr: 2e-5
  batch_size: auto
  lora:
    r: 64
    alpha: 16
  quantization: 4bit
output: ./output

soup doctor reports GPU, dependencies and version if something fails. For a browser workflow, pip install "soup-cli[ui]" then soup ui serves a local dashboard at 127.0.0.1:7860.

Constraints and caveats

• Python 3.11–3.13 are the tested versions; 3.14+ is not supported because the PyTorch stack has not been validated there.
• A CUDA GPU is recommended; Apple Silicon (MPS) and CPU work, the latter only experimentally and very slowly. The README's VRAM table suggests 8 GB for ~7B models via QLoRA, 24 GB for ~34B, 48 GB for ~70B.
• Fine-tuning needs the [train] extra; serving, UI and other extras are separate installs.
• Since v0.75, unknown config keys are rejected instead of silently ignored, and v0.77.0 made stricter config validation that can reject configs which loaded on v0.76. Run soup doctor --config soup.yaml before upgrading; also note soup data commands now refuse an --output equal to an input file.
• Telemetry is opt-in only (SOUP_TELEMETRY=1, off by default). License is Apache-2.0 with NOTICE retention and name-use terms.
• Layer streaming is beta and opt-in; the performance figures in the README come from specific hardware and versions, and the project explicitly separates pre- and post-repair measurements rather than presenting one speedup number.

HotRepo's take

Soup appeared on the all-language daily list on 2026-08-15 with 303 new stars and again on 2026-08-16 with 297, reaching 10,061 stars by its 2026-11-03 Python-language daily record (135 stars that day, best position 11 in that list). It also drew a Show HN thread on 2026-08-04 (139 points, 28 comments) about fine-tuning an 8B model on a 4 GB laptop GPU.

The project's distinguishing habit is measurement hygiene: it publishes raw benchmark records, documents a retracted explanation and a repaired wrong-gradient defect, and explicitly attributes part of a newer speedup to clock frequency rather than the method. The HN discussion is less flattering in one respect — the maintainer acknowledged using an LLM to write replies, which a commenter flagged as driving many dead comments. That is a matter of discussion conduct, not of the code or the training claims.