lemonade-sdk/lemonade

Lemonade is a local AI server and embeddable runtime that runs models on the machine's own CPU, GPU, or NPU and exposes them through familiar API interfaces. It fits people who want to point existing OpenAI-style clients at a local endpoint instead of a cloud service. The models themselves are third-party; Lemonade is the serving layer.

What it does

The server runs chat, coding, speech-to-text, text-to-speech, audio, image, 3D, and text-classification models, routing each request to an inference engine and backend suited to the hardware. Clients connect to a local base URL using standard OpenAI clients (among other language libraries), so an existing application can swap in a local model with minimal change. A CLI handles pulling models, listing and running them, inspecting available backends, and launching coding tools.

A distinguishing mechanism is backend selection: llamacpp covers CPU, Vulkan, ROCm, CUDA, and Metal paths; flm and ryzenai-llm target AMD XDNA2 NPUs; vllm and ds4 are aimed at specific Strix Halo configurations. Because support differs by modality, device, and OS, the documented matrix—and the lemonade backends check—matter more than the general claim of GPU/NPU acceleration.

Model aliases add active-standby failover: an alias can point at one model and later be repointed at another, giving environment-independent names. Cloud offload for OpenAI-compatible providers is also offered, marked experimental.

Use paths

Lemonade Server installs a service on Windows, Linux, or macOS, with Docker, Snap, Arch, Debian, Fedora, and Ubuntu packages documented. Getting models typically means lemonade pull or the Model Manager; custom GGUF/ONNX files can come from Hugging Face or ModelScope. Embeddable Lemonade is a separate portable binary for bundling the stack into another application with no installer, branding, or telemetry.

Requirements and limits

Hardware support is path-specific: for example, CUDA text generation is listed for Turing-or-newer NVIDIA GPUs, NPU engines are limited to XDNA2 devices and specific OSes, and several backends are flagged experimental. Checking the matrix before choosing a model is the practical prerequisite.

On privacy, the README says the program does not send information elsewhere unless requested, but model downloads or registry lookups may contact Hugging Face or ModelScope depending on the chosen model source. The project is Apache-2.0 licensed, with portions under other terms described in the LICENSE file.

Project source and documentation