ggml-org/ggml

What it is

ggml is a tensor library for machine learning written in plain C/C++, with no external dependencies. It targets portable, efficient inference: plain C/C++ implementation, cross-platform builds (x86, ARM, RISC-V, LoongArch, PowerPC, s390x, WebAssembly), SIMD-optimized kernels, CPU/GPU/NPU/browser backends, 2- to 8-bit integer quantization plus MXFP4 and NVFP4 microscaling formats, and zero memory allocations during runtime (README description).

It is the tensor layer behind ggml-based projects, most visibly llama.cpp. If you are integrating model inference into a C/C++ application rather than using a ready-made model runner, this is the layer to build on.

Build from source

The project builds with CMake. From a terminal with CMake and a C/C++ toolchain installed:

git clone https://github.com/ggml-org/ggml
cd ggml

mkdir build && cd build
cmake ..
cmake --build . --config Release -j 8

The README points to examples/simple for a minimal, fully commented matrix-multiplication example — the best starting point for seeing the API in use. Release tags are published on the releases page.

Documentation and file format

• The GGUF file format describes the model container format used by ggml-based tools.
• Introduction to ggml is an external walkthrough of the library.
• GGML tips & tricks collects practical guidance from the llama.cpp wiki.

Constraints and contributing

The library addresses low-level tensor ops and backend selection rather than high-level model loading; model runners such as llama.cpp sit on top of it. Backend support varies by platform, so actual GPU/NPU acceleration depends on what the build detects. The core library is MIT-licensed. Core changes, including the CMake build system, are requested as pull requests to llama.cpp rather than this repository (README contributing).

HotRepo's take

On 2026-11-14 ggml reached 6th on the C++ daily list with 139 stars added, alongside a 311-point HN discussion of the v1.0.0 release (HN thread). That discussion is where the useful signal sits: a long-time user benchmarked a 50K-token prompt before and after a routine upgrade and saw runtime drop from 4:22 to 3:59, crediting steady 5–10% per-update speedups (jacquesm). Another commenter frames llama.cpp as the "rock solid" general choice versus hardware-specialized engines (dust42), and users on Vulkan-only AMD cards report usable local inference through ggml-based software (kodoman). These are user impressions, not benchmarks from this repository, but they line up with the project's stated focus on portable CPU/GPU inference.