Tencent/ncnn

Overview

ncnn is a neural network inference framework written in C++ and aimed at deploying trained models on mobile phones, embedded boards, desktop PCs, and browsers. It has no third-party runtime dependencies and does not require BLAS or NNPACK, so a model can be shipped as a compact library plus a .param graph file and a .bin weight file. Tencent reports using it in production apps such as QQ, Qzone, WeChat, and Pitu.

The framework covers CPU inference tuned for ARM NEON and multi-core scheduling, plus a Vulkan GPU backend on supported platforms. It supports multi-input, multi-output, and multi-branch graphs, fp16 storage and arithmetic paths, int8 quantized inference, custom layers, and direct memory-reference loading of models. Model import is handled mainly by pnnx, which converts PyTorch and ONNX models; legacy converters remain for Caffe, MXNet, and Darknet models.

It suits developers who already have a trained model and need to run it with bounded memory and no heavyweight runtime on edge hardware.

Quick Start

The documented beginner path is PyTorch → pnnx → ncnn. Prepare a Python environment with PyTorch, then install pnnx:

pip3 install pnnx

Export a small PyTorch model. The example builds a convolution, ReLU, global mean, and linear layer, runs it in eval mode, and exports with a 1x3x224x224 input:

import torch
import torch.nn as nn
import pnnx

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Conv2d(3, 8, 1)
        self.relu = nn.ReLU()
        self.fc = nn.Linear(8, 4)

    def forward(self, x):
        x = self.conv(x)
        x = self.relu(x)
        x = x.mean((2, 3))
        return self.fc(x)

model = Model().eval()
x = torch.rand(1, 3, 224, 224)
pnnx.export(model, "model.pt", (x,))

This produces model.ncnn.param and model.ncnn.bin. Load them with the C++ API:

#include "net.h"

ncnn::Net net;
net.load_param("model.ncnn.param");
net.load_model("model.ncnn.bin");

ncnn::Mat in(224, 224, 3);
auto ex = net.create_extractor();
ex.input("in0", in);

ncnn::Mat out;
ex.extract("out0", out);

A Python binding is also available for the same files, using ncnn.Net(), ncnn.Mat, and create_extractor() with the in0/out0 names. The extracted Mat shape can be printed with NumPy. Complete workflows are in pnnx, the PyTorch/ONNX conversion guide, the Python API, and the examples.

If you prefer a prebuilt library, downloads for Android, iOS, macOS, Linux, Windows, HarmonyOS, WebAssembly, and Apple platforms are published with Vulkan and CPU-only variants.

Build from Source

Prebuilt packages are recommended when they match the target. Otherwise, a typical Linux build is:

git clone --recursive https://github.com/Tencent/ncnn.git
cd ncnn
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DNCNN_VULKAN=ON -DNCNN_BUILD_EXAMPLES=ON ..
cmake --build . -j$(nproc)

Platform-specific instructions, including cross-compilation for ARM Cortex-A, Hisilicon, AllWinner D1, Loongson 2K1000, QNX, MIPS, RISC-V, and LoongArch, are in the build guide.

Constraints and Scope

The Vulkan backend is opt-in through NCNN_VULKAN and depends on the target's GPU and driver support; CPU-only packages are provided for platforms without it. The framework targets inference rather than training. Operator coverage is the practical boundary: models relying on unsupported PyTorch or ONNX operators need checking against the supported-operator tables or may need a custom layer. The README notes that classic CNN workloads remain its strength, while audio, generation, and language workloads are represented by community projects where operators are supported. The project is licensed under BSD 3-Clause.

HotRepo's Take

On the 2026-11-12 C++ daily list, ncnn ranked 6th with 9 new stars, while holding about 24,000 stars and 4,500 forks overall. For a framework that has been public since 2017 and is embedded in large consumer apps, that is a modest day-to-day gain rather than a breakout, and the record is a single snapshot, so it does not establish a trend. The notable fact is where it appears: a mature inference runtime still surfacing on the C++ language list nearly a decade after launch, consistent with continued use in mobile and edge deployments rather than novelty interest.