alibaba/MNN

What It Is

MNN is a deep learning framework focused on on-device inference and training, with particular optimization for mobile and embedded hardware. It converts models from TensorFlow, Caffe, ONNX, and TorchScript into MNN format, then runs them efficiently on CPU, GPU, and several NPU backends. The repository also contains MNN-LLM for local large language model deployment and MNN-Diffusion for Stable Diffusion-style image generation.

The project targets mobile and embedded developers who need compact binaries and predictable performance: the Android core library is about 800 KB in the documented armv7a C++ shared-library configuration, and the full iOS static library is around 12 MB (46). It supports CNN, RNN, GAN, and Transformer architectures, dynamic inputs, control flow, and multi-input/multi-output models (47). Backend coverage is broad but uneven by precision; the README's support table distinguishes deeply optimized paths from merely working ones (49).

Alongside the engine, the repository ships MNN-Converter, MNN-Compress, MNN-Express, a lightweight OpenCV-like image module, and MNN-Train (50). Application examples include the Android MNN Chat app for multimodal local LLMs and the TaoAvatar 3D avatar app (30)(28).

Quick Start

The most reliable starting point is the official documentation (6), which covers building from source, conversion, and each runtime. A typical path is:

1. Install the build dependencies for your target platform (CMake, a C++ compiler, and platform SDKs such as Xcode or the Android NDK).
2. Clone the repository and build the engine using the documented CMake options for your platform.
3. Convert an existing ONNX, TensorFlow, Caffe, or TorchScript model with MNN-Converter.
4. Load the converted model with the C++ or Python API and run inference.

The README does not include a complete clone/build/convert command sequence, so follow the per-platform instructions in the documentation rather than guessing flags. For LLM deployment specifically, use the MNN-LLM guide (36); for diffusion models, use the MNN-Diffusion guide (38). Model format conversion details and supported operators are also documented there.

Constraints

• Platform support covers iOS 8.0+, Android 4.3+, and POSIX-based embedded devices, with CPU and GPU hybrid execution available (47)(48).
• GPU and NPU acceleration depends on the backend and precision. For example, Vulkan and OpenCL advertise different INT8/FP16 support levels, and CoreML primarily uses FP16 internally (49).
• Quantization can reduce model size by roughly 50%–70%, but the support table notes that INT8/INT4 ratings do not imply full integer execution for every operator (46)(49).
• MNN_BUILD_MINI reduces package size by about 25% at the cost of fixing model input size (46).
• The project is Apache-2.0 licensed (53).
• Community discussion is mainly in Chinese through DingTalk groups, though English speakers are welcomed (51).

HotRepo's Take

MNN recorded 26 new stars in the daily C++ ranking on 2026-11-07, placing 6th on that language list for the day. That single snapshot shows continued attention among C++ developers but does not establish a trend. The more durable signal is the project's scope: an Apache-2.0 engine that has been in production inside Alibaba apps and now ships both a mobile LLM runtime and a small mobile-native model, MNN-Pocket-0.3B, released under Apache 2.0 (18). For readers comparing on-device runtimes, the precision-by-backend table and the separate LLM/diffusion guides are the concrete material to evaluate against their own hardware.