NVIDIA/nccl

Project overview

NCCL is a standalone C++ library of standard GPU communication routines. It implements all-reduce, all-gather, reduce, broadcast, reduce-scatter, and send/receive-based patterns, and is tuned for high bandwidth over PCIe, NVLink, NVSwitch, InfiniBand Verbs, and TCP/IP sockets. It works with any number of GPUs in one node or across multiple nodes, in single-process or multi-process (for example MPI) programs. It is mainly used by people building or running distributed GPU workloads such as deep-learning training, who need a reliable collective backend rather than to write their own communication code. The user guide covers usage details at the NCCL documentation.

Quick start

Official tested builds are available from the NVIDIA NCCL download page; use those if you do not need to compile from source. To build from a checkout:

cd nccl
make -j src.build

If CUDA is not in /usr/local/cuda, pass its location, for example make src.build CUDA_HOME=<path to cuda install>. Output goes to build/ unless BUILDDIR is set. By default all supported architectures are compiled; to compile only for the target GPU, restrict the architecture, for example:

make -j src.build NVCC_GENCODE="-gencode=arch=compute_90,code=sm_90"

To test, clone the separate nccl-tests repository, build it, and run a collective benchmark such as ./build/all_reduce_perf -b 8 -e 256M -f 2 -g <ngpus> with your GPU count in place of <ngpus>.

Installation packages

The README describes building distributable packages. On Debian/Ubuntu, install build-essential devscripts debhelper fakeroot, run make pkg.debian.build, and find the result under build/pkg/deb/. On RedHat/CentOS, install rpm-build rpmdevtools, run make pkg.redhat.build, and look under build/pkg/rpm/. An OS-agnostic tarball is produced by make pkg.txz.build in build/pkg/txz/. A Python wheel can be produced with make pkg.python_wheel.build; the README's route installs uv first via its install script, so check that dependency choice before running it in a controlled environment. Installing system packages requires root privileges.

Constraints

Building requires a CUDA toolchain, and the build targets NVIDIA GPUs and the listed interconnects; the default build covers all supported architectures, which increases compile time and binary size. Tests are not in this repository but in nccl-tests. The repository's license is reported as NOASSERTION, so confirm the actual licensing terms before redistribution. The Python wheel path adds a Python dependency manager to the build environment.

HotRepo's take

NCCL is a long-standing NVIDIA infrastructure library with a substantial following (over 5,000 stars and nearly 1,500 forks at the time of review). It is not an end-user application but a low-level dependency that distributed training frameworks and HPC applications build on. The main practical decision for readers is whether to consume prebuilt NVIDIA packages or compile from source for a specific GPU architecture and interconnect; the separate nccl-tests project is the right place to validate performance once installed.