deepseek-ai/3FS

What it is

3FS (Fire-Flyer File System) is a C++ distributed file system for AI training and inference storage. It aggregates many SSDs and storage nodes into one shared layer that clients access without worrying about data locality, and it uses CRAQ chain replication for strong consistency. A stateless metadata service backed by a transactional key-value store such as FoundationDB provides a conventional file interface, so applications do not need a new storage API.

Its stated target workloads are data preparation, random-access dataloaders, high-throughput parallel checkpointing, and a KVCache layer for LLM inference. The repository contains the source, design notes, protocol specs, and a USRBIO API reference rather than only an overview.

Quick start

The documented path is a source build on Linux. First clone the repository and initialize its submodules, then apply the included patches:

git clone https://github.com/deepseek-ai/3fs
cd 3fs
git submodule update --init --recursive
./patches/apply.sh

Install system packages for your distribution; the README lists separate apt, yum, and dnf commands for Ubuntu 20.04/22.04, openEuler 2403sp1, OpenCloudOS 9, and TencentOS 4. You also need libfuse 3.16.1 or newer, FoundationDB 7.1 or newer, and a Rust toolchain (1.75.0 minimum, 1.85.0 or newer recommended).

Build with Clang 14, choosing the shuffle method that matches your compiler lineage:

cmake -S . -B build \
      -DCMAKE_CXX_COMPILER=clang++-14 -DCMAKE_C_COMPILER=clang-14 \
      -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
      -DSHUFFLE_METHOD=<method>
cmake --build build -j 32

Replace <method> with g++10 or g++11. New clusters may pick either, but an existing cluster must reuse the method of its deployed binaries because std::shuffle differs across compiler versions and can make binaries incompatible. Prebuilt Docker build images exist for TencentOS 4 and OpenCloudOS 9. To run a test cluster, follow the setup guide; it is a separate deployment step not reproduced in the README.

Constraints and limits

• The build and deployment instructions target Linux; the package lists cover specific distributions, so other environments need equivalent dependencies.
• FoundationDB is a required dependency for the metadata layer, not an optional add-on.
• The README's throughput figures come from large clusters: a 180-node read test with 2×200 Gbps InfiniBand NICs and 16 NVMe SSDs per node reached about 6.6 TiB/s; GraySort on 25 storage and 50 compute nodes sorted 110.5 TiB in about 30 minutes. These describe those configurations and are not a small-cluster or single-machine expectation.
• The shuffle-method choice is a compatibility constraint fixed for the life of a cluster.
• The project is MIT licensed.

HotRepo's take

3FS appeared on the C++ daily list on 2026-11-10 at position 5 with 5 stars gained, but the more informative signal is the February 2025 Hacker News discussion, which drew 361 points and 101 comments, including a note that the design was published earlier and has been used in production for several years design background. The discussion also questions how critical random reads are for assembling training batches random-read question, a fair technical challenge for readers evaluating the design against their own access patterns. The repository is a full infrastructure project with build scripts, specs, and deployment docs, so it is best treated as cluster software to operate, not a drop-in local filesystem.