bojieli/ai-infra-book
Learning ResourcesAn open book that derives LLM inference and training system design from hardware constraints and model architecture.
What the book covers
《深入理解 AI Infra:量化分析与系统设计》 is an open draft of a book by Bojie Li that explains how AI infrastructure is designed from hardware limits and model architecture. Its method is to start from the task and quality requirement, list compute, storage, communication and dependencies, then compare against hardware capacity, bandwidth and throughput. The twelve chapters move from an initial resource estimate for one generation through model architecture, inference and training workloads, accelerators, operators and runtimes, supernodes, data center networks, inference optimization, distributed inference, training systems, scheduling, and edge-cloud collaboration.
It is aimed at people who have called a model API or run a model locally and want to understand why it is slow and how to reduce cost, as well as systems, networking and chip engineers, researchers and students. Readers need some Python, linear algebra and computer systems background; the preface gives the specific prerequisites.
Start reading
You can read online at the book site, or download the Simplified Chinese PDF or EPUB. The repository also carries community translations: English by @tg1482, Traditional Chinese by @edward821220, and a Russian community edition organized by @ilkruglov, each with PDF, EPUB and online reading links.
The manuscript is the single source of truth and lives as Markdown in manuscripts/; changing the text means editing there. If GitHub's Markdown rendering of formulas is incomplete, the PDF or online site is the recommended format.
Run the calculation tools
The companion calculation CLI reproduces the book's numbers or estimates a different model and input. It needs Python 3.10+ and only the standard library, with no GPU or model weights. Clone the repository and run:
git clone https://github.com/bojieli/ai-infra-book.git
cd ai-infra-book
python3 calculations/calc.py models
python3 calculations/calc.py forward --model qwen3-8b --tokens 8192 --format md
The first command lists supported models; the second estimates per-operator resource needs for Qwen3-8B at 8192-token prefill and prints a Markdown table. Model configurations and a result index are documented in the calculations guide.
Repository scope and constraints
Large papers and measurement records are stored in Git LFS, about 20 GB in total. .lfsconfig skips all LFS files by default, so a normal clone leaves pointers; the manuscript, figures and static calculations do not need them. To reproduce an experiment, download only its directory, for example:
git lfs install
git lfs pull --include="experiments/ch05/05-01/**" --exclude=""
Experiments are organized under experiments/chXX/XX-YY/, each with run instructions, input conditions and results; GPU experiments state their hardware and dependency requirements. Building the PDF requires Pandoc, XeLaTeX and fonts, and building the site requires Python 3.10+ plus the website requirements. The original text, figures and companion code are under Apache-2.0, while third-party code, fonts, templates and references keep their own licenses. The manuscript is still a draft and is being revised, so results should be cited together with the book version, model, hardware and input parameters used.
HotRepo's take
The project is substantial as a learning resource rather than a software tool: it ships a full manuscript, a calculation CLI, per-chapter experiments and case studies, which is exactly the kind of content that justifies a repository on its own. On 2026-11-15 it recorded 7,796 stars and 624 forks, with 151 stars added that day and a best position of 4 on the Python daily list; that single snapshot describes one day only and does not establish a trend. The value for readers is the quantitative approach: the book asks what is moved, how much, how often, through where and who must wait, and the CLI lets you check those estimates against your own model and hardware rather than taking the numbers on faith.
Written by AI from the project's README, updated . The project's own docs are the source of truth.