Find your next open-source project

Search by name, purpose, or keyword — or paste a GitHub repository URL
All skills

bojieli/ai-infra-book

Provides quantitative methods for sizing and analyzing AI infrastructure, from hardware and model footprints to serving, training and networking.

  • 12 skills
  • 7.8k stars

The 5 most installed

  • Compares accelerators on capacity, effective compute, memory bandwidth, power and cost for a given model workload.

  • Plans how agent model calls and tool environments are created, kept, released and restored for capacity and latency.

  • Estimates cross-server traffic, network bottlenecks and sync waits for multi-node training or inference.

  • Compares cross-instance inference options like replicas, prefill/decode split, expert placement and shared KV cache.

  • Compares placing a model or task on device, edge or cloud by latency, transfer, energy and recovery.

The other 7 skills
  • Compares batching, scheduling, KV cache, offloading and speculative decoding for a single-instance inference service.

  • Diagnoses GPU kernel, fusion, compilation and runtime submission issues to find what sits on the request critical path.

  • Compares how model or architecture changes affect weight size, KV state, forward compute and memory access.

  • Compares data, tensor, sequence, context, pipeline and expert parallelism by capacity and communication cost.

  • Estimates whether one model execution fits given hardware and locates compute, memory and serial-wait lower bounds.

  • Estimates memory, step time, checkpoints, failure rework and deadlines for training or RL jobs.

  • Turns continuous requests, multi-turn agents, real-time interaction or RL traces into capacity and latency loads.