Compares accelerators on capacity, effective compute, memory bandwidth, power and cost for a given model workload.
bojieli/ai-infra-book
Provides quantitative methods for sizing and analyzing AI infrastructure, from hardware and model footprints to serving, training and networking.
- 12 skills
- 7.8k stars
The 5 most installed
Plans how agent model calls and tool environments are created, kept, released and restored for capacity and latency.
Estimates cross-server traffic, network bottlenecks and sync waits for multi-node training or inference.
Compares cross-instance inference options like replicas, prefill/decode split, expert placement and shared KV cache.
Compares placing a model or task on device, edge or cloud by latency, transfer, energy and recovery.
The other 7 skills
Compares batching, scheduling, KV cache, offloading and speculative decoding for a single-instance inference service.
Diagnoses GPU kernel, fusion, compilation and runtime submission issues to find what sits on the request critical path.
Compares how model or architecture changes affect weight size, KV state, forward compute and memory access.
Compares data, tensor, sequence, context, pipeline and expert parallelism by capacity and communication cost.
Estimates whether one model execution fits given hardware and locates compute, memory and serial-wait lower bounds.
Estimates memory, step time, checkpoints, failure rework and deadlines for training or RL jobs.
Turns continuous requests, multi-turn agents, real-time interaction or RL traces into capacity and latency loads.