Runs Nsight Systems profiling and reads timeline reports to find GPU idle time and NCCL overlap issues.
NVIDIA/TensorRT-LLM
Guides an AI coding agent through building, testing, profiling and optimizing TensorRT-LLM inference on NVIDIA GPUs.
- 38 skills
- 15k stars
The 5 most installed
Ordered by installs on skills.sh- perf-nsight-systems41 installs
- perf-nsight-compute-analysis37 installs
Interprets Nsight Compute reports, classifying bottlenecks by SOL, roofline, occupancy, memory and warp stalls.
- kernel-triton-writing27 installs
Writes and verifies OpenAI Triton kernels such as fused elementwise, reductions, tiled GEMM and attention.
- perf-optimization18 installs
Routes performance work to the right specialist and applies optimizations or kernels in a safe order.
- perf-torch-cuda-graphs15 installs
Applies CUDA Graphs to PyTorch workloads, choosing the right API and handling capture and replay issues.
The other 33 skills
- perf-torch-sync-free15 installs
Finds and removes host-device synchronization points in PyTorch code that stall the GPU.
- perf-workload-profiling15 installs
Adds timing instrumentation to training loops or kernels and labels code regions for profiler timelines.
- kernel-cute-writing11 installs
Writes GPU kernels in NVIDIA CuTe DSL, covering elementwise, GEMM, reductions, tensor cores and pipelining.
- perf-analysis10 installs
Coordinates performance profiling, classifies bottlenecks and produces a structured analysis report.
- trtllm-serve-config-guide7 installs
Generates a source-backed trtllm-serve YAML config for basic single-node PyTorch serving.
- kernel-tileir-optimization6 installs
Tunes existing Triton kernels for the TileIR backend on Blackwell GPUs with occupancy and TMA settings.
Checks whether your machine has GPUs, Docker or Slurm access and reports which run setup fits.
Compiles TensorRT-LLM inside a Docker container on a GPU compute node.
Runs a TensorRT-LLM workload in Docker in the background and reports the results from its logs.
Submits and monitors Slurm jobs on a local cluster, reusing one allocation across repeated runs.
Runs and profiles jobs on a remote Slurm cluster over SSH, including file transfer and persistent sessions.
Compiles TensorRT-LLM on a Slurm cluster by submitting a containerized batch job and verifying the build.
Writes the task brief file used to onboard a new model into the agent-driven modeling workflow.
Diagnoses a TensorRT-LLM serving deployment with benchmarks, SOL projections and per-kernel profiling, naming the main bottleneck.
Analyzes nsys traces to tell whether CPU overhead is limiting TensorRT-LLM inference and where it comes from.
Profiles and reduces CPU overhead in TensorRT-LLM through repeated profile, optimize and validate rounds.
Matches unstable performance symptoms against known TensorRT-LLM instability patterns and their fixes.
Suggests proven TensorRT-LLM optimizations for a given bottleneck, config or hardware, with risks and rollback.
Runs an iterative campaign that benchmarks, profiles and applies TensorRT-LLM serving optimizations one at a time.
Matches a suspected performance regression to known TensorRT-LLM patterns and their precedent fixes.
Runs TensorRT-LLM tests, benchmarks or scripts by checking the environment and picking a Docker image.
Walks you through contributing code to TensorRT-LLM, from fork and DCO signing to testing and review readiness.
Helps the agent explore the TensorRT-LLM codebase first, so it reuses existing code instead of rewriting it.
Fetches the latest flashinfer-python releases, lets you pick a version, and updates every reference in the repo.
Adds a HuggingFace multimodal model (vision, audio, video plus text) to the TensorRT-LLM PyTorch backend.
Guides implementing, modifying or verifying a TensorRT-LLM ModelingV2 catalog entry and its GPU test.
Explains which engine stage owns a TensorRT-LLM feature like KV-cache reuse or speculative decoding, and what the target must honor.
Provides a checklist for designing, implementing, reviewing or debugging TensorRT-LLM PyTorch MoE code.
Builds Slurm scripts or Docker commands for TensorRT-LLM workloads, resolving images, mounts and parallelism settings.
Classifies and runs model-level and module-level TensorRT-LLM tests, asking you for parallelism, datasets and device settings.
Benchmarks a TensorRT-LLM VisualGen model end to end, including bringing up the trtllm-serve server it needs.
Refreshes VisualGen LPIPS golden images or videos after a CI failure or intentional output change, and revalidates tests.
Designs and runs deterministic VisualGen component tests for DiT, VAE, text encoder, scheduler, quantization and caching.