Find your next open-source project

Search by name, purpose, or keyword — or paste a GitHub repository URL
All skills

NVIDIA/TensorRT-LLM

Guides an AI coding agent through building, testing, profiling and optimizing TensorRT-LLM inference on NVIDIA GPUs.

  • 38 skills
  • 15k stars

The 5 most installed

Ordered by installs on skills.sh
  • Runs Nsight Systems profiling and reads timeline reports to find GPU idle time and NCCL overlap issues.

  • Interprets Nsight Compute reports, classifying bottlenecks by SOL, roofline, occupancy, memory and warp stalls.

  • Writes and verifies OpenAI Triton kernels such as fused elementwise, reductions, tiled GEMM and attention.

  • Routes performance work to the right specialist and applies optimizations or kernels in a safe order.

  • Applies CUDA Graphs to PyTorch workloads, choosing the right API and handling capture and replay issues.

The other 33 skills
  • Finds and removes host-device synchronization points in PyTorch code that stall the GPU.

  • Adds timing instrumentation to training loops or kernels and labels code regions for profiler timelines.

  • Writes GPU kernels in NVIDIA CuTe DSL, covering elementwise, GEMM, reductions, tensor cores and pipelining.

  • perf-analysis10 installs

    Coordinates performance profiling, classifies bottlenecks and produces a structured analysis report.

  • Generates a source-backed trtllm-serve YAML config for basic single-node PyTorch serving.

  • Tunes existing Triton kernels for the TileIR backend on Blackwell GPUs with occupancy and TMA settings.

  • Checks whether your machine has GPUs, Docker or Slurm access and reports which run setup fits.

  • Compiles TensorRT-LLM inside a Docker container on a GPU compute node.

  • Runs a TensorRT-LLM workload in Docker in the background and reports the results from its logs.

  • Submits and monitors Slurm jobs on a local cluster, reusing one allocation across repeated runs.

  • Runs and profiles jobs on a remote Slurm cluster over SSH, including file transfer and persistent sessions.

  • Compiles TensorRT-LLM on a Slurm cluster by submitting a containerized batch job and verifying the build.

  • Writes the task brief file used to onboard a new model into the agent-driven modeling workflow.

  • Diagnoses a TensorRT-LLM serving deployment with benchmarks, SOL projections and per-kernel profiling, naming the main bottleneck.

  • Analyzes nsys traces to tell whether CPU overhead is limiting TensorRT-LLM inference and where it comes from.

  • Profiles and reduces CPU overhead in TensorRT-LLM through repeated profile, optimize and validate rounds.

  • Matches unstable performance symptoms against known TensorRT-LLM instability patterns and their fixes.

  • Suggests proven TensorRT-LLM optimizations for a given bottleneck, config or hardware, with risks and rollback.

  • Runs an iterative campaign that benchmarks, profiles and applies TensorRT-LLM serving optimizations one at a time.

  • Matches a suspected performance regression to known TensorRT-LLM patterns and their precedent fixes.

  • Runs TensorRT-LLM tests, benchmarks or scripts by checking the environment and picking a Docker image.

  • Walks you through contributing code to TensorRT-LLM, from fork and DCO signing to testing and review readiness.

  • Helps the agent explore the TensorRT-LLM codebase first, so it reuses existing code instead of rewriting it.

  • Fetches the latest flashinfer-python releases, lets you pick a version, and updates every reference in the repo.

  • Adds a HuggingFace multimodal model (vision, audio, video plus text) to the TensorRT-LLM PyTorch backend.

  • Guides implementing, modifying or verifying a TensorRT-LLM ModelingV2 catalog entry and its GPU test.

  • Explains which engine stage owns a TensorRT-LLM feature like KV-cache reuse or speculative decoding, and what the target must honor.

  • Provides a checklist for designing, implementing, reviewing or debugging TensorRT-LLM PyTorch MoE code.

  • Builds Slurm scripts or Docker commands for TensorRT-LLM workloads, resolving images, mounts and parallelism settings.

  • Classifies and runs model-level and module-level TensorRT-LLM tests, asking you for parallelism, datasets and device settings.

  • Benchmarks a TensorRT-LLM VisualGen model end to end, including bringing up the trtllm-serve server it needs.

  • Refreshes VisualGen LPIPS golden images or videos after a CI failure or intentional output change, and revalidates tests.

  • Designs and runs deterministic VisualGen component tests for DiT, VAE, text encoder, scheduler, quantization and caching.