All skills
gke-inference
Generate a GKE inference manifest for a chosen model, server, and accelerator, then deploy it.
- Part of google/skills
- 8.5k installs
- DevOps & Self-hosting · Containers and clusters
When to use it
You have a model like Gemma or Llama and a GKE cluster but no working inference manifest yet, and you need to pick a valid accelerator and server combination.
For example
You say
Use gke-inference to deploy gemma-2-9b-it with vLLM on an L4 GPU targeting 50ms NTPOT, and give me the manifest to apply.
You get
- inference.yaml manifest
- Deployment commands
- Accelerator choice
How it works
- Uses gcloud container ai profiles to list supported models and valid model/server/accelerator combinations before generating anything.
- Includes a ComputeClass example for targeting GPU nodes on Autopilot.
- Covers autoscaling with GPU duty cycle metrics and tuning tips like quantization and tensor parallelism.
Before installing
A golden path GKE Autopilot cluster, an authenticated gcloud CLI, and sufficient GPU/TPU quota in the target region.
More Containers and clusters skills
See all- kubernetes-specialist14.9kCreates and manages Kubernetes workloads, manifests, Helm charts, RBAC rules and pod debugging.
- dynamo-recipe-runner2.6kSelects, validates, patches, and deploys NVIDIA Dynamo Kubernetes recipes for model serving.
- dynamo-interconnect-check1.9kValidates that a Dynamo deployment's NIXL, UCX, and NCCL interconnect is ready for disaggregated serving.
- kubernetes-architect1kPlans and reviews Kubernetes setups, including GitOps workflows and enterprise container orchestration.
Also in this pack
- cloud-run-basics19.4kDeploys and manages Cloud Run services, jobs and worker pools.
- gemini-api18.8kWrites code that calls Gemini models for text, images, video, audio, embeddings and more across languages.
- google-cloud-recipe-auth17.1kExplains how to authenticate and authorize users and services to Google Cloud APIs, including IAM and ADC.
- google-cloud-recipe-onboarding16.9kWalks a new developer through creating a Google Cloud account, setting up billing and deploying a first resource.