argoproj/argo-workflows

Adopt Argo Workflows if your batch, ML or CI jobs already run in containers and you want Kubernetes itself to schedule and track them; skip it if you have no cluster or your pipelines are small enough to stay in one process.

The problem

A team accumulates a pipeline: preprocess data, fan out training runs, then post-process and notify. Running it as shell scripts on one machine makes parallel branches, retries and per-step logs manual bookkeeping. With Argo Workflows, each step becomes a container and the workflow becomes a Kubernetes custom resource, so the cluster schedules the pods; the result is a visible graph with step-level inputs, outputs and status rather than a log file you scroll through. The change is where the orchestration state lives: in the cluster, not in a script wrapper.

What's different

Workflows are declared as steps or a DAG, and every step is a container. That makes the unit of work a pod, which brings Kubernetes scheduling (affinity, tolerations, node selectors), per-step resources, volumes and parallelism limits into the pipeline. The README also lists artifacts passed between steps (S3, GCS, Git, HTTP and others), templates stored in the cluster, cron scheduling, retries and timeouts at step and workflow level, plus a web UI, REST/gRPC server and CLI. Step outputs can be parameters or artifacts, so chaining is a first-class concept instead of a shared directory convention.

The trade-off is that you inherit Kubernetes as your execution model. You need a working cluster and a controller running in it, and you define steps as images rather than local commands. The listed client libraries (Java, Go, Python via Hera, TypeScript via Juno) lower the authoring cost for application teams, but they do not remove the cluster dependency.

Who should try it

Good fit: a data or ML platform team with an existing cluster that needs parallel fan-out, retries and a shareable graph for recurring jobs; the stated use cases cover ML pipelines, data and batch processing, infrastructure automation and CI/CD. It also fits teams already using Argo CD-style GitOps for other cluster resources. Poor fit: a single developer outsourcing a laptop-scale script, or anyone without a Kubernetes cluster to target, since the container-per-step model then adds more setup than it removes. If the team writes Python, Hera is the documented path to author workflows without hand-writing YAML, and that changes who can adopt it comfortably.

Requirements and limitations

A functioning Kubernetes cluster and the ability to install the controller and CRDs are baseline requirements, so local and air-gapped environments need planning. Steps must be containerized, which pushes image building and registry access into the workflow design. Artifact passing depends on an artifact repository the cluster can reach, such as S3 or GCS, so storage configuration is ongoing infrastructure work, not a one-time install. The project also carries a broad surface (server, UI, SSO, executors, garbage collection), and operators must decide which components to run and maintain.

Start with the quick start and the walk-through examples; a hosted demo environment and interactive training let you inspect behavior before committing a cluster.

HotRepo's take

The project dates to 2017, and HotRepo recorded one daily-board appearance on 2026-10-28 at 17,061 stars with +4 that day. That is a single snapshot; comparable daily history is insufficient, so persistence of current interest cannot be judged. The practical question is less whether the project is popular than whether its Kubernetes-native model matches your execution environment: confirm your cluster version fits the documented quick-start and that an artifact store is reachable before designing pipelines around it.