Argo Workflows

CNCF Graduated K8s-native workflow engine. Workflow CRD with DAG/steps templates, artifacts via S3/GCS, parameters + results, CronWorkflow for scheduled batch, Argo Events for triggers. Powers Kubeflow Pipelines, Couler, BlackRock's data platform.

Overview

Argo Workflows: general-purpose orchestration on Kubernetes

Argo Workflows is the Kubernetes-native general-purpose workflow engine, CNCF Graduated since November 2022. It originated at Applatix, then Intuit (2017+), and is broadly adopted at BlackRock, NVIDIA, GitHub, Adobe and NASA JPL.

Its scope is wider than Tekton's. Tekton is purpose-built for CI/CD; Argo Workflows is the general orchestration engine.

Data pipelines — Extract-transform-load
ML training pipelines — Kubeflow Pipelines is built on Argo Workflows
Batch processing — Image processing, video transcoding, scientific computation
CI/CD — One of many use cases
Scheduled and event-driven — Cron jobs, event-driven workflows

Where Argo Workflows wins

General-purpose orchestration — ML, data engineering, CI/CD and batch on the same primitives; often chosen when CI is one of many workflow types
Strong DAG model — Declarative dependencies, parallel fan-out (withSequence/withItems/withParam), conditions (when), exit handlers (onExit), retry strategies; more expressive than Tekton's runAfter + when
Kubeflow Pipelines compatibility — Kubeflow compiles ML workflows to Argo Workflow YAML; ML on K8s means Argo is in the stack
Better UI than Tekton — Polished DAG visualization and log viewer, real-time updates, suspend/resume buttons, parameter editing
Argo Events — CNCF Incubating companion for Kafka, S3, calendar, custom webhook and GitHub triggers; more flexible than Tekton Triggers for non-CI events
Memoization — Built-in caching of step outputs across runs (memoize: { key, maxAge, cache: { configMap: ... } }) for expensive deterministic steps

Where it loses

Less CI-specific — No Tekton Catalog of CI Tasks, no automatic SLSA signing (Tekton Chains), no GitHub-flavored interceptor; you write the CI integration
Verbose YAML — The templates: list shape and input/output declarations make Workflows longer than equivalent Tekton Pipelines
Operationally heavier — Workflows controller + Argo Server + optional Argo Events controllers
Smaller CI ecosystem — Build steps must be authored

When to pick it

ML pipelines on Kubernetes: Kubeflow runs on Argo regardless — Pure CI/CD shops: Tekton's Catalog, Chains and Triggers are worth more
Mixed CI + data + batch + cron in one engine, one controller, one UI — Smaller orgs without K8s-deep operations
Already on Argo CD: Workflows + CD + Rollouts + Events is "the Argo stack" — OSS projects on GitHub: GHA wins
Heavy DAG-shaped batch (image processing, transcoding, science) where fan-out + retry + memoization fit

Three load-bearing concepts

DAG templates — Declarative dependencies (depends:) and fan-out (withSequence/withItems/withParam) express complex parallelism naturally
Artifacts via S3/GCS — Object storage means artifacts survive the PVC lifecycle and are reusable across workflows
Argo Events — The broader event surface (Kafka, S3, calendar, webhooks) that Tekton Triggers doesn't cover

Key concepts

Workflow CRD — DAG with parameters and artifacts: The complete workflow definition. One Workflow resource = one execution. Notice how artifacts flow: download produces dataset → train consumes as data → train produces model → evaluate consumes. Argo handles upload/download to/from artifact repository automatically.
WorkflowTemplate — reusable workflow: WorkflowTemplate is a saved Workflow definition referenceable from other Workflows. Two scopes: - WorkflowTemplate — namespace-scoped - ClusterWorkflowTemplate — cluster-scoped Referenced from a calling Workflow: Or composed inside another workflow: Equivalent of GHA reusable workflows or Tekton's Pipeline-as-template pattern. Versioning via name suffix or Git ref (since they're typically GitOps-managed).
Artifact repositories: Configured cluster-wide via the artifact-repositories ConfigMap in the Argo Workflows namespace. Other backends: GCS, Azure Blob, MinIO, OCI registry (newer; uses ORAS to push artifacts as OCI artifacts). Per-step artifact configuration: archiveLogs: true also uploads pod logs to artifact repo, queryable via Argo UI even after the pod is gone. Compared to Tekton workspaces: Argo artifacts are object-storage-backed (durable, cheap, queryable cross-workflow). Tekton workspaces are PVC-backed (per-workflow lifecycle, faster I/O within a workflow). Different trade-offs; both valid.
CronWorkflow — scheduled batch: Schedules a Workflow on a cron expression. concurrencyPolicy options: - Allow — multiple concurrent runs - Forbid — skip new runs if previous still running - Replace — cancel previous, start new History limits prune old runs automatically. Compared to standard K8s CronJob: CronWorkflow gives you DAG, artifacts, retries, conditional execution, the full Argo Workflow feature set. CronJob is just "run a Pod on schedule." For non-trivial scheduled batch, CronWorkflow is significantly more capable. Compared to Tekton: Tekton Pipelines doesn't have a native cron primitive — you use a separate K8s CronJob to kubectl create the PipelineRun. Argo's CronWorkflow is built-in and more ergonomic for scheduled work.
Argo Events — triggers: Argo Events is the companion CNCF Incubating project that handles triggers for Argo Workflows (and other targets). Three CRDs: - EventBus — message bus (NATS by default; Jetstream-backed) that fans events between sources and sensors. - EventSource — produces events from external systems (Kafka, S3, calendar, GitHub, GitLab, BitBucket, AWS SNS/SQS, Azure Service Bus, generic webhook, Redis, MQTT, NATS). - Sensor — consumes events and triggers actions (create Workflow, HTTP request, K8s resource, AWS Lambda, etc.). Filter expressions (data filters, time filters, context filters) ensure only matching events trigger workflows. Sensor's trigger creates a new Workflow per event. Compared to Tekton Triggers: Argo Events is broader-purpose (Kafka, S3, calendar, etc. are first-class) but more complex to operate (separate controllers + EventBus + sensors). Tekton Triggers is simpler for pure-CI use cases; Argo Events is the right answer when triggers come from many event sources beyond webhooks.
Recipe: ML training pipeline with GPU + memoization: Production ML pipeline on EKS with GPU nodes, IRSA auth to S3, and memoization of expensive deterministic steps: Notable patterns: - memoize on preprocess: same dataset → cached output, skip recompute. Cache key includes dataset URI; maxAge ensures eventual refresh. - nodeSelector + tolerations: schedules training pod on GPU nodes (Karpenter provisions them on demand). - IRSA via serviceAccountName: pod's S3 access uses federated identity, not stored credentials. - when expression on register: only register the model if accuracy passes threshold. - onExit: notify regardless of outcome. This is the canonical ML pipeline pattern Kubeflow Pipelines also generates under the hood.
Recipe: parallel batch processing with withParam: Process a list of items in parallel, fan-in results. Common pattern for image/video/document batch jobs. withParam: '{{tasks.list-images.outputs.parameters.image-list}}' fans out: list-images outputs a JSON array of image keys; process-one runs once per element with {{item}} interpolated. Argo Workflows handles concurrency: spawns up to parallelism: N pods at once (configurable cluster-wide or per-Workflow), waits for all to complete before continuing the DAG. Equivalent in Tekton: Tekton has matrix at the Pipeline level (Tekton v0.45+) but it's less ergonomic than Argo's withParam for dynamic lists. For batch processing of computed lists, Argo wins.

When to use it

ML pipelines on Kubernetes (Kubeflow Pipelines runs on Argo)
Mixed workload orchestration: CI + data + batch + scheduled in one engine
Already on the Argo stack (Argo CD + Argo Rollouts + Argo Events)
Heavy DAG-shaped batch processing (image/video, scientific compute)
Event-driven workflows beyond CI (Kafka, S3, calendar triggers)

Interview questions

When do you pick Argo Workflows over Tekton?
Walk through ML pipeline architecture on Argo Workflows.
Fan-out / fan-in patterns — withSequence vs withItems vs withParam.
How do you handle artifacts and parameters across Argo Workflow steps?
Quick-fire interview answers — Argo Workflows essentials.