Kubernetes

CronJob schedules a pod. It does not orchestrate anything.

Kubernetes already schedules. What it does not do is express order between jobs, keep the logs once the pod is collected, or reach anything outside the cluster. Dagu runs each step as a Kubernetes Job and adds the graph, the history, and the boundary crossing around them.

Install DaguBrowse recipes
Each step is a real Kubernetes Job, not a shell wrapper around kubectl
Pod logs land in Dagu's run history and outlive the pod
One workflow can span several clusters through kubeconfig contexts
Cluster steps sit next to SSH, HTTP, and approval steps in the same graph
01

What CronJob does not do

CronJob is a good scheduler for one pod. The gaps show up the moment a batch is more than one pod, and every team hits the same three.

  • There is no dependency between CronJobs. Ordering is expressed by guessing at clock offsets, which quietly breaks the first time a job runs long.
  • History is shallow by default: three successful runs and one failed run are retained, and the logs disappear with the pods. Explaining last Tuesday's failure is often impossible.
  • If more than one hundred consecutive schedules are missed and startingDeadlineSeconds is unset, the CronJob stops scheduling entirely and waits for a human to notice.
02

Steps are Jobs, and the logs come back

Dagu creates a single-container Job per step, waits for it, streams the pod logs, and uses the terminated container's exit code. The logs are written into the run history, so they are still there after the pod and the Job are gone.

  • depends expresses order directly, so a failed extract stops the load instead of feeding it nothing.
  • retry_policy retries the step by creating a fresh Job, which is a different thing from backoffLimit restarting a pod in place.
  • Jobs are deleted after completion by default, so a nightly graph does not leave a trail of finished Jobs behind it.

Kubernetes exposes one merged container log stream, so stdout and stderr arrive interleaved as a single stream for this step type.

A three-step batch as real Kubernetes Jobs
# k8s-nightly-etl.yaml
schedule: "0 2 * * *"

kubernetes:
  namespace: batch
  service_account: dagu-runner
  resources:
    requests:
      cpu: "250m"
      memory: "512Mi"

steps:
  - id: extract
    action: k8s.run
    with:
      image: ghcr.io/example/extract:1.4.0
      command: extract --source warehouse
    retry_policy:
      limit: 2
      interval_sec: 120

  - id: transform
    action: k8s.run
    with:
      image: ghcr.io/example/transform:1.4.0
    depends: extract

  - id: load
    action: k8s.run
    with:
      image: ghcr.io/example/load:1.4.0
      resources:
        limits:
          cpu: "2"
          memory: "4Gi"
    depends: transform

handler_on:
  failure:
    run: ./scripts/page-oncall.sh

mail_on:
  failure: true
03

The work that does not fit in one cluster

This is the part CronJob cannot do at any level of effort, because a CronJob is scoped to the cluster it lives in. A workflow that touches two clusters, or a cluster and an on-premises system, has nowhere to live inside Kubernetes.

  • kubeconfig and context are settable per step, so staging and production are two steps in one workflow rather than two deployments of the same manifest.
  • A cluster step can sit between an SSH step on a legacy host and an HTTP call to a SaaS API, because the orchestrator is not itself a cluster resource.
  • A human.task step pauses the workflow for a typed approval, holding no process while it waits, then continues into the production step.
Promote a migration across clusters, with an approval gate
# k8s-promote-across-clusters.yaml
kubernetes:
  namespace: batch
  service_account: dagu-runner

steps:
  - id: migrate_staging
    action: k8s.run
    with:
      context: staging
      image: ghcr.io/example/migrator:3.2
      command: migrate up

  - id: verify_staging
    run: ./scripts/verify-schema.sh staging
    depends: migrate_staging

  - id: approve
    action: human.task
    with:
      prompt: Promote the migration to production?
      form:
        type: object
        properties:
          change_ticket:
            type: string
          confirmed:
            type: boolean
        required: [change_ticket, confirmed]
    depends: verify_staging

  - id: migrate_production
    action: k8s.run
    with:
      context: production
      image: ghcr.io/example/migrator:3.2
      command: migrate up
    depends: approve

  - id: record
    run: ./scripts/record-change.sh "${steps.approve.outputs.change_ticket}"
    depends: migrate_production

handler_on:
  failure:
    run: ./scripts/page-oncall.sh

mail_on:
  failure: true
04

Where Argo Workflows is the better answer

Argo Workflows is the Kubernetes-native answer to the same problem, and it is the right one for a large class of work. The trade is where the orchestrator lives.

  • Argo runs as CRDs and a controller inside the cluster, which is an advantage if everything you orchestrate is already there and a constraint if it is not.
  • Dagu runs as one binary and can sit outside the cluster, which is what makes cross-cluster and cross-boundary graphs possible.
  • If you want workflow definitions to be Kubernetes objects, managed by the same RBAC and GitOps flow as the rest of the cluster, that is Argo's design and not Dagu's.
05

What the executor deliberately does not expose

The Kubernetes step type is narrow on purpose. It covers the common shape of a batch step well and does not attempt to be a general manifest applier.

  • One container per step, with a single command. There is no raw Pod or Job spec passthrough.
  • Job parallelism, completions, completion mode, and success policy are not exposed, so indexed and parallel Jobs stay outside its scope.
  • restart_policy is not configurable, and Windows-specific, SELinux, AppArmor, and proc_mount settings are not available.

If a field is not documented for the step type, assume it is not supported. Work that needs a full Job spec is better applied with kubectl from a command step, or left to Argo.

FAQ

Practical questions before adopting

Does Dagu have to run inside the cluster?

No. It resolves the cluster from an explicit kubeconfig, then the normal kubeconfig loading rules, then in-cluster configuration, so it works either way. Running it outside is what makes multi-cluster and cross-boundary workflows possible; running it inside is fine when everything it touches is in that cluster.

How is this different from a CronJob that runs a script calling kubectl?

That pattern gives you ordering but nothing else: the wrapper pod's exit code hides which stage failed, retries restart the whole script, and the logs are one undifferentiated stream that vanishes with the pod. Dagu creates one Job per step, so each stage has its own status, its own retry, and its own retained log.

Do steps share a filesystem?

No. Each step is a separate Job and therefore a separate pod. Pass data between steps through object storage, a persistent volume mounted into each step, or step outputs for small values, exactly as you would between separate Jobs.

What happens to the Job when a run is cancelled?

Cancellation, kill, and timeout paths force cleanup even when cleanup_policy is set to keep, so a stopped run does not leave the Job behind. During normal operation the default is to delete the Job once it finishes.

Should we use this instead of Argo Workflows?

Only if the orchestrator being outside the cluster is useful to you. If every task is a container in one cluster and you want workflows to be Kubernetes objects under the same RBAC and GitOps flow, Argo is a better fit. Dagu suits graphs that mix cluster jobs with hosts, APIs, and human approvals.

Can I deploy Dagu itself on Kubernetes?

Yes. The official Helm chart (helm repo add dagu https://dagucloud.github.io/dagu) deploys the UI, scheduler, and coordinator, plus optional worker pools. Running Dagu in the cluster and using this step type together is a normal setup; the executor then resolves the cluster through in-cluster configuration.

Next step

Start with one workflow.

Install Dagu, move one script that runs on cron today into YAML, and decide from a real run history.

Install DaguBrowse recipes