Docker

Scheduled container jobs, without adopting Kubernetes to get them.

If your team runs Docker but not Kubernetes, there is no obvious place to put a scheduled containerised job. Dagu is a single binary that drives the Docker daemon you already have, adding dependencies, retries, logs, and a web UI around containers.

Install Dagu Browse recipes

No Kubernetes and no container platform required

Steps share one container, or each step brings its own image

Exec into containers your compose stack already runs

Podman works through the same Docker-compatible API

The gap between docker run and Kubernetes CronJob

Scheduling a container is the point where most setups run out of good options. Kubernetes CronJob is the mature answer, but only if you already have a cluster. Below that line the usual answer is cron calling docker run, which schedules and nothing else.

cron plus docker run gives you no retries, no dependency between jobs, no run history, and no way to see why last night failed.
Adopting Kubernetes to schedule a nightly report is a large amount of platform for a small amount of job.
A general orchestrator that treats containers as one step type, rather than as the deployment target, fits the space in between.

One container for the whole workflow

Declaring a container at the workflow level runs every step inside the same long-lived container. Steps share the filesystem, so packages installed by one step are still there for the next one.

Install dependencies once and reuse them across steps, instead of rebuilding an image or reinstalling per task.
Mount host paths with volumes and set environment variables once for the whole workflow.
Retries, dependencies, and failure handling are ordinary workflow fields, unchanged by the fact that steps run in a container.

Steps run through docker exec in this mode, so the image's ENTRYPOINT and CMD are not invoked for step commands. Put the command in the step.

A nightly job that runs entirely in one image

# nightly-report.yaml
schedule: "0 3 * * *"

container:
  image: python:3.12
  volumes:
    - ./data:/data
  env:
    - TZ=Asia/Tokyo

steps:
  - id: install
    run: pip install -r /data/requirements.txt

  - id: build_report
    run: python /data/build_report.py
    depends: install
    retry_policy:
      limit: 2
      interval_sec: 120

  - id: publish
    run: python /data/publish.py
    depends: build_report

mail_on:
  failure: true

Run maintenance inside the containers you already have running

Exec mode points a workflow at a container that is already running, such as one started by Docker Compose. The scheduled work happens inside the live application container rather than in a fresh copy of it.

Database migrations, cache clears, and queue maintenance run against the real running service.
Naming the container as a string is the whole configuration.
The workflow still gets dependencies and failure notification, which a compose file cannot express.

Scheduled maintenance against a running compose service

# app-maintenance.yaml
schedule: "0 4 * * *"

# docker compose で起動済みのコンテナに exec する
container: myapp-web

steps:
  - id: migrate
    run: php artisan migrate --force

  - id: prune_sessions
    run: php artisan session:prune
    depends: migrate

  - id: clear_cache
    run: php artisan cache:clear
    depends: prune_sessions

mail_on:
  failure: true

A different image per step

When a pipeline crosses tools, each step can bring its own image. The workflow stays one file and one schedule while the steps stay independent.

Extract with a database client image, transform with a language image, load with another client, with no image that has to contain all three.
A step-level container overrides a workflow-level one, so a mostly-uniform workflow can still make exceptions.
Image pull policy is configurable per step for pinned or frequently rebuilt images.

One workflow, three images

# etl-pipeline.yaml
schedule: "0 2 * * *"

steps:
  - id: extract
    container:
      image: mysql:8
      volumes:
        - ./work:/work
    run: mysqldump --host db.internal -u svc app > /work/dump.sql

  - id: transform
    container:
      image: python:3.12
      volumes:
        - ./work:/work
    run: python /work/transform.py
    depends: extract

  - id: load
    container:
      image: postgres:16
      volumes:
        - ./work:/work
    run: psql -h dw.internal -f /work/out.sql
    depends: transform

handler_on:
  failure:
    run: ./scripts/notify-failure.sh

mail_on:
  failure: true

What this needs from the host

Container steps talk to a Docker-compatible API. That is the only requirement, and it is also the constraint worth knowing before you plan a deployment.

When Dagu runs in Docker, mount the host socket into the Dagu container or configure a remote daemon. The daemon creates sibling containers.
A local Docker socket or a remote daemon through DOCKER_HOST both work.
Podman is supported through its Docker-compatible API by setting DAGU_CONTAINER_RUNTIME=podman.
Because Dagu is one binary, the scheduler itself needs no cluster, no metadata database, and no broker.

Mounting the host Docker socket gives workflows control of the host daemon. Enable it only for trusted workflows on an authenticated server.

Where Kubernetes is still the right answer

This page argues for a smaller tool in a specific situation, not for avoiding Kubernetes generally.

If you already run a cluster, CronJob is a reasonable place for scheduled containers and needs nothing new.
If jobs need pod-level scheduling, autoscaling, or bin-packing across nodes, that is a cluster's job rather than an orchestrator's.
Dagu has a Kubernetes step for workflows that need to submit work to a cluster while keeping the orchestration outside it.

FAQ

Practical questions before adopting

Do I need Kubernetes to run containerised jobs on a schedule?

No. Dagu talks to a Docker-compatible daemon directly, so a single host with Docker installed is enough. Kubernetes becomes worthwhile when you need cluster-level scheduling and scaling, not merely to run a container at 3am.

How is this different from Airflow's Docker operator?

Mostly in what it costs to operate. Airflow needs a scheduler, a metadata database, and a Python DAG framework before it runs a container. Dagu is one binary with file-backed state, and containers are declared as a field on a workflow or a step rather than as a Python operator.

Can steps share files with each other?

Yes. With a workflow-level container the steps run in the same container and share its filesystem directly. With per-step images, mount a shared host path into each step so the output of one step is visible to the next.

Does Podman work?

Yes, through Podman's Docker-compatible API. Set DAGU_CONTAINER_RUNTIME=podman on a self-hosted deployment and container steps behave the same way.

Can I run a step inside a container started by Docker Compose?

Yes, that is exec mode. Name the running container and the workflow's steps execute inside it, which is how scheduled migrations and cache maintenance are usually wired against a live compose stack.

Next step

Start with one workflow.

Install Dagu, move one script that runs on cron today into YAML, and decide from a real run history.

Install Dagu Browse recipes