Dagu vs Dagster
Dagu and Dagster solve different problems.
Dagster is a Python data orchestrator built around software-defined assets and lineage. Dagu is a single binary that runs YAML workflows calling commands you already have. This page explains where each one fits.
name: nightly-ops
schedule: "0 2 * * *"
steps:
- id: extract
run: python scripts/extract.py
- id: transform
run: ./bin/transform
retry_policy:
limit: 3
depends: [extract]
- id: notify
run: ./scripts/slack-success.sh
depends: [transform]One self-contained binary, no Postgres, daemon, or webserver to operate
Workflows are declarative YAML, not a Python framework
File-backed state, no external database or message broker
Executors for shell, Docker, HTTP, SSH, sub-workflows, SQL, and Agent Harness steps
At a glance
Dagu vs Dagster at a glance
Declarative YAML workflows that call commands.
Software-defined assets and lineage defined in Python.
One binary with file-backed state, no database.
Webserver and daemon backed by a Postgres database.
Scripts, containers, ops jobs, and Agent Harness steps.
Data platforms that need asset and lineage modeling.
In depth
Where each tool fits
What Dagu gives you
Dagu schedules, retries, and observes commands. You write the workflow as YAML and Dagu runs it as one process backed by files. The Web UI shows runs, logs, and history without extra services.
- Run scripts, binaries, containers, HTTP calls, SSH commands, SQL, and Agent Harness steps.
- Start as one binary and grow to queue-based or distributed execution later.
- Read run history and logs in the Web UI without standing up a database.
How Dagster is different
Dagster treats data assets as the unit of work. You declare assets in Python, and the platform tracks lineage, partitions, and materializations across them. A self-hosted install runs a webserver and a daemon backed by Postgres.
- Software-defined assets model the tables and files a pipeline produces.
- Partitions and backfills let you process and reprocess slices of data.
- IO managers handle reading and writing data so asset code stays focused.
When to choose Dagster instead
If your work is fundamentally about data assets and their lineage, Dagster is the stronger tool. Dagu has no concept of software-defined assets, no IO managers, and is not a Python framework, so it cannot model a data platform the way Dagster does.
- You want lineage and asset-level observability across dbt models and tables.
- Partitioned datasets, backfills, and IO managers are central to your pipelines.
- Your team writes pipelines in Python and wants an asset-first programming model.
FAQ
Practical questions before adopting Dagu
Does Dagu replace Dagster?
Not for data-asset work. Dagster is built for software-defined assets, lineage, and partitions, and Dagu does none of that. Dagu replaces Dagster only when you mainly needed to schedule and observe commands and the asset model was more than the job required.
Can Dagu orchestrate data pipelines?
Yes, when the pipeline can be expressed as commands, containers, HTTP calls, SSH commands, SQL, or sub-workflows. Dagu orchestrates the steps but does not track data assets or lineage between them.
Do I need a database to run Dagu?
No. Dagu keeps state in files and runs as a single binary. There is no Postgres, daemon, or separate webserver to operate, which is the main setup difference from a self-hosted Dagster install.
Next step
Start with one workflow.
Install Dagu, move one fragile script or agent task into YAML, and decide from a real run history.