Dagu vs Prefect
When you want orchestration without writing Python, look at Dagu.
Prefect is a Python framework for data teams who write flows in code. Dagu is a single binary that runs declarative YAML calling the commands you already have, with no database to operate. This page is an honest look at where each one fits.
name: nightly-ops
schedule: "0 2 * * *"
steps:
- id: extract
run: python scripts/extract.py
- id: transform
run: ./bin/transform
retry_policy:
limit: 3
depends: [extract]
- id: notify
run: ./scripts/slack-success.sh
depends: [transform]Workflows are declarative YAML, not Python code
One self-contained binary with file-backed state
No external database or message broker to run
Executors for shell, Docker, HTTP, SSH, SQL, sub-workflows, and Agent Harness steps
At a glance
Dagu vs Prefect at a glance
Declarative YAML that calls commands.
Python functions with @flow and @task decorators.
Single binary with file-backed state.
Prefect server plus a database, with separate workers.
Script, container, and ops automation where you want a small self-hosted runtime.
Python-centric data teams that want dynamic flows and an optional managed cloud.
In depth
Where each tool fits
Declarative YAML instead of a Python framework
Prefect models work as Python functions wrapped with @flow and @task decorators, so the workflow lives inside your codebase. Dagu keeps each step as a command and wraps schedules, retries, and dependencies around it in YAML.
- Define steps that run shell, Docker, HTTP, SSH, SQL, or sub-workflows.
- Keep workflow files readable for operators who do not write Python.
- Version the YAML in git next to the scripts it calls.
One binary with file-backed state
A production Prefect setup runs a Prefect server backed by a database such as Postgres, plus the workers that execute flows. Dagu ships as a single binary and stores run history, logs, and queue state on the local filesystem.
- Start with one download and no database to provision.
- Back up and upgrade by handling files and one binary.
- Grow from local runs to queue-based or distributed workers when needed.
When to choose Prefect instead
Prefect is the better tool when your workflows belong in Python and your team is comfortable operating its services. Some needs map to Prefect more naturally than to Dagu.
- You want dynamic DAGs whose shape is computed in Python at runtime.
- Your pipelines pass rich Python objects between tasks rather than calling commands.
- You want a managed option, which Prefect offers through Prefect Cloud.
FAQ
Practical questions before adopting Dagu
Does Dagu replace Prefect?
Not for every team. If your workflows are Python and rely on dynamic flows or passing Python objects between tasks, Prefect fits better. Dagu is the better choice when the work runs as commands, containers, or service calls and you want YAML and no database.
Can I use Dagu without writing any Python?
Yes. Dagu workflows are YAML that call commands. A step can run a Python script if you have one, but the orchestration layer itself needs no Python and no framework imports.
What infrastructure does Dagu need compared to Prefect?
Dagu needs one binary and a filesystem, since state is file-backed. A typical Prefect deployment runs a server with a database and one or more workers. Dagu has no managed cloud, so you self-host it.
Next step
Start with one workflow.
Install Dagu, move one fragile script or agent task into YAML, and decide from a real run history.