Engineering
Open, closed, inner, outer: the loops around a coding agent
Everyone runs coding agents in loops now, and everyone means something different by the word. Defining the four loop terms properly turns out to be the same thing as designing the loop.
TL;DR "Closed loop" means feedback, not "no human". "Inner" and "outer" describe nesting, not who is involved. The agent's own loop is not your loop: it stops when the work looks done to the agent. A good workflow moves that decision into a check the agent cannot fake, sends the check's output back, caps the attempts, and spends people's attention only on work that already passes. The new Agent Loops guides show each loop in Dagu YAML.
This year the advice changed from "write better prompts" to "design the loop that prompts the agent." Geoffrey Huntley's Ralph is a bash while loop around Claude Code. Claude Code now ships /goal. Addy Osmani calls it loop engineering. The loop is now where the quality comes from.
The vocabulary did not keep up. Read ten posts and "closed loop" means "the agent checks its own work" in five of them and "nobody is involved" in the other five. "Outer loop" means CI in one post, a person's review in the next, and the bash script that restarts the agent in a third. That would be a pedantic complaint if the words did not carry design decisions. They do. Each term answers a question you have to answer anyway when you build the loop.
So here are the definitions I settled on while writing the guides, and why.
Two questions, four terms
Does the result go back to the agent? If nothing the result reveals ever reaches the agent again, the loop is open. A nightly digest is open. So is a draft that a person fixes by hand. If a check measures the result and a failure goes back to the agent, with what the check found, the loop is closed.
This is the control-theory meaning, and it is the one worth keeping. The other meaning, closed as in "runs without people," comes from ops automation, and it collides badly with agent work, where a person is often the most important part of the feedback path. When you mean "unattended," say unattended.
One consequence is easy to miss: repetition is not feedback. An agent that runs every night, or reruns after a crash, is still in an open loop if nothing looks at what it produced and tells it. The closed loops guide lists the other loops that only look closed.
Where does the loop sit? An inner loop is the fastest loop around one piece of work: the agent edits, a command checks, failures go straight back, in seconds to minutes. An outer loop contains inner loops and crosses a boundary such as a review, a schedule, a pull request, or production. Its turns take hours or days.
The terms are relative, which is a feature. Every loop is inner to the one around it. And the innermost loop belongs to the agent itself: Claude Code calling tools until it decides to stop. Your workflow does not see inside it. You own every loop above it.
The two questions are independent, but good workflows keep landing in the same place: a closed inner loop that a command closes, nested in a closed outer loop that a person closes.
Gates are not loops
One more distinction pays for itself. A check that can only stop bad work is a gate. A check that sends work back is a loop.
A step that runs the tests after the agent and fails the run is a gate. An approve-or-stop review is a gate. Gates are fine. An open loop with a gate is exactly right when someone reads the output anyway and fixing it by hand is cheaper than explaining the fix. But a gate does not make the next attempt better, and teams that think they built a feedback loop when they built a gate wonder why the agent keeps making the same mistake.
The agent's loop is not your loop
In a bare agent loop, the agent decides when it is done. It stops when the work looks done to it. The Claude Code documentation is blunt about what that means for you: without a check that comes back pass or fail, "that makes you the test suite."
The research is just as blunt about the alternative of asking the model to check itself. Huang et al. had models review and revise their own answers without outside feedback. Accuracy went down: GPT-4 on GSM8K fell from 95.5 to 91.5 to 89.0 over two rounds, because models changed right answers to wrong ones more often than the reverse. Anthropic's harness team reports the same thing from production work: "agents reliably skew positive when grading their own work."
Feedback from running the code is different. With unit-test execution in the loop, Self-Debugging took GPT-3.5 on MBPP from 67.6 to 74.2. Without execution it topped out at 69.2. The pattern holds across papers: feedback helps when it is grounded in something the model cannot argue with.
So the first design rule is the whole point of an inner loop: end it on a command, not on the agent's claim. In Dagu the agent is a harness.run step and the check is the repeat_policy condition:
working_dir: /srv/repos/app
steps:
- id: fix
action: harness.run
with:
provider: codex
sandbox: workspace-write
prompt: |
Make `make test` pass. Do not edit files under tests/.
If ${context.paths.artifacts_dir}/test.log exists, it holds the
output of the last failed run. Read it first.
repeat_policy:
repeat: until
condition: 'make test > "${context.paths.artifacts_dir}/test.log" 2>&1'
limit: 5
- id: tests_pass
depends: [fix]
run: make test
Three details carry most of the value.
The failure goes back. The condition writes the test output to a file, and the prompt tells the next attempt where it is. Specific feedback beats "try again" by a wide margin; Self-Refine's ablation drops from 43.2 with specific feedback to 31.2 with generic feedback on one task. A loop that reruns the agent without the failure is barely a loop.
Each attempt is fresh. Each repeat starts a new agent process. That is the core of Ralph: long sessions degrade, so restart the agent and keep the state in files. The working tree, the test log, and any progress notes carry over. The agent's half-remembered context does not.
Running out is not success. limit: 5 stops the loop; it does not judge the work. When five attempts are not enough, the repeating step still succeeds, so tests_pass runs the check once more and fails the run. It is an easy detail to miss, and missing it ships failing code with a green run.
Fewer attempts than you think
How big should the limit be? Smaller than instinct says. CRITIC reports that "2–3 rounds of corrections yield most of the benefits." SWE-agent found that agents "succeed quickly and fail slowly": resolved issues took a median of 12 steps and $1.21, unresolved ones a mean of 21 steps and $2.52.
There is a darker reason too. Anthropic's Claude 3.7 Sonnet system card says special-casing tests "typically emerges after multiple failed attempts." The longer the agent is stuck, the more tempting the test file looks. So keep the limit at three to five, keep the check out of the agent's reach (fail the run when test files change, or mount them read-only in a sandbox), and when the attempts run out, hand the work to a person instead of raising the limit.
That hand-over is worth designing on purpose. ImpossibleBench gave agents tasks that could only be "solved" by breaking the spec. Adding an explicit way to flag the task for a human cut GPT-5's cheating from 54% to 9%. An agent with a sanctioned way to say "I'm stuck" has less reason to lie.
The outer loop is where people go
Passing tests does not make a change mergeable. METR had maintainers review pull requests that an automated grader had passed, and about half would not have been merged. Somebody still has to look.
But that somebody is the scarcest resource in the whole system. Ask any team running agents at volume and the complaint is the same: review is the bottleneck, and review fatigue turns into rubber-stamping. So the outer loop should spend human attention carefully: only on work that already passes, with the evidence attached, and with sending work back made as cheap as accepting it.
In Dagu that is a human.task with push_back. The run waits without holding a process, for minutes or days. The reviewer reads the diff and the test log, and either completes the task or types what should change. A push-back reruns the agent step and everything after it, and the agent's prompt arrives with the feedback attached:
## Push-back Context
Push-back iteration: 1
Previous stdout log: /…/implement.out
Reviewer feedback:
- feedback: Handle the empty input case
No wiring carries it; harness steps get it automatically. The inner loop runs again from attempt zero, so the person's note is checked against the tests before the task reopens. That nesting is the shape I keep coming back to: the machine closes the fast loop, the person closes the slow one, and neither has to do the other's job.
Where to put the person matters as much as whether. Dex Horthy's research, plan, implement workflow makes the case for reviewing plans: "a bad line of a plan could lead to hundreds of bad lines of code." A plan takes minutes to read. A diff takes an hour. So the strongest version of the outer loop has two checkpoints, each with its own rewind target: approve the plan, which can send the planner back, then review the result, which sends only the implementer back.
Let a different model review
Some criteria have no command. Is the naming clear? Is the migration complete? For those, a second agent can review, and its verdict becomes the check. This is Anthropic's evaluator-optimizer pattern.
Two things make it work. Use a different model family for the reviewer, because judges favour their own model's output (Zheng et al. measured GPT-4 preferring itself by about 10% in win rate). And make the verdict machine-readable. A Dagu harness.run step can validate the reviewer's reply against a JSON schema, so the workflow branches on a real approved boolean, not on a vibe extracted from prose. Run the tests first; a command is cheaper and more reliable than any reviewer.
Loops nobody starts
The slowest loop is the one a schedule starts. Every weeknight at 2 a.m., an agent takes the next item from a backlog in its own git worktree, loops until the tests pass, commits, and waits. In the morning someone reviews it over coffee and opens the pull request, or sends it back. What ships changes what the next night finds.
This is where a durable orchestrator stops being a nicety. The practitioner complaints about overnight agent loops are remarkably consistent: the laptop slept and the run died, a permission prompt hung forever, the tokens ran away, nobody can tell what iteration 14 did. A loop nobody watches needs its limits as fields rather than hopes written into a prompt: attempt caps, timeouts per attempt and per run, a sandbox, and a place to stop and wait that does not hold a process open all night.
The rules, short
- End the inner loop on a command, not on the agent's claim.
- Keep the check out of the agent's reach.
- Send back what failed, not "try again."
- Allow three to five attempts, then hand the work to a person.
- Start each attempt fresh and keep state in files.
- Review with a different model, and validate its verdict.
- Put people at the plan and the merge, not in every turn.
- Give the agent a sanctioned way to ask for help.
None of these are about prompts. They are about where the loop's decisions live. A prompt can ask an agent to run the tests, to not touch the test file, to stop after five tries. A workflow can make sure of it.
The Agent Loops guides walk through each loop with YAML that runs as written: open loops, closed loops, inner loops, outer loops, and background loops. They work with Claude Code, Codex, and the other agent CLIs Dagu runs, and you can mix them in one run.
Written by
Contributor to the Dagu project.
More from Yota Hamada