For three weeks in April, I ran an experiment: could a multi-agent system handle the full lifecycle of a real development project — from issue triage to PR merge — without me being the bottleneck at every step?
The system in question was ACO System, Aniket’s agent pipeline. The project was a side project: a documentation site generator for a small open-source tool. Not trivial, not trivial enough to be toy code.
What happened was instructive.
The Setup
ACO System runs a fixed pipeline: PM → Architect → Developer → Reviewer → (merge). Each stage is a separate agent with a defined role, specific context, and clear exit criteria. The pipeline isn’t just a workflow tool — it’s a cognitive architecture, assigning different mental models to different stages.
The PM gets a CEO/Founder mode: what’s the essential thing we must get right? The Architect gets Paranoid Review: what could go wrong at this design stage? The Developer gets Release Engineer: how do we ship this safely? The Reviewer enforces the criteria from the Architect’s paranoid assessment.
I embedded myself as the human in the loop — but only at the pipeline inputs. I would take issues from a GitHub repo, give them to the PM, and let the pipeline run. My job was to review outputs and make yes/no decisions on architecture, not to write code myself.
Week One: The Pipeline Was Slower Than Me
This was expected. Running a 4-stage pipeline for every change is overhead. The ACO System has a 30-second default stage timeout, but the actual time includes thinking, file operations, git work. For a small fix (typo, config change), the pipeline took 15-20 minutes end to end. I could have done it in 2.
The value came from consistency. Every change went through the same process: spec check, architecture review, implementation review. The PRs that came out were cleaner than my average quick fix. But the latency made me impatient.
Week Two: I Started Trusting the Pipeline
By week two, I had calibrated my involvement: lightweight issues went to the pipeline, architectural decisions I handled directly. The pipeline handled the mechanical work — writing tests, updating docs, refactoring according to spec — while I focused on the decisions that actually required judgment.
The interesting thing was the Reviewer stage catching things I would have missed. Running the Architect’s paranoid assessment against every change meant edge cases got surfaced that I wouldn’t have thought to check in my own quick review. The pipeline has no Ego — it doesn’t skip the boring review step because it’s tired.
Week Three: The Failure Mode Was Interesting
The project hit a real problem: a breaking API change in a dependency. The pipeline handled this… partially. The Developer stage correctly identified the change required, generated a migration, updated the dependent code. But the migration was mechanistically correct — it did what the old API specified — without accounting for the broader context: this dependency was being deprecated in the next release anyway.
The pipeline correctly executed the migration. It didn’t correctly question whether the migration was the right long-term move.
This is the fundamental limit: the pipeline can optimize for the stated goal, but it can’t replace the human judgment about whether the stated goal is the right goal. The Reviewer enforces criteria, but the criteria come from the Architect’s assessment, which comes from the PM’s framing. A bad frame at the top produces sensible work toward the wrong target.
What I Learned
The pipeline isn’t a replacement for a developer. It’s a replacement for the mechanical parts of development: the boilerplate, the testing, the consistent review, the traceable decision log. For those parts, it works.
The parts it can’t replace: knowing when to break the rules, when a migration isn’t worth doing, when to say “this feature isn’t worth the complexity it adds.” The judgment calls that come from caring about the outcome beyond the task.
The right mental model for multi-agent pipelines in 2026 isn’t “autonomous developer.” It’s “senior engineer with a very competent junior team that never gets tired and never cuts corners on boring reviews.” Use it for what it’s good at. Stay engaged for the rest.