How We 6x'd Our PR Merge Throughput

At 5:45 PM, Slack pinged: master merge completed. One branch had just carried ten pull requests onto master in a single merge - ten PRs, all written by coding agents, landed at once. Forty minutes later, ten more. Then seven. Then five, until the queue was basically drained. In about two hours, 32 agent-written PRs went to production, each one deployed and checked by smoke tests. Not one check was skipped.

The thing that merged them was also an agent.

Until we built this, the queue just piled up. Nothing was wrong with the old serial queue except its speed: it landed one PR at a time, and our agents open PRs faster than one-at-a-time can drain them. So the backlog only grew, and it did not reliably clear even overnight, because the agents never stop opening new ones.

We call the system the landing lander. It is not a product feature, and it did not start elegant. It is an agent skill running on a /loop, plus a handful of small CLI tools, hardened over several days of running live while we fixed everything it got wrong. And it got plenty wrong. Our favorite early failure: author agents kept trying to land their own PRs the moment their checks went green, racing the owner and stomping its half-built batches - one direct push to master killed a ten-PR batch mid-assembly. Another owner once closed a perfectly healthy batch because its own watch timer got bored. Every one of those became a rule, and the rules became the system.

This post is the architecture, the real numbers from one afternoon, and enough detail to build your own. The actual skill is public now, if you want to skip ahead.

The bottleneck moved

Ottto is built almost entirely by coding agents - Claude Code and Codex sessions, often 30 or more running at a given time, some spawning sub-agents of their own. Writing code stopped being the constraint months ago. What remained was everything after: merge, test, deploy, verify.

Our old flow landed one PR at a time through a serial queue, with full verification on each. That worked out to roughly an hour per PR. With one PR in flight at a time, a 19-deep queue meant the last PR waited most of a day - while agents kept opening new ones.

The cost side hurt too. Every landing is a full CI plus deploy round on AWS CodeBuild, and with all the validation those rounds add up to a large line in our bill - a few thousand dollars a month. Nineteen landings meant nineteen rounds.

The tempting fix is to relax checks: trust the PR’s own green CI and just merge. We refused. Agent-written code is exactly the code you want fully checked - the author never gets tired, but it also never gets suspicious. The question was never “which checks can we skip”. It was “how do we keep every check while landing far more PRs”.

What every landing must survive

Here is the full journey, every time, no exceptions:

1. Autoreview. Before a PR even enters the queue, a second AI model reads the diff and hunts for bugs - a different model than the one that wrote the code. Fresh eyes, no author bias.
2. PR checks. The PR’s own continuous integration (CI) checks must be present and green on the exact commit. A missing check means retry, never skip.
3. Combined-tree validation. The queue merges the PR onto the current master and validates that combined tree: lint, type checks, and the tests that the changed files touch, plus a rebuild of any generated files.
4. Baseline check. If validation fails, the queue asks one question: does master itself fail the same check? If yes, master is broken and the PR is innocent. If no, the PR broke it, and it gets bounced with the evidence.
5. Freshness at push time. Master is re-read right before pushing. If it moved, rebuild on the new tip or abandon cleanly. A stale tree never gets pushed.
6. Merge, deploy, smoke. Post-merge CI runs, deploy workflows roll the changed services, and production smoke tests confirm the new commit is actually live and healthy.
7. Evidence. Every PR gets comments and labels recording exactly what happened: which batch carried it, the merge commit, and the deploy and smoke results.

Nothing below weakens any of these steps. The whole trick is paying for them less often.

The architecture: batch the payment, not the risk

If ten PRs ride one branch - we call it an integration PR - the pipeline above runs once for all ten. The reason teams fear batching is attribution: when the batch fails, who broke it? Our answer: spend cheap local compute on that question before the expensive pipeline, and make failure routing a normal state instead of a fire drill.

An always-on owner that costs nothing while idle

The “landing owner” is a headless agent session that owns the queue. It is packaged as an agent skill - a markdown rulebook plus scripts - and driven by a /loop: a scheduler wakes a small shell probe every 5 minutes. The probe only calls the GitHub CLI. It counts PRs in the intake labels and checks for open batches. No work? It exits. No agent spawned, no tokens spent. An idle queue costs a few API calls.

When there is work, the probe spawns a fresh owner session with the skill as its mission, on a model sized to the backlog (more on that in the cost section). If the primary engine will not start within three minutes - a quota wall, an auth hiccup - the runner hands the same job to a fallback engine with the same prompt. The queue does not stall because one vendor had a bad hour.

Exactly one owner at a time

A lease file on the machine records who owns the queue. Every command that changes anything checks the lease first and refuses to run for anyone else.

The interesting part is liveness: is the current owner actually alive and working, or has it died and left the lease sitting there? We never trust one signal to answer that, because any single signal can lie. The guard checks five: the owner’s last heartbeat, the lander’s own activity marker, when the session transcript last changed, whether the process is still running, and whether the owner said “I’m waiting on CI until 6:10”. A healthy mix keeps the lease. Silence across all five means stale, and the next probe takes the lease and starts a fresh owner where the old one left off.

One detail we love: during a long step, the runner only sends a heartbeat if the engine’s log file actually grew. Real output means real work. A stuck-but-silent process gets no heartbeat, goes stale, and gets replaced. Working hard and being wedged finally look different.

A label state machine with no ambiguous states

Every open PR carries exactly one of four labels, and the transitions are the workflow:

Zero labels means a PR is stranded: invisible to intake, unmarked for repair. So every change swaps labels in one atomic edit. We learned this the hard way - an owner died mid-change and left a PR with no label at all. The worst interruption should leave two labels for a moment, never none.

The two side exits carry a distinction that cost us a real incident. repair-requested means “a machine can fix this” - the owner spawns a fixer agent that merges master into the branch, resolves the conflict, validates, pushes, and hands the PR back. operator-hold means “a human has to decide” - and the human can approve from anywhere: a desktop dialog, Slack, or a /approve comment on the PR (a workflow checks their permissions first). Before we split those two, a PR waiting on a human sat in the repair label, and the loop kept spawning expensive owner sessions to “fix” something no agent could fix. Five spawns in 24 minutes, each correctly deciding there was nothing to do.

Batch assembly: bisect to green, then check the clock

The batch builder takes up to ten ready PRs, oldest first, and merges them into one branch. If the combined tree fails, it does not give up. It rebuilds from empty, adding PRs one at a time and keeping each only while the tree stays green. The ones that break it get dropped and sent to repair with the evidence. The rest ship. The batch goes out with whoever passes.

Then, right before opening the batch PR, the builder re-reads the master tip. If master moved during assembly, it drops anything that already landed, rebuilds the survivors on the fresh tip, and only then hands it off. This freshness check exists because of one bad afternoon: a ten-PR batch was assembled over 25 minutes while a direct push landed content that was already inside the batch. The handoff failed, the batch was torn down, 40 minutes gone.

Right of way: the push gate

That incident exposed the deeper rule: under load, the queue must be the only lane that moves master. We had that rule written down. It did not matter, because a long-running agent session loaded its instructions days ago and never reads the update.

So the rule became mechanical. A git pre-push hook, installed on every clone, refuses any direct push to master whenever three or more PRs are ready or batched, or any batch is open. Getting around it takes an explicit incident flag with a reason, which is logged and alerted to Slack. And if GitHub cannot be reached to check the load, the hook lets the push through - a broken safety mechanism must never block an emergency fix.

Policy text cannot reach a long-running agent session that loaded it days ago. A hook runs fresh at push time. When agents are the workforce, the rules you care about have to live in mechanisms, not documents.

One afternoon

The date does not matter; the shape of the run does. All times local.

It started at 4:59 PM. The owner had ten ready PRs, so it built one batch branch carrying all ten, validated the combined tree once, and handed it to the queue. At 5:44 the queue merged it - ten PRs on master in a single commit. That is the 5:45 ping at the top of this post.

Forty minutes later, ten more. Then a batch of seven. Then five. By the time the last one merged, about two hours after the first, 32 agent-written PRs were on master, deployed, and smoke-verified. The queue that everyone used to wait on was basically empty.

It was not a clean afternoon, and that is the point. Earlier in the day a stale base killed one batch, and a red master - someone else’s failing lint - blocked another. The lander’s job is not to prevent every failure. It is to name each one correctly, send it somewhere useful, and keep the rest of the queue moving.

Compare that to before. Serial, one PR an hour, 32 PRs would have been most of a week of queue time - and in practice the queue never fully drained, because the agents run day and night and open new PRs faster than a person can merge them. Averaged over weeks the lander runs about six times faster. On a batchy afternoon like this one, far more.

The cost mechanism, stated plainly

One batch equals one CI-plus-deploy round. A ten-PR batch pays for that round once, where landing them one by one would pay for it ten times. That part is arithmetic, not a claim.

The agent itself costs almost nothing until there is real work, and it scales in three steps. When the queue is empty, it is free: the every-5-minutes probe is a GitHub CLI call, not a model, so if there is nothing to land, no agent is spawned. Overnight, when nobody is opening PRs, the lander costs a few API calls and zero tokens.

When one or two PRs are waiting, there is no wide batch to reason about, so the runner spawns the owner on a smaller, cheaper model instead of the frontier one. If that light-queue owner wakes up to find the backlog has grown to three or more, it does not try to force a big batch through the small model - it hands off cleanly and the loop respawns at the higher tier. Only a real batch, three or more PRs, gets the frontier model, because that is where the batching and conflict-repair judgment earns its cost.

The pipeline cost - the CI and deploy round - is the same whichever model runs the shift. Tiering only saves the owner’s own thinking. But the small shifts where the cheap model runs are most of them, so the expensive model only wakes up when a batch actually justifies it.

The system supervises its own agents

Here is the part we did not see coming. The hardest thing about the landing lander was not the batching or the lease. It was getting an agent to run the same loop, well, thousands of times without drifting.

The fix generalizes, and we learned it the hard way: when you turn a task into something an agent repeats - a scheduled job, a loop, an orchestration - you need a second agent watching the first. Not forever. During bring-up, a reviewer agent reads the first agent’s actual runs, grades them against hard numbers, finds where it went wrong, and files the fix. You iterate that until the loop is boring. Once it is stable, the reviewer can step back.

For the lander, that reviewer is its own skill. It grades every owner shift: at least 70 percent of batches should carry more than one PR when there is a backlog, batch width at least 2.5 on average, no forgotten worktrees, and every deferral must show its evidence - the files, the commit, what it tried, what happens next. Owners write a short handoff note at the end of each shift, and skipping that note is itself a graded failure, because the next owner starts from it.

The failure-mode table is the artifact we would grab in a fire. Eleven ways the lander broke, every one seen live, every one now tied to the commit that fixed it. The owner that closed a healthy batch because a timer ran out. The owner that cancelled a queue run seconds before it would have merged. The fixer that handed a repaired PR back through the wrong command and left it with no label. The session that pushed its work to the background and walked away from the queue.

One of them is worth its own line. A long-running owner degraded partway through a shift and kept repeating a mistake we had already fixed - its rules had changed, but the running session had not reloaded them. The reviewer caught the drift, rotated the lease to a fresh owner, and the lesson became a rule that day: when the guidance changes, restart the owner, because a running agent never re-reads its instructions.

And here is where it goes next. Self-healing is now part of the lander’s own job, not just the reviewer’s. It watches its own runs and fixes small problems itself. We kept a close eye on that early on, half-expecting it to make things worse - and instead watched it recover cleanly, over and over. The reviewer is still there for the hard calls, but the loop increasingly repairs itself.

Every one of those failures cost us an hour or an afternoon, once. None can cost us twice, because each is now a check or a rule the next owner starts with. That loop - a failure becomes a rule becomes a mechanism - turned out to be the real product. The batching was the easy part.

What is actually in the box

Fewer moving parts than it sounds. There are only three agents. Everything else is a small tool they drive, plus one piece you bring yourself.

The three agents

• The owner - a headless agent session that runs the skill and owns the queue for one shift. It reads the backlog, decides what to batch, resolves easy conflicts, and closes PRs once they land. Exactly one owner runs at a time.
• The fixer - a one-shot agent the owner spawns at a single broken PR: merge master, resolve the conflict, validate, push, hand the PR back, clean up, exit.
• The reviewer - the agent that grades the owner (the section above): metrics, failure-mode hunting, and turning new failures into rules.

The tools they drive

• The skill (SKILL.md) - the owner’s rulebook: batching policy, the four labels, when to defer, what evidence to leave behind. The owner reads it first.
• The lander CLI - the owner’s safe hands. The owner never touches the queue directly; it calls this to move labels atomically, watch runs, and confirm deploys. Every action refuses to run unless the caller holds the lease, and it starts in a watch-only mode until you switch it on.
• The batch builder - assembles up to ten PRs into one branch, bisects to green when the tree fails, and re-checks the master tip before opening the PR. The owner calls it; it is not a separate agent.
• The lease guard - the who-owns-the-queue file: claim, heartbeat, release, takeover, and the five-signal liveness check.
• The loop and runner - the shell that wakes every five minutes, checks for work with the GitHub CLI, sizes the model to the backlog, spawns the owner, and falls back to a second engine if the first will not start.
• The push gate - the git hook that blocks direct pushes to master while the queue is loaded.

What you bring

• The server-side queue - the CI workflow that actually merges. It re-validates the batch on the server, classifies failures against the baseline, pushes, and triggers deploys. This is specific to your repo; the skills sit on top of it.

Three agents, six tools, one thing you supply.

Build one yourself

You do not need our stack. The smallest version that works:

1. Four labels. ready, batched, needs-repair, needs-human. Exactly one per open PR, and only ever change it in a single edit.
2. A batch script. Merge the oldest few ready PRs into one branch. If the combined tree fails your tests, rebuild it from scratch adding one PR at a time and drop whoever breaks it. Re-check your main branch right before you open the batch PR.
3. Let CI merge. A workflow that re-validates the batch branch against current main on the server and fast-forwards only if it is green. Never merge from a laptop.
4. An owner on a loop. Every few minutes, count ready PRs with the GitHub CLI; if there are any, spawn a headless agent whose prompt is your rulebook. Give it a lease file so two never run at once.
5. Route failures, do not stop for them. A broken PR gets the needs-repair label and a comment with the exact evidence; the rest of the batch keeps going. A one-shot fixer agent can merge main and hand it back - that part automates surprisingly well.
6. Write every failure down. A table: what broke, what now prevents it. This is the step that keeps the other five working.

Our real skill - rulebook, lease guard, runner, batch builder, prompts - is at github.com/ottto-ai/agent-skills, sanitized, Apache-2.0. Bring your own server-side queue; the README says exactly what to swap.

Why this matters beyond our repo

The usual picture of agentic development is an agent writing code while a human merges it. That picture has a scaling wall in it, and the wall is not the model. It is the slow, serial, human-paced path between “the diff exists” and “the diff is live and verified”.

Our experience: that path automates with no loss of discipline, if you treat landing as a stateful, supervised workflow instead of a button. None of the pieces are exotic. A lease. Four labels. Batch validation that can point at the PR that broke it. A freshness check before every push. A hook that gives the queue right of way. Full checks per batch. And a reviewer agent with teeth.

When your agents outrun your merge process, do not skip checks and do not slow the agents down. Make landing a system, give it one owner, batch the payment, and write down every failure where the next agent will read it.