AI-Native Series · ANDON · Agentic PR Review and CI/CD · Episode 4 of 5
The Auto-Merge Envelope: Blast Radius Is a Path List
Agentic PR review and CI/CD, episode 4. On 2026-09-01 GitHub shipped a setting under which a Copilot approval can count toward a repository's required approvals. It is off by default, and the debate it started is about whether the AI is good enough to approve. That is the wrong question. The question is: approve what? This episode is the O in ANDON, only inside the envelope. It builds an auto-merge envelope that is a list of paths rather than a confidence score, with five conditions that must all hold at once, and reports the day a real PR passed the envelope, failed on exactly one condition, and could not have merged anyway because the branch-protection API returned 403.
1-minute takeaway — what you'll walk away with
What this is. Episode 4 of ANDON, a five-part series on agentic PR review and CI/CD. This one is the rule only inside the envelope, built into a real repository as an auto-merge envelope: an allow list and a deny list of paths, five conditions that must hold at once before a bot may approve and merge, two gates that check who wrote the change and how big it is, and the honest state of the one thing that is still missing.
Why it matters. For an engineer: the merge button has moved. A vendor's AI may now approve, the same vendor's agent will resolve review feedback and failed checks for you, and the only design decision left is the setting: where, exactly, that is allowed. For a founder: Stripe merges over a thousand agent-produced PRs a week with human review; the merge queue, not the reviewer, is the narrow pipe now. For a student: the oldest idea in this series, a limit written down, applied to the newest button.
What you can do after reading. Write the envelope as a path list in data, with deny winning over allow. Make auto-merge the conjunction of five checks, and list every failed condition rather than the first. Refuse bot sign-offs in the commit trailer. And report the state of your branch protection as it is, because an envelope that classifies is not yet an envelope that merges.
ANDON — the series spine
- A · Assess — The PR flood is measured, not felt episode 1
- N · Never silent — Jidoka for repositories episode 2
- D · Data, not prose — The rule that cannot drift episode 3
- O · Only in the envelope — Blast radius is a path list this episode
- N · Negative test — A gate is installed only once it has gone red episode 5
What this episode covers, and why the auto-merge envelope comes fourth
Episodes 1 to 3 built the parts that decide: the machine gate that has no opinions, the third state that says when a gate could not run, and the rules file that everything else is rendered from. This episode is about the one action those parts can now take on their own, which is to merge. It comes fourth because the action is only safe once the three earlier rules hold; an envelope drawn around a two-state gate with rules in prose is a hole.
The thirty days before this episode moved the button. On 2026-09-01 GitHub announced that a Copilot code review approval can count toward required-approval rules, off by default and configurable per enterprise, organisation or repository, dismissed on new pushes as a human's would be, and with one sentence that matters: "An approval assessment alone does not count toward merge requirements." [1] Three days later the weekly release notes described the next step: "Agent Merge is now in public preview and gets your pull request ready to merge by resolving review feedback, failed checks, and merge conflicts." [2] Stripe had already reported, in February, that "Over a thousand pull requests merged each week at Stripe are completely minion-produced, and while they're human-reviewed, they contain no human-written code." [3] And the 2026 merge-queue report found that AI-assisted PRs, 14.4 percent of private merges in its sample, "broke main half as often, even after controlling for PR size and repository." [4]
So the argument "the AI is not good enough to merge" is losing on the evidence, and the argument "the AI is good enough" is the wrong shape. Goodness is a confidence score, and a confidence score does not tell you what happens when it is wrong. Blast radius does. This episode's rule: where a bot may merge is a list of paths, and the list is the design.
The 30-year version: test the merge result, then serialise the door
The software lineage of safe automatic merging is a lineage of narrowing what "merge" means. In 2013 the Rust project's bot, bors, made the merge result the thing that gets tested, in a sentence from a 2015 description: "Homu will take the pull request, merge it with master into a new branch, and submit that branch to a testing backend." [5] A green PR is not the same as a green main, and bors was the first widely copied admission of that. In July 2023 GitHub shipped its merge queue generally, describing the goal as "ensuring the branch is never broken by incompatible changes." [6] Chromium's commit queue added the retry rule that every large project ends up with, stated with unusual honesty in its documentation: "If the retry succeeds, then the failure is ignored." [7] And by July 2026 the observation that the reviewer was no longer the bottleneck had a name: "The constraint didn't disappear. It moved downstream, to the narrowest pipe in the system: the serialized path between 'approved' and 'on main.'" [8]
Two more strands matter for the envelope. The Linux kernel's rule for coding assistants is the clearest statement anywhere of the line between a machine's output and a human's responsibility: "AI agents MUST NOT add Signed-off-by tags. Only humans can legally certify the Developer Certificate of Origin (DCO)." [9] The kernel asks for the same declaration as an assisted-by trailer; the Apache Software Foundation's name for it is a generated-by token in the commit message [10]. And the 2026 census of AI policies in the top two thousand repositories found 83.3 percent permit AI use, 48.8 percent require disclosure and 14.9 percent forbid it [11]. The census records who may write; it does not record where a bot may land, and that is the question this episode is about.
The kitchen picture from episode 1 has a name for this. The cold station. Salads, sauces, the small plates: the sous-chef tastes them and sends them out without calling the chef, because if one is wrong the cost is one plate. The hot line never works that way. Nothing off the grill leaves without the chef, however good the sous-chef is, because the cost of a wrong plate there is the evening. The envelope is the list of what the cold station may send, and it is written on the wall, not carried in the sous-chef's confidence.
| Kitchen | Repository |
|---|---|
| The cold station's list: which plates may go out without the chef | The allow list: docs, plans, tests, case data, Markdown and spec files |
| Anything that touched the grill goes to the chef, even a garnish | The deny list, which wins: workflows, scripts, application code, migrations, the agent guides |
| The sous-chef must have tasted it, found nothing, and had no question for the chef | Conditions 3 to 5: AI measured with verdict merge, zero cited findings, zero human-only items |
| The pass is closed tonight; the plates are ready but nobody may send them | Branch protection returns 403 on this plan: the envelope classifies, the merge step has never run |
The build: five conditions, two gates, and the state of the door
1. The envelope is data, and deny wins
The envelope lives in the same rules file as episode 3, under its own key: a list of allowed path prefixes, a list of allowed suffixes, and a list of denied prefixes. Allowed: the docs and plan directories, the case library and its intake folder, the agent-configuration directory, the covenant directory, the backend and frontend test trees, and any Markdown or test-spec file by suffix. Denied: the whole GitHub configuration tree including the workflows, the scripts directory, the Makefile, the backend application and migration trees, the dependency manifests, the frontend source, the eval runner, and the two agent guides. The check is one function: if any denied prefix matches, the file is outside; otherwise it is inside if an allowed prefix or suffix matches; otherwise outside. Deny is evaluated first and needs no model. A PR with one file outside is outside.
Why the deny list is as long as the allow list, and why the scripts directory is on it, is the point of the series: the gates themselves are the highest-blast-radius code in the repository. A change to a gate is a change to what every future PR is allowed to do. Nobody but a human may land one.
2. Five conditions, all at once, every failure listed
A PR is Tier-1, meaning the bot may approve it and open auto-merge, only when five conditions hold together: every changed file is inside the envelope; every machine gate is green; the AI review has actually run and its verdict is "merge"; the brief carries zero findings at block or warn severity; and the list of human-only questions is empty, with the brief printing the model's reason for the empty list or saying that none was given. The classifier does not stop at the first failure. It returns all of them, and the brief prints all of them under the heading "conditions not met". This is a small design decision with a large effect: a PR author reading "outside the envelope, and the AI was unmeasured, and one gate is red" fixes three things, where a PR author reading "outside the envelope" fixes one and comes back twice.
When all five hold, the workflow's own identity approves the PR with a comment saying it is doing so on behalf of the code owners for a low-risk change, and opens auto-merge so the PR lands when the required checks are green. When either step fails, the failure text goes into the brief: the repository setting that must allow workflow approvals, or the plan and protection prerequisites for auto-merge. The bot never approves anything outside the envelope, however good its review was. The precursor article's reason still stands: an AI reviewer whose approval is a merge condition becomes something people optimise against [12].
3. Two gates the envelope depends on: who wrote it, and how big it is
Two gates from the flood specification sit underneath the envelope. The provenance gate has three parts. Any commit that carries an agent marker in its trailers must also carry a trace marker, a session link or an assisted-by line, so that the agent's work can be found again. Any PR whose commits are agent-assisted must carry a verification section in its body, with the evidence pasted in, and, per episode 2, a push event with no body is "unmeasured" rather than a failure. And no commit may carry a sign-off in a bot's name, which is the kernel's rule made mechanical: the sign-off is a human's legal act, and a trailer with a bot address in it is red.
The size gate reads the thresholds file from episode 3: warn at 400 effective lines, block at 3,000 lines or 60 files, and block an agent-assisted PR at 1,500. The lower limit for agents is not a judgment about agents. It is a judgment about reviewers: the thirty-month record says the agent PRs that do not merge tend to be the large ones [13], and a human asked to review 1,500 agent-written lines is being asked to do the thing the 2006 review data, with its 200-to-400-line window, says cannot be done well [14].
The proof, and the door that is still closed
On 2026-09-09 the envelope was tested on the only kind of PR it is meant for: a docs-only change, the builder-loop decision record: five Markdown files, two under the docs directory plus the README pair and the changelog, every one inside the envelope by prefix or by suffix. The brief on that PR reported the tier classification with exactly one unmet condition: "AI review unmeasured". The path envelope had passed; the gates were green; the AI had not run because the repository's key was still rejected with a 401. That is the correct outcome, and it is the first practical evidence that condition 1 works on a real diff and that the classifier reports the remaining failure rather than a generic "needs human".
The honest half of the finding is what would have happened with a valid key. Nothing. The repository is private on a plan under which the branch-protection endpoint returns a 403 with the words "Upgrade to GitHub Pro or make this repository public", the auto-merge setting reads back as false, and the approve-and-merge step has therefore never executed. On the day of writing the owner's plan does not read back as Pro on the token this repository uses, and the series will not describe a step as working because a billing page changed. The loop audit from episode 2 carries the criterion "Tier-1 has merged once inside the envelope" and keeps it at "unmeasured", alongside "the AI has measured a PR". Both will turn green on evidence and not before. The script that applies branch protection fails closed with the upgrade URL, so the day the plan reaches the repository, one command installs the door.
Five whys: why the envelope is a path list and not a confidence score
| # | Why? | Because | Evidence |
|---|---|---|---|
| 1 | Why not let a sufficiently confident AI review merge anything? | Because confidence describes how likely the review is right, and the merge decision needs to know what happens when it is wrong; those are different quantities. | GitHub's own rule that an approval assessment alone does not count toward merge requirements [1] |
| 2 | Why is "what happens when it is wrong" a property of paths? | Because in a repository the cost of a wrong merge is set by what the change can reach: a wrong docs page costs a correction, a wrong migration costs the data, a wrong gate costs every future PR. | The deny list contains the gates themselves; a change to a gate changes what all later PRs may do |
| 3 | Why must deny win over allow? | Because a PR is one unit and lands whole; one file on the hot line makes the whole plate a hot-line plate, and evaluating deny first means the allow list can be generous without being dangerous. | The envelope function checks deny prefixes before allow prefixes and suffixes |
| 4 | Why five conditions rather than the envelope alone? | Because the envelope bounds the cost of a wrong merge; the other four bound the probability. bors taught that a green PR is not a green main, and episode 2 taught that "could not run" is not "found nothing". | [5, 6]; condition 3 fails on "unmeasured" |
| 5 | Why report the 403 instead of waiting to publish until the door opens? | Because a series about never rounding up cannot round up its own worked example; the envelope's classification is evidence, the merge step is not yet, and Shewhart's "within limits" applies to claims as well as parts [15]. | The loop audit's P6 row is UNMEASURED; the dossier records the 403 and the false auto-merge flag on the day |
The root, in one sentence: the merge button's safety is a function of what it can reach, and what it can reach is a list you can read, diff and deny, which a probability is not.
Build it in 30 minutes: the envelope contract
Thirty minutes is the contract's budget for a reader who copies it; it is not a time I measured. The envelope and classifier took an afternoon; the provenance gate a morning; the proof was a PR that already existed.
| Element | Specification |
|---|---|
| Envelope | In the rules file: allow_prefixes, allow_suffixes, deny_prefixes. Deny is evaluated first. The gates, workflows, application code, migrations, dependency manifests and agent guides are denied. One file outside ⇒ the PR is outside. |
| Five conditions | inside the envelope ∧ every machine gate green ∧ AI measured with verdict "merge" ∧ zero block/warn findings ∧ zero human-only items, the brief printing the model's reason for the empty list or saying none was given. The classifier returns every failed condition. |
| Action | All five ⇒ the workflow identity approves with a comment naming the rule, then opens auto-merge. Any failure of the action itself is written into the brief with the setting or plan it needs. |
| Provenance gate | Agent marker ⇒ trace marker required; agent-assisted PR ⇒ verification section in the body (no body ⇒ unmeasured); bot sign-off ⇒ red. |
| Size gate | From the thresholds file: warn 400, block 3,000 lines or 60 files, agent-assisted block at 1,500; effective lines exclude generated files. |
| Door state | Branch protection and auto-merge are read from the API and reported in the audit as they are; the protection script fails closed with the upgrade URL until the plan is active. |
| Deliberately missing | No confidence threshold; no "AI may approve application code with N reviewers"; no bypass list. Each is a way for the hot line to leave without the chef. |
# The envelope is one function; deny first, no model.
def in_envelope(path, env):
if any(path.startswith(d) for d in env["deny_prefixes"]): return False
return any(path.startswith(a) for a in env["allow_prefixes"]) or \
any(path.endswith(s) for s in env["allow_suffixes"])
# Tier-1 is a conjunction; every failed condition is returned, not the first.
def classify_tier(brief, env):
reasons = []
if any(not in_envelope(f, env) for f in brief.files): reasons.append("outside the envelope")
if any(not g.ok for g in brief.gates): reasons.append("a machine gate is red")
if not brief.measured: reasons.append("AI review unmeasured")
else:
if brief.review.needs_human: reasons.append("human-only items present")
if brief.review.hot_findings: reasons.append("block/warn findings present")
if brief.review.verdict != "merge": reasons.append("AI verdict is not merge")
return ("auto" if not reasons else "human"), reasons
Same story, five exits
| Reader | The one decision | The one action |
|---|---|---|
| Student | "Good enough" and "safe enough" are different questions. | Take any automated action you rely on and write down where it is allowed to act, as a list. |
| Engineer | Deny first, then allow; five conditions; every failure listed. | Add the envelope to your rules file and make your bot's approve step read it before it reads the model. |
| Founder | The merge queue is the bottleneck now, not the reviewer. | Decide this week which directories a bot may land in without a human, and write the reason next to each. |
| Executive | The setting is the design. | Ask who can flip the AI-approval setting in your organisation and what path list constrains it once flipped. |
| Investor | A team that reports its own 403 will report its numbers. | Ask a team to show you the PR where their auto-merge was refused, and why. |
Patterns, anti-patterns, and the first principle
Patterns that held. Envelope as data with deny first. A conjunction of conditions, all returned. Approval by the workflow identity with a comment naming the rule. Provenance as three mechanical checks. Size limits from a file that cites its basis. The door's state read from the API and reported unmeasured until it has opened once.
Anti-patterns this replaced. "The AI can approve if it is confident." A bot that stops at the first unmet condition. Agent trailers nobody checks. A sign-off with a bot's address in it. A release note that says auto-merge works because the plan was purchased.
The mechanism, named. The envelope bounds the cost of a wrong merge and the other four conditions bound its probability; the product of the two is the risk, and only the first factor is under the repository's direct control, so it is the one written as a list.
The first principle, in one sentence. Let the machine act where a mistake is cheap to reverse, and write "cheap" as a set of paths, not a feeling.
Reality mission
Episode 3 promised that by episode 4 the envelope would have been tested against a real docs-only PR with the branch-protection state reported honestly. Both are done: the docs-only PR passed the envelope and failed on one condition, and the door is 403. By episode 5, every gate in the repository will have been proven able to go red by its own self-test, the negative-test PR from the precursor article will either have produced the reviewer's first measured finding or still be waiting on the key, and the article will say which.
Read next
Episode 4 of ANDON · Agentic PR Review and CI/CD. Every number is scoped where it appears; the kitchen is a model, not a measurement. The envelope, the classifier, the two gates and the 403 exist in a private repository; their shapes are reproduced in the contract so nothing depends on access to it. The four-horizon dossier this episode cites from is logged in the portfolio's plans directory.
References
- GitHub Changelog, Copilot code review can now approve pull requests, 2026-09-01. github.blog
- GitHub Changelog, weekly releases for 2026-08-31, posted 2026-09-04. github.blog
- Stripe, Minions, 2026-02-09. stripe.dev
- Mergify, State of Merge Queues 2026, 2026-07-27. mergify.com
- Wilson, Rust infrastructure can be your infrastructure, 2015-03-17. huonw.github.io
- GitHub, Pull request merge queue is now generally available, 2023-07-12. github.blog
- Chromium, Commit Queue documentation. chromium.googlesource.com
- Pan, The Merge Queue Is the New Bottleneck, 2026-07-02. tianpan.co
- Linux kernel documentation, Using AI coding assistants. docs.kernel.org
- Apache Software Foundation, Generative Tooling Guidance. apache.org
- Hora, Robbes, Zacchiroli, "We Permit the Use of AI, but […]", arXiv:2609.07542, 2026-09-07. arxiv.org
- Paul Jialiang Wu, AI PR Review That Cannot Lie, 2026-09-08. agentic-portfolio-lovat.vercel.app
- Ehsani et al., Where Do AI Coding Agents Fail?, arXiv:2601.15195, 2026-01-21. arxiv.org
- SmartBear, Best Practices for Peer Code Review (Cisco study, 2006). smartbear.com (vendor source; interest flagged by Rigby and Bird)
- Shewhart, Economic Control of Quality of Manufactured Product, Van Nostrand, 1931 (archive.org scan). archive.org