AI-Native Series · Agent Engineering, Compared: Prompt · Context · Harness · Loop · Graph

Graph Engineering Is a Schedule, Not a Cast of Personas

Episode 5 of 6. This series compares the ways people engineer AI agents — prompt, context, harness, loop and graph engineering: where each came from, what it controls, who drives it, which principles survive measurement. This episode is the rung where one agent stops being enough, and where almost everybody buys the wrong thing.

By Paul Jialiang Wu · agentic-portfolio-lovat.vercel.app · 2026-09-11 · Episode 5 of 6

What this episode covers, and why graph engineering comes fifth

There are now six named disciplines for building AI agents, arrived at in roughly this order: prompt engineering (2020), loop and evaluation engineering (2022), graph engineering (2023), harness engineering (2024), context engineering (2025). The clearest published statement of the first five is Feng et al.'s survey (arXiv:2608.21156), which orders them by what each lets you control rather than by age: prompt → context → harness → loop → graph. The sixth rung is not theirs and not a citation — evaluation engineering is this series' own call, and Episode 6 argues for it rather than assuming it. This series takes one per episode, in that order, and asks the same four questions of each — where did it come from, what do you control, who drives it, and which failure pushes you to the next rung.

Graph engineering is the rung you reach when the thing in short supply is no longer the agent's ability but its organisational capacity: several kinds of expertise at once, sub-tasks that depend on each other, work that can run in parallel, a result somebody has to check, state that survives between steps. You stop tuning one worker and start designing who hands what to whom — and this is where the industry's most confident engineering is being done on its least checkable object, a list of job titles.

Who this is for. An engineer about to add a second agent, or debugging a system with five. A founder deciding whether "multi-agent" is an architecture or a pitch deck. An executive told the answer is more agents, who would like to see the bill. A high-school builder who wants to know what "orchestration" means without the org-chart mysticism. Prerequisite: having chained two model calls once.

Where graph engineering came from, and who is driving it

[DEF] A graph here is what it is in a discrete-maths class: nodes and directed edges. The nodes are units of work — an agent, a tool, a check. The edges say who may hand work to whom. Graph engineering is designing that structure deliberately rather than letting it emerge. The history is how the industry got from "two chatbots talking" to "a compiled execution graph".

[DESIGN] Three things to notice. First, the field started in 2023 with conversation as the primitive and has spent three years replacing it with structure: CAMEL has two speakers, ChatDev an order, AutoGen patterns, LangGraph an edge list, and the 2026 work a compiler. Second, verification arrives early — MetaGPT is verifying intermediate results in August 2023 — then largely disappears from the marketing, which is the gap this episode is about. Third, the sharpest empirical result in the table says you may not need the graph at all: sampling independent answers and voting scales performance with no edges whatsoever.

Who drives it now. The framework vendors (LangGraph, the AutoGen lineage) own the abstractions engineers actually type. The model labs publish the production experience — Anthropic's write-up is the most detailed public account of what a multi-agent system costs to run. Google's A2A turns an edge into something two companies can share. The academic side has split usefully: one half systematising the paradigm (Feng et al.), the other half measuring how it breaks (Cemri et al., the 2026 topology papers). Read the second half first.

The mental model for the series: the kitchen ladder

One picture carries all six episodes, stated as a tool rather than a story. Every cook climbs the same ladder: for yourself, then for family, then a restaurant kitchen, then a fast-food chain, then the national supply chain feeding several chains. Every AI engineer climbs a matching one — school learner, junior developer, senior developer, AI tech lead, CTO, frontier leader.

This episode is the national supply chain: many kitchens, one schedule, one fleet of trucks. Notice what the job is at that altitude. You do not cook; you do not even taste. You decide which depot ships to which kitchen, which shipment is inspected before it reaches a shelf, and what happens to a pallet that fails. Nobody running a national food system believes the safety of the food is a property of how the drivers describe themselves.

Two axes, not one. This is not a ladder you climb and leave behind: every kitchen has all six layers at once. A home cook has a card, a pantry, a stove, a taste, a supplier, and a memory of what went wrong last time. What changes as you climb is how many layers you may touch and how rigorously you run each one: by feel, then measured, then gated, then self-correcting. A national chain with a beautiful diagram and no inspection step is less mature than a single restaurant that weighs every plate. So the disciplines are layers (what you control); seniority is maturity (how you run them); and graph is not automatically more advanced than loop. That error is the expensive one here, because a decorative graph is unusually easy to draw and unusually satisfying to present.

The analogy may introduce a claim, never evidence one — every element is pinned in the 1:1 map below. And the episode ends with the discovery that a perfectly scheduled system with an inspection on every route still cannot tell you whether the inspection is any good. That is Episode 6.

The ladder is not mine, and neither is the word "graph"

[DEF] The vocabulary, pinned to its source — because this rung has more competing labels than any other (orchestration, multi-agent systems, agent workflows, swarms, agentic mesh) and most name the cast rather than the structure.

Feng et al.'s survey (arXiv:2608.21156v2, revised 2026-08-26, read as the arXiv abstract on 2026-09-11) names the four rungs below this one, then argues for this one. Its diagnosis of why a fifth rung is needed is a list of five things, not one of them a personality:

"many tasks require heterogeneous expertise, interdependent subtasks, parallel execution, independent verification, and persistent state, exceeding any single agent's organizational capacity"
— Feng et al., 2026

Heterogeneous expertise. Interdependent subtasks. Parallel execution. Independent verification. Persistent state. Every one is a property of a structure: which node knows what, waits for which, checks which, and what survives between them. The survey calls the goal System Intelligence, defined as "an agent system's ability to organize and coordinate multiple intelligent components into a coherent, adaptive whole pursuing a shared objective", and it is explicit that you do not get there by adding headcount: "Achieving it requires more than adding agents; it demands explicit structures to organize work, coordinate heterogeneous agents, and maintain evolving execution states."

And what it builds is, in the survey's own words, "explicit, dynamic, evolving graph structures representing tasks, agents, and system states". Tasks, agents, states. Three kinds of node. Zero kinds of costume.

[OPEN] Note what the survey does not claim: that this rung supersedes the last, or that a graph beats a loop. It is a survey of principles, methodologies and applications — a map, not a benchmark result. Anyone quoting it as evidence that multi-agent beats single-agent is quoting a map as a measurement.

The claim, classed

[DEF] The claim: in a multi-agent system the edges are the design — who may talk to whom, when, carrying what state — and the personas are costume. A persona is a string in a prompt; an edge is a fact about what can reach what. Only one of the two can be checked without running the system.

[EVIDENCE] The first reason to take that literally is that somebody catalogued how these systems actually fail, and the catalogue is not about characters. Cemri et al. (arXiv:2503.13657, read as the arXiv abstract on 2026-09-11, which serves v3 revised 2025-10-26) open by declining to assume the premise: "Despite enthusiasm for Multi-Agent LLM Systems (MAS), their performance gains on popular benchmarks are often minimal."

What they built instead is a dataset — "1600+ annotated traces collected across 7 popular MAS frameworks" — and a taxonomy derived from close reading of 150 of them with human annotators, "validated by high inter-annotator agreement (kappa = 0.88)". The result, MAST:

"This process identifies 14 unique modes, clustered into 3 categories: (i) system design issues, (ii) inter-agent misalignment, and (iii) task verification."
— Cemri et al., arXiv:2503.13657

[DERIVATION] Read the three names as an engineer reads a stack trace. System design issues — the structure is wrong. Inter-agent misalignment — the edges carry the wrong thing, or at the wrong time. Task verification — nobody checked. Not one is "the Researcher agent was insufficiently curious", and a full third of the taxonomy's top level is the absence of a check.

[EVIDENCE] The second reason is the most honest production write-up on this rung. Anthropic's account of its multi-agent research system (2025-06-13) reports a large win: "We found that a multi-agent system with Claude Opus 4 as the lead agent and Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2% on our internal research eval." A real number from a real system, with the scope stated in the sentence itself: one internal eval, one lead-and-subagent arrangement, one model family.

And in the same post, the price, which almost nobody quotes:

"In our data, agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats."
— Anthropic, 2025-06-13

[DERIVATION] Fifteen times is not a rounding error; it is a second employee. Side by side, the two sentences give you the decision this rung actually asks for — not is multi-agent better but is this task worth fifteen chats. A cast list never makes you do that arithmetic. An edge list does, because every edge is a message and every message is tokens.

[EVIDENCE] The third reason is the counter-evidence, strong enough that this episode has to face it rather than footnote it. Li et al.'s More Agents Is All You Need (arXiv:2402.05120) shows that sampling many answers and taking a majority vote improves performance as you add agents — no coordination, no roles, no edges at all. If that is the effect, most of what people build on this rung is elaborate machinery for a result you can get from a for loop and a counter.

The 2026 follow-up makes the reconciliation precise. Ann, Liu and Tan's The Interaction Tax (arXiv:2608.23541, 2026-08-24) tests eleven verifier-scored optimisation tasks under matched budgets and finds talking can destroy the thing you bought several models for: "when agents read each other's complete outputs, their proposals converge within one round, erasing the diversity that motivates using multiple models". Their conclusion is this episode's thesis from the opposite direction — "multi-agent performance depends less on the number of agents than on the information they exchange".

[DERIVATION] So the rule that survives both papers: adding agents does not reliably help and adding conversation can hurt; what helps is designing what crosses each edge and when, with an independent check on the way out. Independent sampling with a vote is a perfectly respectable graph — a fan-out with an aggregator and no cross-talk — and if your problem has a cheap verifier it may be the best graph you can build. That is a decision about edges. It is not a decision about who is the Critic.

1:1 technical map

Every element of the metaphor maps to one thing you can point at in code.

[DEF] The three numbers the rest of this episode leans on, defined once:

• End-to-end error rate — over N identical runs, the fraction in which at least one field of the output is wrong.
• Cost — messages and tokens per run. An edge is not free, and a topology comparison that omits cost is an advertisement.
• Unverified path share — of all simple paths from a source to the output, the fraction that miss the verifier. Computed from the graph alone: no sampling, no model, no seed.

The third is the contribution: the first two are sampled and move with your model, your temperature or your luck, while the third is decidable. That is why it belongs in CI and the other two belong on a dashboard.

Prediction Gate

Before you scroll, commit — a prediction you did not record is one you will believe you got right.

Four specialist workers, one field each of a four-field answer. Every parameter, stated before you guess: per-worker corruption 0.15 · relay corruption 0.05 · merge corruption 0.05 · verifier recall 0.85 · 40 tokens per field per message. Three wirings: a star (a hub delegates and merges, and can corrupt one field while merging), a chain (each stage writes its own field and relays everything upstream of it), and a DAG whose four workers feed one verifier before the output.

Q1. Rank the three by end-to-end error rate, worst first. Then predict the size of the gap between best and worst.

Q2. Rank the same three by tokens per run, most expensive first.

Q3. Now take the DAG, leave the verifier exactly where it is, and add one edge from the fourth worker straight to the output — the shortcut somebody adds for latency. One field in four now skips the check. What happens to the end-to-end error rate: does it roughly double, or worse?

Answers are in the Notebook and the Failure Room. If Q1 put the chain in the middle, run the lab.

Failure Room

Break it deliberately — a mechanism you cannot break is a claim.

Failure 1 — the cast is the design. You name five agents, give them job titles, and consider the architecture done. But no runtime transition reads a job title: the router reads the graph, the aggregator reads the state, the check reads the fields. My own published work on expert panels measured this from the psychology side — persona labels did not improve factual accuracy across 162 roles and 2,410 questions, and what worked was grounding an agent in a person's corpus rather than a description of one. A corpus is data. A costume is a string. One Opinion Wearing Ten Hats has it in full.

Failure 2 — the verifier that is technically present. The one worth the price of the article. In the lab, the gated DAG and the same DAG with one bypass edge both contain a verifier node, and both pass any review that asks "do you have a checking step?". The bypassed one runs at 0.240 end-to-end error against 0.080 — about 2–3× worse at N=100, 2.4× in expectation — at exactly the same 640 tokens, because the check still runs on the other three fields. You pay for the inspection and skip it on one route. The static number catches this in microseconds: unverified path share 0.000 → 0.200. The sampled numbers catch it too, eventually, at N=100.

Failure 3 — the chain that looks tidy. The cheapest topology in the lab (400 tokens against the star's 480 and the DAG's 640) is also the worst, at 0.580: field one is written once at 0.15 and then relayed three times at 0.05, so it has four chances to go wrong against the last field's one. Error propagation is not a property of the workers but of how often the topology lets a value be re-handled.

Failure 4 — the conformity that reports itself as consensus. Make the agents talk more and they stop disagreeing, which feels like progress and is the opposite: proposals converge within one round when agents read each other's complete outputs. In my dinner-party project the persona-only chat arm caught 50 of 224 cross-cutting conflicts, and the ones it missed were not hard. Three specialists agreed unanimously on a dessert containing walnuts. The consensus was flawless. So was the allergy.

Failure 5 — the metric that is 0.000 and useless. Get unverified path share to 0.000 by making the verifier return True. Same shape as Episode 1's determinism metric reading 1.000 on an empty object: the reachability invariant must be reported next to an outcome metric, never alone. The gate proves a check is on the path. It says nothing about whether the check is any good — which is precisely Episode 6.

Answer to Q3: It roughly triples in this draw (0.080 → 0.240); the lab's parameters put the expected value at 0.206, about 2.4×. Either way the mechanism is the point: the bypassed field's corruption now survives with certainty instead of with probability 0.15. A gate you route around for one case in four is not 75% of a gate.

Notebook

Lab 05 — The schedule, not the personas.

05_graph_schedule.ipynb · Colab free tier · no API key · seconds of CPU · every cell executed before publication, and the numbers below are the numbers the run printed.

Eight cells. The Graph object holds nodes, edges, an output node and optionally a verifier; the four topologies differ in no other way.

parameters: per-worker corruption 0.15 · relay corruption 0.05 · merge corruption 0.05
            verifier recall 0.85 · 40 tokens per field per message · 4 fields

topology                            err    msgs   tokens  paths  unverified
star (hub + 4 workers)            0.510     9.0      480      4       1.000
chain (4 stages)                  0.580     4.0      400      1       1.000
DAG + verifier                    0.080    10.0      640      4       0.000
DAG + verifier, 1 bypass edge     0.240    10.0      640      5       0.200

  orphaned output   -> RAISES: no path reaches 'OUT' — the graph is
                       disconnected, not verified

[EVIDENCE] Three findings, in order of surprise.

The cheapest topology is the worst. I expected the chain between the star and the DAG; it is worse than both at a lower message count.

The bypass edge costs nothing and breaks everything. Identical token count, about 2–3× the error at N=100. The shortcut has no efficiency argument here — it does not even save a message — which makes it a clean model of the real case, where it saves a little latency, costs a lot of correctness, and nobody measures either.

The gate is boring, which is the point. Cell 5 runs the reachability check on all four graphs and prints the exit code CI would use: microseconds, no calls, fully deterministic. Two of the four fail before a single run happens — and a fifth graph, whose output nothing reaches, makes the check raise rather than score a comfortable 0.000.

The sweep in cell 6 is the cell worth stealing: it shows the bypassed graph tracking the unguarded arms rather than the gated one as the problem gets harder.

The gated column stays flat-ish while every other column climbs — the signature of a structural fix rather than a poultice, and invisible if you measure at one noise level, which is how almost everyone measures.

[DESIGN] Cell 7 is the blunt one. run_dag takes a personas argument; I passed it Senior Chef, Michelin Inspector, Nutritionist and Logistics Lead, which it attaches to the four workers and carries on the state — the cell prints the carried dict, so you can check they are really there. The error rate moved by exactly 0.000 and the token count by exactly 0, and the reason is the finding rather than an excuse: no line in that function branches on the roles. In a system whose behaviour is its graph, a costume has nowhere to enter. That is not a measurement of language models; it is a demonstration of where the causal arrow can live. The titles are still on the org chart.

[DESIGN] Why a stub. A simulator reproduces where error enters a topology and what removes it, in seconds, free, with no key. It does not reproduce magnitudes: nothing it prints is evidence for Anthropic's 90.2% or for anything in MAST. Cell 8 is the bridge — swap the coin-flips for model calls and the Graph, the gate and the three metrics do not change, because they are model-free by construction.

Projects

Two builds, from nothing. No frameworks, nothing beyond the standard library, both seeded, both run before this section was written, both in the notebook's gist.

Project 1 — real life: the-dinner-party

p1_dinner_party.py · python3 p1_dinner_party.py 100

Dinner for six. One guest is vegan, one has a nut allergy, the budget is 80 dollars, there is one oven rack, people sit down at 19:00. Three specialised stubs each write one part of the plan — agent_menu picks dishes, agent_shopping prices the basket, agent_timing assigns oven slots — and none sees the others' output. Each is individually correct and individually confident; the failures live between them. Each of the five rules that catch them needs facts from two specialists, which is the structural reason no specialist can own one:

allergen  needs  menu + the guest list      a dish contains nuts
diet      needs  menu + the guest list      the main is not vegan and one guest is
budget    needs  menu + shopping            the basket cleared $80
oven      needs  menu + timing              two dishes want the single rack at once
timing    needs  menu + timing              a dish finishes after people sit down

[EVIDENCE] Two arms, same specialists, same seeds. Arm A is a persona-only chat: they speak in turn and whoever speaks next may object. Arm B is one schedule with a verifier node: all three write into shared state, then one check node runs the five rules.

conflicts present            : 224
caught by persona-only chat  : 50    (0.223 recall)
caught by schedule+verifier  : 224   (1.000 recall — by construction, see below)

dinners served with an unresolved conflict
  persona-only chat          : 85/100  (0.850)
  schedule + verifier        : 0/100   (0.000)

by rule: allergen 83 · oven 74 · timing 36 · diet 22 · budget 9

[OPEN] What this does not prove: the chat arm's noticing probability is a dial I set to 0.30 with a decay, standing in for the tendency to agree with three confident paragraphs rather than object. It is not a measurement of any model, and the 0.223 is only as real as that dial. What is structural is that a rule needing two specialists' facts has no owner until the graph gives it one. Nobody's job description contains "the interaction between my output and someone else's". And the schedule arm's 1.000 is 1.000 by construction: the verifier node runs the same five rules that define a conflict, so it cannot miss one. The quantity this measures is the chat arm's loss, not the verifier's skill — the same vacuous-metric shape Failure 5 warns about, disclosed here because it is mine.

Project 2 — enterprise: the-incident-room

p2_incident_room.py · make gate · make bypass · make measure

An incident-response graph:

page → triage → diagnose → mitigate ─┐
                └────────→ rollback ─┼→ check → resolve
       triage → comms ───────────────┘

The invariant, in one sentence: no path from any node to resolve may skip check. A property of the graph, not of a run, decided by enumerating simple paths — no model call, no sampling, no flakiness. It is the same shape as a module-reachability rule I run in my own research harness — unpublished, so take it as a practice, not a citation.

$ make gate
  paths to 'resolve'        : 3
  paths that skip 'check'   : 0   (share 0.000)
  invariant: every path to 'resolve' crosses 'check' -> HOLDS     exit 0

$ make bypass          # after adding ("rollback", "resolve")
  paths to 'resolve'        : 4
  paths that skip 'check'   : 1   (share 0.250)
    BYPASS: page -> triage -> diagnose -> rollback -> resolve
  invariant: every path to 'resolve' crosses 'check' -> VIOLATED  exit 1

[EVIDENCE] The gate names the path rather than saying "something is wrong somewhere" — the difference between a failing build somebody fixes and one somebody disables. Over N=100 seeded incidents, four cross-field rules the check node owns:

                                            gated     + one bypass edge
rule violations raised (all runs)              63                    63
incidents blocked at the check node   52/100 (0.520)        37/100 (0.370)
INCIDENTS RESOLVED WHILE DIRTY         0/100 (0.000)        15/100 (0.150)

The first row counts violations, the other two count incidents — a single incident can raise more than one violation, which is why the columns do not subtract. Identical stubs, prompts and personas in both columns. One edge changed, and fifteen incidents in a hundred closed with an unfixed violation — monitors not green, a revenue-path outage with no customer comms, no root cause and no rollback. That bypass edge is the one a real on-call engineer adds at 03:00 on a Sunday for a genuinely good reason, and which is still in the repository in March.

Build it in 30 minutes — the contract

Everything the gate needs, in the order an engineer asks.

INPUTS   edges: list[tuple[str, str]]     # the whole schedule, as data
         output:   str                    # the node that ends the run
         verifier: str                    # the node every path must cross

OUTPUT   unverified_path_share: float in [0, 1]
         offending_paths: list[list[str]]  # named, not counted

INVARIANT   unverified_path_share == 0.000

STOP RULE   the gate never retries and never repairs.
            It refuses, and prints the path. Repair is a human's job.

def unverified_share(edges, output, verifier):
    paths = simple_paths_to(edges, output)       # every source → output
    if not paths:                                # a graph nothing reaches is not verified
        raise ValueError(f"no path reaches {output!r} — the graph is disconnected, not verified")
    return sum(1 for p in paths if verifier not in p) / len(paths)

# CI gate (make gate)
#   unverified_path_share == 0.000   → exit 0
#   anything above                   → exit 1, printing each bypass path

Deliberately missing, because leaving it out is the design: cycles (the walker refuses to revisit a node, so a retry loop forces you to decide what "a path" means before you can gate it); edges added at runtime by an agent that rewires itself (a schedule that changes during the run needs a different invariant); any judgement about whether the verifier is correct; an output no path reaches (the gate raises on a disconnected graph rather than scoring it 0.000, because a naive zero would hand a broken graph a green build); and any live model — swap the stubs for real calls and the gate does not change, because it never reads a run.

[DESIGN] Four properties make this enterprise-shaped rather than a demo, and each earns its place:

• The schedule is data, not code paths. An edge list can be diffed in a pull request. A graph implied by which function calls which cannot, so the shortcut arrives invisibly.
• The invariant is static, so it is cheap enough for every commit. A check that costs a model call gets run nightly, then weekly, then never.
• The gate names the path, not the count. "Unverified share 0.250" is a number to argue with. page → triage → diagnose → rollback → resolve is a sentence somebody fixes.
• The cross-field rules live in the check node. Every rule needing two agents' facts belongs to the node both edges reach. In an agent's prompt it is that agent's opinion; in the graph it is the system's precondition.

[OPEN] What neither project proves: both worker sets are stubs and both golden sets are synthetic. The enterprise build's honest status is runnable and gated, not deployed and measured. What would make it real is the same gate run against a production orchestration graph exported from whatever framework you use, with the bypass count reported on day one — and my pre-registered prediction is that most production graphs have at least one and nobody knows.

Same story, five exits

One spine, a door for each reader. Take one decision and one action.

Reality Mission

Fifteen minutes, on something you already run.

1. Take one system of yours with more than one model call and write it as an edge list — pairs of node names, nothing else. If you cannot, that is the finding: the graph exists, it is just not written down.
2. Name the output node and the verifier node. If there is no verifier, stop and write one; everything below is about whether it is reachable, not whether it is good.
3. Enumerate the simple paths from every source to the output and count how many miss the verifier. That is your unverified path share.
4. For each bypass path, find out who added it and why. There is always a reason and it is always a good one, which is why it will happen again.
5. Put the check in CI at a threshold of zero. Then add a bypass edge deliberately and confirm the build goes red — a gate you have never seen fail is indistinguishable from no gate.

If step 1 turns up exactly one node, you are not on this rung and you should not be. Go back to Episode 4 and make that loop measured first. A well-measured loop beats a decorative graph every time, and this episode's argument is about which of the two you actually built.

Research Challenge

Hand this to an agent and pre-register the prediction.

Take one task with a checkable output and four specialist workers. Hold the workers, their prompts and their role names fixed. Vary only the topology: star, chain, independent sampling with a majority vote, and a DAG whose every path crosses a verifier. Report end-to-end error rate, tokens per run and unverified path share for each, at three levels of worker noise. Then add one bypass edge to the gated DAG and report all three again.

Pre-register: whether independent sampling with a vote beats the verified DAG when the verifier is cheap, and whether the ordering of the four topologies is stable across noise levels or crosses over.

My prediction, recorded before you run it: sampling-and-vote wins on cost and loses on the invariant, because a vote is a verifier only when the majority is right. Somewhere there is a crossover where a cheap unverified topology beats a verified expensive one on any metric that ignores the tail; finding it on a real task would be a publishable disagreement with this episode, not a bug in your setup.

What moved

State of this rung at press time, from a live weekly feed sync across 30 pioneer sources in research-anything (make watch-sync), ranked by technical signal and never by audience reach. Every item was fetched and read as its arXiv abstract page on the sync date.

Sync date: 2026-09-11.

• arXiv:2609.05774 (2026-09-04) — Inference-Time Graph Engineering for Multi-Agent LLM Workflows, Tieu et al. Compiles a query and a set of agents into a sequence of directed communication graphs, one per reasoning stage, each edge carrying an instruction about what travels along it. Their conclusion is this episode's thesis in a sentence: "effective multi-agent orchestration depends not only on which agents communicate, but also on engineering executable workflow graphs that encode when, why, and how information should flow during reasoning".
• arXiv:2608.23541 (2026-08-24) — The Interaction Tax: When Communication Erases Diversity in Multi-Agent Teams, Ann, Liu and Tan. Eleven verifier-scored optimisation tasks under matched budgets; full-solution interaction measures as a weak default, and independent proposal generation avoids the collapse. The strongest counter-evidence to this rung existing at all, and why this article treats sampling-and-voting as a legitimate graph rather than a strawman.
• arXiv:2608.21156v2 (revised 2026-08-26) — Feng et al.'s survey, the source that names this rung and this series' spine. Its argument for a fifth paradigm is organisational, not cognitive.
• arXiv:2608.12921v2 (v1 2026-08-13, v2 2026-08-14) — Discovering Efficient and Explainable Communication Topologies … via Causal Inference, Li et al. Treats "which edges actually mattered" as causal attribution, measuring how masking each channel changes the outcome, then pruning the redundant ones. Two years ago the question was which agents to hire; this is a paper about which edges to fire.

[OPEN] What has not moved: I could find nothing publishing a reachability-style structural invariant as a shipped CI gate for agent graphs. The 2026 papers optimise, explain and prune edges; the frameworks let you draw them; nobody seems to be failing a build on "a path reaches the output without crossing the verifier". If that gate exists under another name, I have not found it — and I would rather say so than imply novelty from a shallow search.

Exit test

Check yourself without scrolling.

1. Name the five things Feng et al. say exceed a single agent's organisational capacity. How many of them are personality traits?
2. MAST clusters fourteen failure modes into three categories. Name them, and say which one a cast list can never address.
3. Anthropic reports both a 90.2% win and a token multiplier. What is the multiplier, and what decision does it force that the win alone does not?
4. Why is unverified path share the only one of the lab's three metrics that belongs in CI?
5. A system has a verifier node and still scores 0.240 where the gated version scores 0.080. What is the defect, and which of the two metrics — sampled or static — finds it faster?
6. Give the strongest argument that this entire rung is unnecessary, and the 2026 paper that sharpens it.

Cliffhanger

You draw the schedule.

Every node named, every edge deliberate, every path to the output crossing the check. The gate sits in CI at zero; somebody adds the Sunday-night shortcut and the build goes red in ninety seconds, naming the path. The walnut tart never reaches the table.

And then one week the check node approves something that should never have shipped.

Not because it was bypassed. Because it ran, read the merged state, applied every rule it owns, and said yes.

You look at the rules. They are all still there. They all still pass. And you realise that everything on this ladder — the specification, the pantry, the envelope, the loop, the schedule — has been borrowing its notion of right from one place nobody has ever measured. Worse: the moment you measure against it, everything upstream begins optimising for it, including the parts you did not tell to.

Episode 6 — The Recall Log. Evaluation engineering: the truth function every other layer borrows, why a system's real maturity is how it learns from what came back broken, and the uncomfortable fact that an eval is a target — and a target can be gamed by the very system you built to hit it.

Read next

• Prompt Engineering Is a Specification, Not a Spell — Season 7, episode 1 — the rung everything here is bounded by, and the one free-text step that beat a whole harness
• One Opinion Wearing Ten Hats — the persona question from the psychology side: 162 roles, 2,410 questions, and the conformity gate nobody ships
• Two Viral Articles Racked Up ~486,000 Views on "Graph Engineering." — what happened when the term was actually built, in 7 hours, with the evidence gates turned on the viral articles themselves
• The Governor — Season 6, episode 3 — why none of these labels can express stability, and the 1877 criterion that could

Episode 5 of Intelligence Engineering Adventures, Season 7 — The Kitchen Ladder. Claims are tagged by class — definition, derivation, evidence, engineering choice, open question — and a metaphor may introduce a claim but never serves as evidence for it. Every episode ships a Colab notebook that runs on the free tier with no API key, plus two applications built from nothing — one personal, one enterprise-shaped. The first five rungs are Feng et al.'s (arXiv:2608.21156), not mine; the sixth — evaluation engineering — is this series' own call, and Episode 6 argues for it. "What moved" comes from a live weekly feed sync of 30 pioneer sources, ranked by technical signal and never by audience reach. This article contains no material from any employer or client. — Paul Jialiang Wu · agentic-portfolio-lovat.vercel.app

References

1. Feng, Y., Xiang, Z., Yang, C., Ma, Q. et al. (2026). Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence. arXiv:2608.21156, v1 2026-08-21, v2 revised 2026-08-26. — artifact read: the arXiv abstract page, 2026-09-11. Source of the four quotations in "The ladder is not mine".
2. Cemri, M., Pan, M. Z., Yang, S., Agrawal, L. A. et al. (2025). Why Do Multi-Agent LLM Systems Fail? arXiv:2503.13657, v1 2025-03-17, v3 revised 2025-10-26. — artifact read: the arXiv abstract page as it stood on 2026-09-11, which serves v3. Source of the MAST quotation, the 1600+ traces across 7 frameworks and kappa = 0.88. Every number quoted is from the abstract; the paper's per-category percentages are in the PDF and are not quoted.
3. Anthropic (2025-06-13). How we built our multi-agent research system. anthropic.com/engineering. — artifact read: the live post, 2026-09-11. Source of the 90.2% sentence and the 4× / 15× token-multiplier sentence, both quoted verbatim.
4. The 2023 landmarks in the history table, each read as its arXiv abstract page on 2026-09-11 and described rather than quoted except where noted: Li, G. et al., CAMEL (arXiv:2303.17760, v1 2023-03-31); Qian, C. et al., ChatDev (arXiv:2307.07924, v1 2023-07-16); Hong, S. et al., MetaGPT (arXiv:2308.00352, v1 2023-08-01 — source of the "verify intermediate results and reduce errors" quotation); Wu, Q. et al., AutoGen (arXiv:2308.08155, v1 2023-08-16).
5. Li, J., Zhang, Q., Yu, Y., Fu, Q., Ye, D. (2024). More Agents Is All You Need. arXiv:2402.05120, v1 2024-02-03. — the counter-evidence: sampling and voting scales with the number of agents and needs no edges.
6. Ann, S. E., Liu, H., Tan, C. (2026). The Interaction Tax: When Communication Erases Diversity in Multi-Agent Teams. arXiv:2608.23541, 2026-08-24. — source of the two quotations on diversity collapse and on what performance depends on.
7. Tieu, K., Fu, D., Xia, Y., Li, H., Yan, H., He, J. (2026). Inference-Time Graph Engineering for Multi-Agent LLM Workflows. arXiv:2609.05774, 2026-09-04. — source of the "when, why, and how information should flow" quotation in "What moved".
8. Li, J., He, P., Ji, Q., Wang, W., Liu, L., Sun, C. (2026). Discovering Efficient and Explainable Communication Topologies for LLM-based Multi-Agent Systems via Causal Inference. arXiv:2608.12921, v1 2026-08-13, v2 2026-08-14 is the version read. — edge-level causal attribution and pruning; not quoted.
9. Tooling and protocol, read 2026-09-11: LangChain's LangGraph (github.com/langchain-ai/langgraph), the graph as a first-class object; Google's Announcing the Agent2Agent Protocol (A2A), 2025-04-09 (developers.googleblog.com, specification at a2a-protocol.org).
10. Series-table landmarks for the other rungs: Anthropic, Building Effective AI Agents, 2024-12-19 (anthropic.com/engineering); Dhage, S., Harness Engineering for Predictable Agentic Systems (arXiv:2608.26197, 2026-08-25), Episode 1's central measurement.
11. Wu, P. J. One Opinion Wearing Ten Hats. agentic-portfolio-lovat.vercel.app, 2026-08-06. — my own measurement of persona panels: 162 roles across 2,410 questions with no gain over no persona, corpus-not-costume, and the conformity gate.
12. Lab 05 and both projects, executed before publication: public gist — notebook, p1_dinner_party.py, p2_incident_room.py, both READMEs, the Makefile. Every number in this article came from these runs.