AI-NATIVE SERIES · HARNESS ENGINEERING

Everything Is a Plugin. The Gauges Are the Work.

By Paul Jialiang Wu · agentic-portfolio-lovat.vercel.app · 2026-08-17

1. Four days, 147,620 stars, and the wrong headline

deepseek-ai/deepseek-harness (dsh) went public on 2026-08-13 under MIT. When I pulled the GitHub API on 17 August it read 147,620 stars, 15,083 forks, last pushed that morning. Coverage framed it as a competitive move — an open runtime aimed at Claude Cowork, a model company claiming the layer above the model.

That framing is probably right and it is not useful to me. Star velocity tells you about distribution, not engineering. So I did what the repository's own AGENTS.md recommends — "We recommend using an agent to explore the codebase" — cloned 81 MB of it, and went looking for rules I could steal.

I found one that made me close the laptop for a minute, because it is a mechanism for a discipline I had been claiming in prose for two weeks and had never built.

2. What I actually read, and how

Four research windows, which is how I scope anything I intend to act on: what is being talked about now, what has crossed from demo to default over thirty months, what is still load-bearing after thirty years, and what survived three hundred. The last window is not decoration. It is the only one long enough to have killed the ideas that merely sounded good.

Primary sources only, and counted rather than skimmed:

• the repository itself: docs/architecture.md, root and package AGENTS.md, docs/testing.md, the .agents/notes/ tree, scripts/, and one real invariant.ts from each category;
• Agent Harness Engineering: A Survey — a 37,794-word review from CMU, UAB, Tulane, Yale, Northeastern, Stanford, Amazon, UChicago, Virginia Tech and Rutgers, mapping 170+ open-source projects onto a seven-layer taxonomy;
• the maritime and manufacturing record for the two long windows, checked against museum and standards sources rather than recalled.

Everything numeric below is either a count I ran myself or a figure quoted from a named source. Where I could not verify something, I say so.

3. The one rule worth stealing

From packages/AGENTS.md, verbatim:

Every package owns ./invariant. Register the manifest name; check an event/data relation or give empty installers package-specific No runtime invariant: reasons. Generated companions, unexplained empties, and ignored reporters fail verify-package-invariants.

What I measured in their tree:

An actual explained empty, from packages/code-runtime/code-runtime/src/invariant.ts:

/**
 * No runtime invariant: this package exposes no independent event sequence or mutable
 * data relation beyond contracts enforced at its owning seam.
 */
const install: InvariantInstaller = () => {}

Read the ratio again: 84% of the output is a signed statement that there is nothing to assert. A colleague looking at this sees waste. I see the whole design. The gate's product is not 35 assertions; it is 184 sentences that a human can disagree with, in place of 184 silences that nobody can.

And then the sentence that actually stung, because it describes every drift gate I have ever written:

Runtime invariants assert owned relationships. Check authoritative event streams or mutable data, not service or method presence, plugin metadata or effects, or fixed pure examples. Without a plausible relationship, an explained empty companion is correct.

My harness has a drift gate that fails if a documented command does not exist. It has a doctor that fails if the installed CLI hash differs from source. Both are presence checks. They catch a rename. They will pass, green, forever, on a module that has been quietly wrong since the day it was written — which is exactly the failure I keep shipping. Six days ago I shipped a bilingual card reader with 945 passing tests and found two real defects by listening to it. Presence was never the problem.

4. 1785: the claim is cheap, the gauges are the work

On 8 July 1785, in the courtyard of the Château de Vincennes, Honoré Blanc did something more radical than building modular parts: he disassembled musket locks, mixed the parts together, and reassembled them. He did not present a design and argue for it. He shuffled real components in front of witnesses and let the result speak.

To make that possible he built 63 gauges. Not 63 machines — 63 instruments for checking. Jean-Baptiste de Gribeauval had already formalised the go/no-go gauge for cannonballs, which answers a subtler question than "is this the right size": it asks whether a part falls inside a tolerance band, and returns one of exactly two answers, accept or reject.

Then the sobering part. Springfield Armory adopted the ambition in the 1810s, had 11 master gauges by 1825 — and did not actually achieve interchangeability until 1849. Sixty-four years from Blanc's demonstration to parts that reliably fit at scale. The idea was never the bottleneck. The instruments were.

This is the lens that made dsh legible to me. "Everything is a plugin" is Springfield's 1810s ambition restated with a type system. The interesting thing is that they shipped 219 gauges and 36 inspection stations on the same day as the claim. And Blanc's mix-and-reassemble demonstration has a direct descendant in their testing policy: "Registry contributions prove disposal through the HMR-safety test: dispose the fiber and observe removal." You do not certify that a plugin can be unloaded by reading its code. You unload it and watch.

5. 1876: a limit you paint yourself is not a limit

The second historical finding is the one that turned this from an interesting read into a diagnosis of my own work.

In the 1870s British merchant ships were being deliberately overloaded and lost, insured, and the crews with them — coffin ships. Samuel Plimsoll got a Royal Commission in 1872 and then the Merchant Shipping Act 1876, which made a load-line marking compulsory: a twelve-inch disc with an eighteen-inch line through it, painted amidships on both sides of the hull. A hard limit, visible from the dock, legally required.

Except the 1876 Act left the shipowner to decide where on the hull to paint it.

The Board of Trade published load-line tables in 1886; the position was fixed by law in 1890, and further settled in 1894. So the sequence is: fourteen years between "you must declare a limit" and "you do not get to choose your own limit." The first act made overloading visible. The second one made it stop.

Now look at my own instruments with that in hand.

• My agent skill file has a byte budget enforced by a test. I chose the number, and the test's own failure message says a ceiling may be raised "when the required content genuinely needs more space." I have raised it. Self-painted.
• Every delivery in my system files a report card with key results scored 0–1. Evidence tiers cap the score, and a needle caps it again by consequence — real constraints. But the author still sets the targets. My own campaign log has carried this open thread for weeks, in my own words: "the Mate still sets its own targets, and sets them low." I had diagnosed it and left it.
• dsh is not exempt: its doc-budget manifest is nine hand-chosen integers (AGENTS.md: 1950 words, docs/architecture.md: 2400 …) and its own rule permits raising a ceiling. Same Plimsoll line, painted by the owner.

The generalisable rule, and it took a 150-year-old shipping act to state it cleanly: a threshold set by the party the threshold constrains is a disclosure, not a control. Both are worth having. Only one of them saves ships. Knowing which one you have is the whole game.

6. 2024–2026: where the literature says the holes are

Agent Harness Engineering: A Survey maps 170+ projects onto ETCLOVG — Execution, Tool interface, Context management, Lifecycle/Orchestration, Observability, Verification, Governance — and its aggregate finding points exactly where I was already standing:

Execution, tool interfaces, lifecycle orchestration, and verification have the densest visible coverage… Observability and governance are thinner in open-source coverage and more often appear through commercial platforms, SDK features, or engineering writeups, suggesting that operational control has matured later than runtime and benchmark infrastructure.

Four findings from that paper that changed what I planned to build:

Most agent failures are not model failures. AgentFixer, deployed against IBM's CUGA production system with 15 validation tools, reached 64–88% issue detection and found that 38% of task failures trace to parsing errors — described by the authors as "a failure mode fully addressable by deterministic checks." More than a third of the failures in a production agent system were mechanical. That is the empirical case for gates, and it is stronger than any argument from taste.

Everyone watches; half as many judge. The paper cites LangChain's 2026 survey: 89% of teams use observability, 52.4% run offline evaluations. The authors call it the observability–evaluation gap, and name the consequence precisely: "teams can see what agents did without systematically judging whether the behavior was correct."

Final scores are the wrong instrument. From §12.3: "a failed rollout may originate from model reasoning, a misleading tool schema, sandbox misconfiguration, stale context, flaky tests, benchmark ambiguity, judge instability, or orchestration loops." The recommendation is trace-native evaluation — traces as the primary object, not the final number — and it is listed as an open problem, not solved practice.

Multi-repository orchestration is a named ecosystem gap. One of five gaps that "recur at layer boundaries," alongside cost attribution, failure recovery, cross-tool interoperability, and human-agent handoff. My fleet tooling has been living in that gap for a month without knowing it had a name.

And one principle from the same paper that retroactively justifies the least popular thing I do — deleting my own prompt text:

The harness-as-assumption principle. Every harness component encodes an assumption about what the model cannot do on its own. As models improve, observability systems must detect not only when agents fail, but also when harness components have become unnecessary overhead.

7. Three areas, and what is actually built

All four windows converge on one sentence: assert a real relation, prove it on the real assembled output, and do not let the author set the bar. That decomposes into three programmes. I built the first one this week. I am specifying the other two and saying plainly that they are not built, because the alternative is what I did six days ago — shipping a capability written against a published API that had never been run, then discovering within ten minutes of installing it that two of my assumptions were wrong.

R&D 1 — Invariant-or-explain, per module, gate-enforced (built)

anyagent invariants [repo] [--fleet] [--gate --baseline …]. Every module must carry one of two declarations in its header:

Runtime invariant: concatenating a text's segments reproduces that text EXACTLY, so
segmentation can never drop or duplicate a character.
@invariant-test: test_invariant_segmentation_is_lossless

…or an explicit No runtime invariant: <reason>. Four states: asserted (names a relation and a test that exists), declared (honest empty, with a reason), absent (the violation), broken (claims a relation but names no test, or names one that does not exist).

Three design decisions worth defending. An honest empty counts as covered — if it failed, authors would invent hollow assertions to clear the gate, which is worse than "nothing here." The gate refuses to judge quality: whether an invariant is a good invariant is a review question, and a script pretending to rule on it would rebuild the presence check one level up. With no baseline it fails rather than passes, because an ungrounded green is the exact defect the whole exercise is about.

Writing the four invariants immediately earned its keep. Claiming that my language segmenter is lossless obliged me to try to break it, so I fuzzed it over 20,000 random bilingual strings before asserting anything, and committed a seeded 4,000-case version so a failure is reproducible rather than a flake. It held. But I would never have run that fuzz if the declaration had not required me to name a test — the invariant was the forcing function, not the test.

R&D 2 — Trace-native evidence: assembled-transcript snapshots (specified, not built)

dsh has a rule I cannot argue with and have not implemented:

Model-visible ⟺ logged: anything that reaches a model request must be reconstructable from the session log; a new model-visible input requires a session event.

…enforced by a runtime invariant, with a matching testing policy: "Package tests, e2e assertions, mock/test-only compositions, and PR rationale do not replace the assembled transcript." Their coverage doc adds the line I want carved into my own: "Line coverage is necessary, never sufficient — it proves lines ran, not that the feature works as shipped." And, wonderfully: "An uncovered line is often dead code the gate is correctly flagging for deletion, not a missing test to bolt on."

My evidence that I need this is embarrassing and recent. 945 unit tests green; two defects found by putting the output in my ears. A trimmed sentence carried a "say more" invitation into the middle of a briefing, so it sounded finished before it began. A singular key result was read as a plural. Neither is detectable in a unit test of a pure function; both are obvious in one recording. Blanc did not inspect the drawings — he shuffled the parts. Springfield's descendant rule says dispose the fiber and observe removal. Mine should say: replay the assembled output and diff it.

R&D 3 — Externally-fixed thresholds (specified, not built)

The Plimsoll fix. Every threshold in the fleet gets classified as self-set or externally fixed, and the self-set ones are labelled as such wherever they are reported. A budget I chose, a target I scored, a coverage bar I picked — each is a disclosure, and should print as one. The 1876→1890 lesson is that visibility comes first and is not sufficient; the honest interim step is to stop letting a self-painted line be read as a survey mark.

8. What it measured on me: 0.27%

I pointed the new gate at 27 repositories on this machine. First run, uncorrected:

For comparison: dsh at 219/219 on the day it became public.

Two things about publishing that number. First, it is the correct result — the tool that grades everything I ship on the doctrine no evidence means not measured had never once declared what must always be true of its own modules. Doctrine without a mechanism is a mood. Second, it is ratcheted, not absolute: 1,501 silent modules would make an absolute gate red forever, and a signal that never changes stops being read. A new silent module fails the weekly run. Inherited silence is a debt with a number on it, which is the difference between a backlog and a vibe.

The four declarations are real relations, not presence checks: segmentation is lossless (fuzz-verified); a playback result conserves its parts; every spoken topic fits its budget in both English and Chinese; and an empty voice-engine selection always carries a reason — that last one being the no-silent-fallback rule finally expressed as a property of data instead of a promise in a docstring.

A correction I owe this article, earned while publishing it

Publishing this piece failed my own CI twice, and both failures were the pattern I have just spent 3,000 words saying my fleet lacks.

The first: a promo file must carry a URL, a LinkedIn post, a comment and a thread, or the build fails. I shipped it without the thread. The second is the better one — an article must have translations in four languages or an explicit, dated, expiring deferral, and the gate's own error message reads: "no translation AND no explicit deferral — the silent-gap state this gate exists to forbid."

That is invariant-or-explain. It has been running in my portfolio repository for weeks. It even has the expiry my new gate does not: a deferral older than 90 days fails. So the honest version of the 0.27% is narrower than the headline number suggests — the fleet has almost no per-module runtime invariants, and it does have this discipline at repository level, in at least one place, built before I read anything from DeepSeek. I did not connect the two until a gate I wrote refused to publish an article about gates.

Which is the argument, arriving from the wrong direction: the mechanism was already in my hands and I could not see it, because it was not named as a general rule anywhere. Reading someone else's codebase is often not learning a new idea. It is being handed a name for one you were already using in one place and had never generalised.

9. What I deliberately did not copy

Reading an impressive repository, the failure mode is imitation. Three things I am leaving alone, with reasons:

219 packages. That granularity comes with a team and a per-file 100% coverage gate. For a one-person-plus-agents fleet it would produce ceremony, not clarity. Steal the gauge, not the org chart.

693 Agent Notes. For a repository four days into public life, that implies a long private history, not a target. What I am stealing is the lifecycle — proposed/ → implemented/ → archived/, a closed class set, and one excellent rule: an archived note is permanently frozen and must not be treated as authority for current behaviour. Stale documents cited as truth is a failure mode I have and had not named.

"Do not use metaphors." Their prose standard bans them and demands exact nouns instead of contract, boundary, shape. For contract prose that is straightforwardly right, and I have adopted it there. But my system narrates every delivery as a chapter of one continuing story, deliberately, for a reader who is not going to read a changelog — with grades and evidence printed verbatim beside it, never inside it. Both rules are correct in their own domain. The mistake would be applying either everywhere.

10. How this gets applied, not just noted

Shipped this week, so the finding cannot decay into a bookmark: the gate exists as a command; the weekly fleet workflow runs it on the same ratchet as the six-dimension audit and uploads its report as an artifact; the baseline is committed; the contract lives in the upgrade playbook as §8 alongside the measured 0.27%; and 22 new tests cover the gate and the four declared invariants, including the case where the gate has no baseline and must refuse to pass.

The proactive half is a rule for new work rather than a cleanup project. A new module declares its invariant or its reason when it is written, while the author still remembers what must always be true. Retrofitting 1,501 modules is not the goal and would mostly produce 1,501 hollow sentences. The goal is that the number only moves one way.

Honest limits

• Four days is not a track record. dsh is in developer preview and says so loudly: "THERE WILL BE COMPATIBILITY-BREAKING CHANGES." I am reporting its disciplines, which I can read and count, not its durability, which nobody can know yet.
• I ran their gates' logic, not their gates. My counts come from the filesystem — packages, companions, explained empties, gate files, notes. I did not execute their CI.
• My 0.27% depends on my definition of a module. I scope to the engine source root and exclude tests, package markers, pure type files and vendored trees. A first pass that scanned everything returned 196 modules for this repo instead of 178. Different scoping, different denominator.
• Two of the three programmes are not built. R&D 2 and 3 are specified. Saying otherwise is the exact mistake I made last week.
• The survey's figures are theirs, not mine. The 89%/52.4% split, the 38% parsing-error finding and the 64–88% detection range are quoted from named sources in that paper; I have not replicated any of them.

One number for the road. Between Blanc's demonstration and interchangeable muskets at scale: 64 years. Between "declare a load limit" and "you may not choose your own load limit": 14 years. Both gaps were spent building instruments, not having ideas. If your harness is a plugin architecture with no gauges, you are in 1785 — which is a fine place to be, as long as you know what year it is.

Sources

1. deepseek-ai/deepseek-harness — repository, docs/architecture.md, AGENTS.md, packages/AGENTS.md, docs/testing.md, .agents/notes/README.md, scripts/. Counts taken from a shallow clone on 2026-08-17; metadata (147,620 stars · 15,083 forks · created 2026-08-13) from the GitHub API the same day.
2. Cordis and A Programming Paradigm for Spatiotemporal Composability — the plugin framework dsh is built on.
3. Li, Xiao, Zhang, Liu et al., Agent Harness Engineering: A Survey (the copy I read and quote from; also on OpenReview as eONq7FdiHa, which serves a browser-verification wall rather than the PDF) — ETCLOVG taxonomy, 170+ project mapping, the observability–evaluation gap, the harness-as-assumption principle, §12 open problems.
4. RUCAIBox/awesome-agent-harness — the survey's companion project index.
5. Royal Museums Greenwich — Samuel Plimsoll and ship safety and Lloyd's Register Foundation Heritage — What is the Plimsoll line? — Merchant Shipping Act 1876, the owner-chosen position, Board of Trade tables 1886, position fixed in law 1890.
6. 230 Years of Interchangeable Parts and go/no-go gauge — Blanc's 1785 demonstration and 63 gauges, Gribeauval's tolerance bands, Springfield's 11 master gauges by 1825 and interchangeability in 1849.
7. Chatterbox, Qwen3-TTS — the licence-gated voice work referenced as this week's own evidence.