Paul Jialiang Wu agentic-portfolio 中文 Español 한국어 日本語✉️ Free list
← Back to portfolio

AI-Native Series · Agentic Engineering

The App Kept Saying “Done.” It Was Lying. So We Made It Audit Itself.

By Paul Jialiang Wu · agentic-portfolio-lovat.vercel.app · 2026-08-01

1-minute takeaway — what you'll walk away with

An app I'm building with an AI co-developer told users “login link sent!” — with no email service behind it. Instead of fixing bugs one by one, we wrote down every promise the product makes (80 of them) and put a machine check behind 20. The honest scoreboard went from 6 green to 15 in a day — and the auditor caught an admin door that let anyone in. One rule made it work: no evidence, no pass.

A build-in-public story about a biography app for elders, a success toast that lied, and the claim ledger that ended the whack-a-mole. ~8 min.

The app kept saying done — it was lying. A scoreboard going from 6 to 15 verified claims, with the lies it told on the left
One day of honest auditing: the promises the app was making vs. the ones it could prove.

The toast that lied

I'm building 新遗产传记 (New Legacy Biography) — an app where an AI interviewer sits with an elder, asks the questions a grandchild never thinks to ask in time, and turns the answers into chapters of a book. It's the kind of product where a bug isn't an inconvenience. If software loses your grandmother's story about the river she grew up beside, it has lost something you cannot re-download.

My co-developer on this project is an AI agent. It ships fast. The unit tests were green — all 135 of them. TypeScript strict mode: clean. Lint: zero errors. And then I, playing the role of an ordinary user, tried to log in with an email link. The screen said: “登录链接已发送！” — your login link has been sent, check your inbox.

No email ever arrived. It couldn't have. There was no email service configured anywhere in production. The code had a “development fallback” that printed the email to a server log nobody reads — and then reported success to the user. Every gate we trusted was green, and the product was lying to the one person who mattered.

It got better. The “invite your family” feature — the heart of the product, where relatives contribute photos and voice memos to a birthday memory book — generated invitation links that began with http://localhost:3000. A link to a computer that exists only on my desk. Sharing it with your aunt in another city does exactly nothing. And the registration page's only path was that same phantom email — meaning, for a while, no new human being on Earth could sign up for the product at all, while every test stayed green.

Why green tests didn't save us

Edsger Dijkstra said the quiet part in his 1972 Turing Award lecture: “program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence” [1]. Fifty-four years later, AI-assisted development has made his warning sharper, not softer — because generative tools are extremely good at producing code that looks finished. The unit tests test the functions. Nobody was testing the promises.

Hamel Husain, who has spent years building ML products, puts the same diagnosis in modern terms: “I’ve found that unsuccessful products almost always share a common root cause: a failure to create robust evaluation systems” [2]. His reasoning is the part worth stealing: without an evaluation system, fixing one failure just surfaces another — his term for it: “a game of whack-a-mole” [2]. That was exactly my week. Fix the email lie, discover the localhost links. Fix the links, discover voice input needed a model file that was never deployed. Every mole a surprise, because nothing was keeping score.

The mental model: a claim ledger

Here's the whole idea, simple enough to re-run tomorrow without this article. Every promise your product makes — every button, every nav item, every line of marketing — is a claim. A claim without a machine check is a rumor. So:

  1. Write the claims down as data. Not in your head, not in a doc — in a file the code can read. Ours has entries like “亲友无需注册可上传照片” (relatives can upload photos without registering) and “新用户可以完成注册” (a new user can actually sign up).
  2. Give each claim one check that exercises it the way a user would. A real browser fills the real form on the production site. A cookie-less request opens the invite link the way your aunt would.
  3. Score honestly with three grades, not two: PASS, FAIL, and NOT MEASURED. The third grade is the load-bearing one — a claim you couldn't check is excluded, never quietly counted as green.
  4. Gate on it. The run exits angry if a gated claim fails. An honest ❌ beats a fake ✅.
The claim ledger loop: promise becomes claim, claim gets a machine check against production, scored PASS / FAIL / NOT MEASURED, failures get fixed, the loop reruns
The claim ledger in one picture. The third grade — NOT MEASURED — is what keeps the other two honest.

What one day of honesty produced

The first full run against production scored 6 PASS, 11 FAIL, 3 NOT MEASURED across 20 machine-checked claims. Not because the app was mostly junk — because for months nobody had asked the app to prove anything. By the end of the day: 15 PASS, and each remaining FAIL had a named, scoped unblock instead of a vague bad feeling.

Three finds worth the price of admission:

Patterns and anti-patterns

Patterns that held up:

Anti-patterns we paid for:

The mechanism, named: generative tools optimize for plausible completion — code that looks like what done looks like. Nothing in that objective requires the email to exist. So the pressure toward fake-done is structural, not moral, and the countermeasure has to be structural too.

The first principle, one sentence: no evidence, no pass.

Steal this on Monday

Timeboxed and specific — one working session:

  1. (30 min) Open your product's landing page and nav. Write down 20 promises it makes, verbatim, in a JSON file.
  2. (2–3 hrs) For each, write the dumbest possible check that exercises it like a user — a headless browser or a curl, against production, not staging. Mark each claim gate: true/false.
  3. (10 min) Run it. Publish the honest table where your team can see it — PASS, FAIL, NOT MEASURED.
  4. Measure success by one number: gated FAILs at zero, and the run wired into CI so it can never quietly rot. You'll know it worked the first time it blocks a release you were sure about.

Ours is in the repo as eval/claims.json + eval/run.mjs — 20 claims, one command, honest output. The pattern is an application of the OEC loop (Observe → Evaluate → Control) I keep as a standing discipline: observability precedes evaluation, and evaluation without a control hook is just a scoreboard.

The part that isn't about software

The app this happened to is live: an AI interviewer that speaks and listens in Mandarin, follows a memory into the kitchen — in one test interview last week, one that smelled of red dates and dried longan — and drafts it into a chapter — with a companion agent that answers “how far along is my book?” from real data. This week it gained something rarer than any feature: a habit of telling the truth about itself. If you have a parent whose stories you keep meaning to record, it's here — and the scoreboard of what it can and cannot yet do is public, which is exactly how I'd want to be sold anything.

References

  1. Dijkstra, E. W. (1972). The Humble Programmer (ACM Turing Award Lecture), EWD340: “program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence.” cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD340.html
  2. Husain, H. (2024). Your AI Product Needs Evals. “I’ve found that unsuccessful products almost always share a common root cause: a failure to create robust evaluation systems”; on symptoms: “Addressing one failure mode led to the emergence of others, resembling a game of whack-a-mole.” hamel.dev/blog/posts/evals

More in the AI-Native series

Part of the AI-Native series. Everything in this article is reproducible from the public eval ledger — including the failures. No evidence, no pass. You own the Publish button.