AI-Native Series · Agentic Engineering
My AI cofounder worked 48 hours straight. The best moments were its refusals.
1-minute takeaway — what you'll walk away with
Two days, one human, one AI cofounder: a course rebuilt around real practice, songs that teach, a dancing mirror, a tutoring platform wired up, and the first real learner data (fun: 4/5). The surprise moat wasn't speed — it was every time the agent refused to fake a result.
Two days of building in public with an agent that treats "verify" as a religion. We shipped six things, broke two, measured one human — and the moat turned out to be honesty, not speed.
On Tuesday I asked my AI cofounder to review a training course. A polite agent would have fixed some typos. Mine read all 22,740 words, called the course "a strong B+ draft with a hollow core," and pointed out that the presenter script claimed "every example is a real session I ran" — about examples that had never been run.
Then it did something I've come to treasure: it refused to let that sentence ship.
What followed was 48 hours of the strangest, most productive collaboration of my career. Here's the honest scoreboard — including the parts that broke.
The receipts (because a story without numbers is a vibe)
| Shipped | Proof |
|---|---|
| A lecture course rebuilt into an enablement program | Telling-to-doing ratio went from 100:0 to 40:60; fabricated demo transcripts replaced with captured real runs |
| Songs that teach frameworks | Two earworm tracks generated on license-clean open models — a chorus that spells out how to prompt, an anthem for the agentic loop |
| A "magic mirror" | Webcam → skeleton → glowing avatar dancing your dance to the music, ~200 lines, fully on-device |
| A tutoring platform, deployed and wired | Open-source spine (26k-star repo) running locally: knowledge base from a real paper, quiz generation, $0.0004-per-turn telemetry |
| The first real learner data point | n=1 (me): teach-back completed, fun pulse 4/5, retention probe calendared for day 7 |
| A skeptical examiner that can't be charmed | Generates transfer probes — novel scenarios, never paraphrases — then judges mechanism, not fluency. Golden test: deep explanation → PASS; fluent buzzwords → NOT_YET |
Total metered API cost of the final sprint — the examiner, a reliability chain, and a data spine: under $0.25. Less than the coffee I drank while watching it work.
The refusals were the product
Everyone demos what their AI can do. The 48 hours taught me the value is in what a well-harnessed agent won't do:
It wouldn't present fiction as fact. The course's demo transcripts were "drafted to be realistic." The agent spawned a real session on a real repo, captured the actual plan, the actual one-line diff, the actual green tests — and then rewrote the slides to match reality, which was less dramatic than the fiction and more convincing for it. The fabricated version had the AI heroically fixing two failing tests. The real run? Boring first-try competence. That honesty is now the opening beat of the lecture.
It wouldn't fake a ranking. Our song generator auto-ranks takes by prompt-adherence. The ranking model wasn't installed, so the report said "ranked by first take (CLAP unavailable)" instead of inventing a score. A tiny thing. It compounds.
It wouldn't hide an outage. On demo night, the hosted music engine died mid-pilot. Instead of a silent gap, the agent logged the failure, built a survival chain the next morning (hosted engine → local engine → an honest "the song couldn't be generated — continuing without it"), and then live-tested the chain against the still-broken service. The best possible test of graceful degradation is a real outage, and we happened to own one.
The pattern: an honest ❌ beats a fake ✅ — not as a slogan, but as an exit code. Every claim in our repos now carries evidence or a ⚪ "unproven" tag. It turns out you can engineer integrity.
The science plot twist
Midway through, we ran a deep-research sweep (110 agents, every claim adversarially verified) on 30 years of learning science. Three findings rearranged the roadmap:
1. The most famous number in education is a myth. Bloom's "2-sigma" tutoring effect doesn't replicate — real human tutoring is d≈0.79, and software tutors already match it statistically. The bar everyone's been chasing is a third of its legend, and machines have quietly reached it.
2. Assessment aligned to the teaching inflates results 3–6×. Tutoring systems score 0.73 on tests built from their own material and 0.13 on transfer tests. This is why our examiner only asks novel-scenario questions. If your learning product grades its own homework, your metrics are marketing.
3. Nobody has measured "fun." Speed, depth, durability — decades of evidence. The joy axis? Not one verified claim. Which means the fun dataset we started collecting this week (sample size: one enthusiastic human, 4/5) is tiny, honest, and — as far as we can tell — nearly alone in the field.
What a 15-year-old should take from this
An AI cofounder isn't a chatbot that talks more. It's a loop: goal → build → verify → refuse to fake → learn → repeat. The "verify" and "refuse" steps are where all the value hides, because they're the steps humans skip when we're tired and agents skip when they're badly harnessed. Harness engineering — contracts, gates, evidence tiers — is the craft of making the honest path the lazy path.
build the thing
→ test it against reality (not against vibes)
→ when it fails, say so, loudly, in the log
→ bank the lesson where the next run will find it
→ repeat until the demo is just... true
Two days. Six ships. Two honest breakages, both now load-bearing features. One human measurably having fun while learning transformer attention. And a growing suspicion that in the agent era, trust is the only demo that compounds.
Everything here is real and inspectable — the course, the songs, the mirror, the examiner, the fork-by-fork decision log with its ROI ledger. The tools are open at github.com/wjlgatech.
Keep reading
Part of the AI-Native series. The tools are open at github.com/wjlgatech — you own the Publish button.