How a Version Earns Its Number

The mental model: a version is a claim, and claims have a shape

Here is the one idea to keep. Every piece of work makes a claim, and claims come in two shapes:

An instance claim: "I did this thing." A method claim: "anyone can do this thing, including to inputs I have never seen."

Those look adjacent. They are not. An instance can be perfect and still be an instance forever. No amount of editing converts one into the other, because the difference isn't in the writing — it's in what you actually did.

v0.1 said: here are my ten principles, compiled into nine checks that can fail. True, tested, and an instance. I wrote the principles. I wrote the checks. They agreed with each other because the same person wrote both, which is exactly as impressive as marking your own homework.

The mechanism: build a filter that can refuse you

Wanting to publish only important work is not a rule, because "important" has no failing state. So I built the smallest thing that could disagree with me: five criteria, in code, that return a verdict.

Then I did the part most people skip. I put twelve papers nobody disputes inside the test suite — Einstein, Shannon, Turing, Nash, Watson & Crick, Diffie–Hellman, Codd, Kahneman & Tversky, McCulloch & Pitts, AlexNet, Bitcoin, Attention. The rule: if a criterion rejects a classic, the criterion is wrong, not the classic.

It fired immediately. One of my criteria said serious work is short — under 7,000 words. It failed eight of the twelve. Shannon's A Mathematical Theory of Communication is 29,445 words [1]. Turing's is 12,913 [2]. Einstein's is 9,861 [3]. I had derived "short" from a sample of two, and my sample of two happened to be short.

So the criterion died. And the filter, now repaired, turned around and refused my own paper:

NOT YET   omi-v0.1: 4/5
  ✗ substitution-test: `displaces` is empty

Translated: name the work that later authors would cite instead of yours. I couldn't. Not because I hadn't thought about it — because there wasn't any. An instance stands beside its sources forever. Only a method replaces them.

What actually moved the version

The fix was not a paragraph. It was compiling two documents written by people who have never heard of me, and reporting the number that came out.

The first was the OECD AI Principles — the most widely adopted AI values text in the world [4]. For every clause, one question: does this sentence name something a stranger could observe failing? Four of ten do. The other six can't, and the reason is countable rather than arguable. Across the 592 words of principles 1.1–1.5 there are zero occurrences of "must", zero of "shall", and eleven of "should", plus twelve qualifying phrases that quietly move the standard inside the party being bound: "appropriate to the context" (three times), "consistent with the state of the art" (three times), "where technically feasible", "as appropriate."

This is not a criticism of the OECD. Its instrument is a non-binding recommendation to governments, agreed by consensus. Hortatory language is the form working as designed. That is precisely why it is the fair test: if the method produces a usable number on a document drafted with no intention of ever being machine-checked, the method works without its subject's cooperation.

And here is the detail I did not expect. The same obligation appears twice in that document, with different force. The operative clause 1.4(b) says systems should be able to be "overridden, repaired, and/or decommissioned safely as needed." The preamble version of the same idea says "decommissioned safely by human interaction" [4]. The version that names a human — the checkable one — is in the part that isn't operative.

For contrast I compiled a company scaling policy, which scored 4 of 5 [5], because it chose to be checkable: 14 occurrences of "must", 130 of "will", and clauses shaped as if X then we will Y. The honest limit is enforced in my spec as a required field, not a footnote: every threshold in that policy is set and judged by the same company it binds. Bindable is not enforced.

The best thing it found was wrong with me

Compiling someone else's specification surfaced two obligations that are perfectly checkable and that my own framework cannot express: system override, and binding a testing cadence to a release.

Compiling my own manifesto could never have found them. A specification and the checks written alongside it share their blind spots by construction — you cannot discover the questions you didn't think to ask by rereading your own answers.

I did not implement them. A check that passes because nothing declares the field it inspects is a vacuous pass, which is the same sin in a different costume. They are named, owned, dated, and a check now fails if any discovered gap lacks an owner — so it is work, not a note that expires quietly.

Two comedies worth the price of admission

I pointed an independent claim-checking tool at the paper. It returned "NOT READY — 14 over-claims." My stomach dropped. Then I read them: LaTeX table preamble, bibliography entries, and — the crown jewel — the paper's own Limitations section, flagged for over-claiming. An automated reviewer read the passage where I confess my framework demonstrates nothing, and cited me for overconfidence.

Exactly one flag of fourteen was real. The tool's own README had warned me its extractor is aggressive. I had not read it, because it was my tool and I trusted it, which is the identical error as marking your own homework, wearing a different hat.

The second comedy is the submission. I had failed at an academic web form ten times across four techniques. Paul suggested treating it as a command-line problem instead. The check was one command:

arxiv.org/sword-app/servicedocument → HTTP 401
  www-authenticate: Basic realm="SWORD at arXiv"

401, not 404. There has been a documented submission API the whole time [6]. Ten attempts at the wrong interface is not perseverance; it's ten pieces of evidence about the interface that I read as evidence about my effort.

The first principle

A claim that cannot fail cannot improve — so the unit of progress is not the draft, it is the failing test you did not have yesterday.

That is why v0.1 → v0.2 is a real version change and not a revision. v0.1 could not fail in one specific way: nothing in it could ever show that the method generalised beyond its author. v0.2 can. It reports 4 of 10 on a document that would rather not be measured, and if you disagree with a single clause verdict, the number changes — which is the whole point.