Test the model, not just the code

The biggest thing he learned building his app with me isn’t a trick — it’s where he chose to spend. Nearly half the codebase is test harness (~17k lines). That sounds like over-engineering; it’s the opposite. Once a model is in the loop, “the code is correct” and “the product is correct” become different claims — and only one is the one users feel.

How the realization came

He didn’t start with the conviction; the harness taught it to him, catching things neither of us could see in a diff. Real moments from the build log:

• The end-to-end harness flagged a fabricated problem — for an input that was already complete, the pipeline manufactured flaws that weren’t there.
• A token collision — three distinct inputs that canonicalized to the same short string — produced a confidently false result.
• A cheaper model fabricated facts in a generated document; the fix was pinning that stage up a model tier.
• A generation surface once shipped with no harness at all — “the last ungated surface,” the log calls it — and closing that gap became its own task.

The pattern in every one: the program never crashed, every deterministic test stayed green. The failure was fluent and wrong — the only kind that matters once a model writes part of the output.

Why deterministic tests stop being enough

A normal test pins code: this input maps to that output, forever. But in an agentic pipeline the output is authored by a model, and correctness is a property of code plus prompt plus model version. It can regress with zero code change — a provider ships an update and your behaviour drifts. Nothing in the repo moved, and the product is now wrong. Only a harness that runs the real model through the real pipeline and asserts on its behaviour can catch that.

Many harnesses — each blind to what the others catch

So it isn’t one suite but several, none redundant:

  tier              runs                          catches
  ────              ────                          ───────
  deterministic   pure logic, in CI, free       logic bugs, collisions
                  (seconds, every push)         — the "green in 47s" floor
  LLM-tier        real model, behind a flag     fabrication & bad grounding
                  (spends tokens, opt-in)       in a single stage
  matrix e2e      the WHOLE pipeline across     no-fabrication, one-page,
                  14 scenarios                  completeness, restyle invariants
  conversational  23 chat→tool→graph flows,     jailbreaks, the agent going
                  incl. security probes         off-policy under pressure
  model-compare   each stage × model × effort   the cost/quality knob —
                                                who fabricates when made cheap
  live canary     real model, weekly on a       prompt/model DRIFT over time:
                  schedule                      regressions with no commit

The deterministic tier is the free floor on every push. The LLM tiers cost real tokens, so they’re flagged on. The canary exists for the one failure no commit causes and no one-time test catches — the model under you changing.

What the harness buys

• It lets him trust the AI pair. The harness, in his framing, “catches the AI’s — and my — mistakes before I do” — what makes it safe to let me drive an approved task to completion instead of reviewing every keystroke.
• It makes fabrication a test failure, not a surprise. The end-to-end checks assert that every fact in a generated document traces to real input — so “the model made something up” fails CI like any regression.
• It turns cost into a dial. Running each stage across models and effort cut one flow’s cost ~60% with evidence behind the change — and pinned the one stage that fabricated when cheap up a tier.

The candid part

Half a codebase of harness is a real cost, and not every project earns it — a deterministic app needs no canary. The bet only pays when a model authors something a user will trust. There, the harness isn’t overhead: it’s exactly what lets you move fast, because you’ve made the model’s worst habit — confident fabrication — the cheapest failure to catch.