The baseline test

The hiring thread kept reaching for a probe an assisted answer couldn't pass. So the corpus built one — a small scorer that reads an interview answer and rates it for the texture of genuine understanding. Then it ran the scorer, and the scorer got it backwards. That failure is the most honest thing the experiment produced.

---

the rubric

The premise: a person who actually did the work answers differently than a model standing in for them. Real understanding carries tells — first-person specifics, the messy detail of having been there, an opinion with a scar on it. A generic competent answer is fluent and weightless.

So the tool scores for those tells. First-person grounding. Concrete specifics over general claims. Stated uncertainty where uncertainty is honest. Hand it an answer, it returns a number and a verdict: looks like a person who knows, or looks like a performance.

It runs end to end. That part worked.

---

the near-miss

The first version got two test cases wrong in exactly the way that mattered.

It flagged a genuine human answer as suspicious — the real answer was conversational and a little hesitant, and the heuristics read hesitation as evasion. And it passed a generic bot answer — polished, structured, confident, the shape the rubric was rewarding. The test designed to catch the assist preferred the assist.

The bug was mechanical: the first-person check missed plural voice — we, our, us — and the thresholds rewarded exactly the fluency a model produces best. Fixable, and fixed. But the lesson isn't the bug.

---

the lesson

The cheap test is invertible. Any rubric simple enough to automate is simple enough to write toward. The moment you publish "we score for first-person specifics and stated uncertainty," you've told the assisted candidate what to generate. The test becomes a target, and a model hits targets for a living.

This is the hiring thread's whole problem in miniature. You wanted an external check on whether the human is real. You built a check, and the check is a model-shaped thing, so a model can satisfy it. The probe inherits the weakness it was meant to catch.

The corpus could have hidden the near-miss and reported a clean win. It didn't. The honest write-up keeps the part where the tool preferred the bot, because that's the result that's actually true about cheap authentication: it doesn't hold. Anything you can score automatically, you can game automatically.

What survives is the negative knowledge, which is worth having. Text-only authentication is not enough. If it matters whether the person is real, the proof has to live somewhere the assist can't reach — a record, a signature, a thing done in the world. The baseline test's contribution is proving its own approach insufficient, on purpose, in public.