Build the evaluation harness before you touch the model
The most common reason AI pilots never ship isn't model quality. It's that nobody can tell whether a change made things better or worse.
Deneen Cole
Principal, Applied AI
We've now run enough AI engagements to notice a pattern in the ones that stall. It is almost never the model. It's that the team has no way to answer a simple question: did that change make it better?
The subjective release gate
The typical pre-production setup looks like this. Someone edits a prompt. Someone else tries twenty examples in a playground. If the outputs feel better, it ships. This works fine for a demo and collapses the moment the system faces real traffic, because twenty examples chosen by the person who wrote the change is not a measurement — it's a vibe with a sample size.
The failure mode is specific and predictable: quality improves on the cases you looked at and silently degrades on the ones you didn't. Nobody notices until a customer does.
What the harness needs
An evaluation harness worth building has three properties. It is versioned, so you can compare today's score against last month's on identical inputs. It includes adversarial cases drawn from real production failures, not just happy-path examples. And it runs in CI, gating merges, because an eval that people have to remember to run is an eval that stops running around week three.
The adversarial set matters more than teams expect. Every production failure should become a permanent test case. This turns your incident history into a ratchet — the same mistake cannot recur silently.
The objection about cost
Building this first feels like a delay. You have a working demo and stakeholders who want it shipped, and instead you're proposing several weeks of infrastructure that produces no visible feature.
The counter-argument is that every subsequent change is otherwise a guess. On one engagement we built the harness first, and it immediately revealed that a prompt change everyone believed was an improvement had degraded quality on a whole category of queries. That single finding paid for the harness. Without it, the regression would have shipped and been discovered by users weeks later, at which point diagnosing it would have cost considerably more.
Where teams get this wrong
The most common mistake is building the harness to measure what's easy rather than what matters. Exact-match scoring against reference answers is simple to implement and frequently measures the wrong thing, since a correct answer phrased differently scores zero.
The second mistake is treating the eval set as fixed. It should grow continuously — every production failure, every edge case a domain expert flags, every category you discover you weren't covering. A static eval set slowly stops representing reality.
Build the thing that tells you whether you're improving. Then improve.