Field Note
Value·15 June 2026·7 min readWhat separates a pilot from a product
The gap between a working demo and something people rely on.
Most GenAI pilots clear the bar of "it worked in the demo." Very few clear the much harder bar of "people rely on it every day." This piece sets out what actually separates the two, and why the gap is rarely a model problem.
The executive problem
Boards are asking the same question in different words: we've run a dozen pilots, so where's the value? The honest answer is that a pilot proves a capability exists. It doesn't prove anyone will use it under real conditions, with real data, when it's wrong 5% of the time.
Why it matters now
The cost of running a pilot has collapsed. A small team can stand up a working GenAI demo in a sprint. That's created a glut of proofs of concept and a shortage of things people actually rely on. Moving the bottleneck from "can we build it" to "will it survive first contact with production."
Why the conventional response stalls
The usual fix is to bolt on more engineering after the pilot succeeds: better infrastructure, more guardrails, a bigger team. That treats production-readiness as a phase instead of a design constraint. By the time the gaps show up, in accuracy under edge cases, in what happens when the model is confidently wrong, in who owns the outcome when it fails. The pilot's architecture is already load-bearing, and retrofitting reliability into it is slower and more expensive than building for it from the start.
Our Point of View
A pilot and a product answer different questions. A pilot asks: can this be done? A product asks: can this be trusted, every day, by someone whose job depends on it? We design for the second question from day one, not because it's more rigorous for its own sake, but because retrofitting trust is the single biggest cause of stalled AI investment we see.
What this looks like in practice
We see the same failure pattern across sectors. A claims assistant nails 300 curated test cases. Then it meets an edge case nobody scripted: a policy exclusion buried in a rider, a customer typing in shorthand. It gives a wrong answer in the same confident tone as a right one. Nobody notices until someone downstream acts on it. The fix isn't a smarter model. It's designing for the moment it's wrong before launch, not after someone gets burned.
“A demo has to work once, in front of you. A product has to work every day, when you're not watching.”
Before you call it a product, check:
- Define the failure mode before the happy path. What does the system do when it's wrong, not just when it's right?
- Attach an owner to the outcome, not just the output, someone accountable for what happens when the model's answer is used.
- Instrument for drift and edge cases before launch, not after the first incident.
- Set the bar at "a colleague would stake their reputation on this," not "it worked in the demo."
Founder, LoweLabs
With more than 25 years’ experience across consulting, technology and industry, he has built and scaled multiple Data & AI practices, led complex transformation programs, and worked with some of Australia and Asia’s largest organisations. An INSEAD Global Executive MBA graduate, he combines commercial leadership, deep technical expertise and a practical understanding of how emerging technology can be turned into measurable business value.
Evaluating agents like you evaluate people
Why AgentOps borrows more from performance management than from software testing.
Field NoteSynthetic data, real judgement
Where synthetic profiles help, and where they quietly mislead.
Point of ViewWritten at the Wrong Altitude
Why your AI strategy keeps expiring, and why the next rewrite will not fix it.
Point of ViewThe Answer Is in the Portfolio
Eight figures on AI pilots, nothing on the P&L. Why "double down, redirect, or stop" is the wrong question.
Point of ViewNobody Should Own AI
The turf war between your CDO/CIO, CTO and CFO isn't a governance failure. It's a category error.
Point of ViewDesigning the AI Op Model
Before an organisation can delegate work to machines, it has to understand what it is delegating.
Next steps
Got a pilot that needs to become a product?
We help teams close the gap between a working demo and something people rely on. Let's talk about what that takes for your use case.