Most AI projects that stall do not stall on the technology. They stall because nobody decided, in advance, what the system does when the model is wrong. That decision is the whole engineering problem, and it is usually treated as a detail.
The property that matters
A language model is a component that is usually right, cannot tell you when it is not, and is wrong in ways that look exactly like being right. Conventional software fails loudly. This fails plausibly.
So the question to ask of any proposed AI feature is not 'can it do this'. It is: what does a confident wrong answer cost here, and who catches it?
Where it earns its place
Work with three characteristics is where models pay off reliably.
- The input is messy and unstructured. Documents, email, free text, PDFs, transcripts — the cases where writing rules has always been miserable and brittle.
- A human reviews the output anyway. Drafting a reply, summarising a case file, proposing a classification. The model produces a starting point and a person keeps final say.
- Being wrong is cheap and obvious. A weak search result costs a second. A wrong payroll calculation costs considerably more.
Concretely, that tends to mean: search and retrieval over your own documents, extracting structured fields from unstructured input, triage and routing, first-draft generation, and summarising long material for someone who then acts on it.
Where conventional software is simply better
Anything with a correct answer that can be computed. Tax, interest, eligibility rules, discounts, entitlements, reconciliation. If the logic can be written down, write it down. A deterministic function is cheaper, faster, testable, auditable and always right — and it will not change its mind next quarter when the provider updates a model.
Using a model where a rule would do is not innovation. It is paying per token for arithmetic you could have tested.
What production actually requires
The gap between a prototype and a feature is mostly made of these five things.
- 01An evaluation set. Fifty to two hundred real cases with known good answers, so a prompt or model change can be judged on evidence. Without this you are not iterating, you are redecorating.
- 02Prompts and context in version control. They are code. They get reviewed, diffed and rolled back like code.
- 03Defined failure behaviour. What happens on a timeout, a refusal, a malformed response or an answer that fails validation. Every one of these needs an answer before launch.
- 04Guardrails on authority. Be explicit about what the system may do unaided and what requires a human. An agent that can draft an email is a different risk from one that can send it.
- 05Cost and latency budgets. A feature that works beautifully at five requests an hour and becomes unaffordable at five hundred is not finished.
A sensible first project
Pick one workflow where your team currently reads something and decides something. Measure how long it takes today and how often it goes wrong. Build the narrowest possible version, keep the human in the loop, and measure the same two numbers again.
That produces something more valuable than any strategy document: evidence about whether this class of problem, in your business, with your data, is worth pursuing further. Whatever the answer, you will have learned it in weeks rather than after a year of committee work.