The question that comes before the model
Most conversations about AI start in the wrong place. Someone asks which model to use, whether to fine-tune or use retrieval, which vendor to sign with. Those are real questions, but they are the second set of questions.
The first question is simpler and much harder: what decision or task are we trying to improve, and how will we know if it improved? If you cannot answer that in a sentence, no model choice will save the project.
A four-part test
Before we recommend building anything with AI, we work through four checks with the client. If a use case fails any of them, we say so.
- Is there a measurable problem? Not "we should use AI" but "support answers take four hours and customers churn while waiting".
- Does AI solve it better than ordinary software? A well-placed database index or a rewritten form often beats a model.
- Can it be implemented safely? Consider what happens when the system is confidently wrong, and who is accountable when it is.
- Can the result be measured? If success is unfalsifiable, the project will never be finished, only abandoned.
Where it tends to work
The strongest cases we see share a shape: high-volume, repetitive, language-heavy work where being approximately right quickly is worth more than being exactly right slowly.
Reading and classifying documents. Answering questions that already have answers buried in a knowledge base. Summarising long threads for someone who has to act on them. Routing requests to the right team. Drafting a first version a human then edits.
Notice that each of these keeps a person in the loop at the point where being wrong is expensive.
Where it tends not to
Anything where a wrong answer is costly and undetectable. Anything where you need the same answer every time for compliance reasons. Anything where the underlying data is thin, stale, or contradictory, because a model trained or retrieving on bad data produces confident bad output faster than a human could.
And anything added mainly so the product can be described as AI-powered. Users notice.
From prototype to production
A demo proves an idea is possible. Production requires it to be reliable, secure, consistent and affordable at scale. That gap is where most of the engineering effort actually goes: evaluation harnesses, retrieval quality, guardrails, monitoring, retraining, fallbacks for when the model is unavailable.
Budget for that gap at the start, not after the demo impresses everyone.