What 'production ready AI' actually means

"Production-ready AI" is one of those phrases that gets used very loosely by people selling AI products and very strictly by the engineers who have to keep those products running at 3am. This piece is the engineering version — the bar a system has to clear before we will call it production-ready and hand it over.

The gap between a working demo and a production-ready system is, in practice, the entire engagement. Almost every failed AI pilot we have walked into had a working demo at some point. The work that turns the demo into something the business depends on is what makes the difference, and it is the work most people underestimate.

The seven dimensions of production-ready AI

A system is production-ready when it clears the bar on all seven of these. Not five or six. Seven.

1. Evaluation

The system has a documented evaluation set built from real (not curated) data, with known correct outputs. The eval runs automatically when anything in the system changes — a model version, a prompt, a retrieval-corpus update. Regressions show up in the regression dashboard before they show up in production. If the evaluation drops below a defined threshold, the change does not ship.

If a system does not have an eval set, it is not production-ready. Full stop.

2. Observability

You can see what the system is doing. At minimum, you have dashboards for: token cost per request, end-to-end latency, error rate, retrieval recall at k (for RAG systems), hallucination rate (for generative systems), refusal rate, throughput, and per-user usage. You alert on anomalies in any of these.

The unglamorous half of AI engineering. The half that decides whether the system is still working in six months.

3. Cost control

You know what the system costs per request, per user, per day, per month. You have a budget. You have alerts when the cost trajectory is going to overshoot the budget. You have the ability to throttle, batch or fall back to cheaper models when cost spikes. Production AI without cost control is the fastest way to turn a successful pilot into a CFO conversation.

4. Refusal and escalation

The system has explicit refusal behaviour for questions outside its scope, content it cannot verify, or cases where its confidence is below threshold. Refusals route to a defined next step — usually a human reviewer with the context the system gathered. The refusal logic is tested. You can produce an audit trail of refusals.

The opposite of refusal is confident hallucination, which is how most early AI deployments embarrass their owners.

5. Reproducibility and audit

You can reconstruct any output the system produced. The model version, the prompt, the retrieved documents, the timestamp, the user, the result, the downstream action — all logged with retention. If a regulator, an auditor or a court ever asks "how did the system produce this output?", you can answer end to end.

For regulated industries this is non-negotiable. For non-regulated work it is still the right default — it costs little and gives you a way out if something goes wrong.

6. Integration and lifecycle

The system is wired into your real production systems via stable interfaces — your TMS, CRM, EHR, case-management system, support tooling, whatever the operations actually run on. It has a tested deploy and rollback process. There is a documented runbook for the on-call engineer when something breaks. Model updates can be applied without re-engineering the integration.

7. Ownership and human in the loop

There is a named person inside the client organisation who owns the system. They have the access, the documentation, the training and the authority to make decisions about it. The human-in-the-loop boundary is documented — what the AI decides, what the human reviews, where escalations go. The owner treats the system like a product, not a procurement.

If no one inside the client owns the system, it is not production-ready — it is a dependency on the consultancy that built it.

What we do not include in production-ready

A handful of things some consultancies sell as "production-ready" features that are, in our view, not actually part of the bar:

• An "AI platform" with a UI. Most systems do not need one. They are services other systems consume.
• Continuous fine-tuning pipelines. Rarely useful. Almost always over-engineered.
• Custom evaluation harnesses. The eval set matters; the harness can usually be an off-the-shelf framework with a small wrapper.
• Multi-model routing layers. Sometimes useful at high volume; usually a premature optimisation.

How long does production-ready take?

For most of the AI engagements we ship for Liverpool clients, the production-readiness work — observability, eval automation, cost dashboards, integration, runbooks — takes about half of the engagement timeline. If a vendor is telling you the production hardening takes two weeks on a six-week build, they are either skipping something or planning to skip it.

Our typical six-week ship looks roughly like:

• Weeks 1–2: scope, eval set, proof of concept.
• Weeks 3–4: production build, integration.
• Weeks 5–6: observability, cost control, refusal logic, runbooks, handover.

If your timeline does not have weeks 5–6, your system is not going to be production-ready. It is going to be a prototype with the words "production" written on the box.

The diagnostic question

If you want to test whether an AI system someone has built for you is genuinely production-ready, ask to see the regression dashboard, the cost dashboard, and the runbook. If those three artifacts exist, the system is probably production-ready. If they do not, it is not — regardless of what was promised.

If you would like an honest view on whether a system you have (or are building) clears the bar, book a 30-minute discovery call.

Related reading

• The state of AI in Liverpool, 2026 — the city's shipping-vs-not-shipping landscape.
• AI projects we ship most often for Liverpool businesses — production-AI projects priced in plain GBP.
• Why most AI pilots fail — the gap between production-ready and demo-ready.