Most organisations we work with have a governance document. Some have a governance committee. Very few have governance that actually runs in production – baked into the architecture, not bolted on afterwards.

There’s a difference, and it matters more than most people realise until something goes wrong.

Governance as a document vs governance as a system

When clients come to us with AI governance questions, the conversation usually starts in the wrong place. They want a framework, a policy, a set of principles. Those things have their place. But a governance document doesn’t catch a hallucinated response before it reaches an end user. It doesn’t log what your model did at 2am on a Tuesday. It doesn’t give you a defensible record when a regulator asks how a decision was made.

Real governance is an engineering problem as much as a policy one. It has to be designed into the system from the start – not reviewed in at the end.

What we built for a client in a regulated environment

We recently delivered an AI system for a client operating in a regulated sector where a wrong answer doesn’t just create a bad experience – it carries direct risk to the people using it. That constraint shaped every architectural decision we made.

Hallucination wasn’t something we mitigated with a disclaimer. We minimised it by design – using Knowledge Graph retrieval combined with RAG to ground every response in vetted source documents, with transparent attribution so users could see exactly where an answer came from.

Human oversight wasn’t a sign-off step at the end of a workflow. It was a continuous layer – every AI response reviewable and rateable by certified domain experts, creating a feedback loop that improved the system over time.

Explainability wasn’t optional. Answers surfaced with reasoning and sources visible, so the person using the system could assess confidence rather than just accept an output.

Auditability was full and permanent – every query, response and expert rating logged, giving the client a defensible record for regulatory scrutiny.

The system launched commercially and is in active use. Responsible AI wasn’t a constraint on what we could build. It was what made the product viable in that environment in the first place.

Where most organisations get it wrong

The most common mistake we see is treating governance as a gate – something you pass through before go-live, then move on from. That works fine when the stakes are low. It doesn’t work when the AI is making or influencing decisions that carry real consequences.

The second mistake is separating governance from the people doing the build. When governance lives in a policy team and engineering lives somewhere else, the two rarely meet in the right way. The people who understand the risk surface need to be in the same room as the people writing the architecture.

The third – and most costly – is retrofitting. Auditability, explainability, human-in-the-loop design: these are hard to add after the fact. They change the architecture. If they’re not in scope from day one, you’ll either skip them or rebuild.

What “production-ready” actually means for AI governance

Production-ready AI governance means you can answer these questions at any point: what models are running, who owns them, what data they’re touching, how outputs are monitored, and what happens when behaviour changes. If those answers live in a document rather than a system, you’re not there yet.

Getting there isn’t as complicated as it sounds – but it does require treating governance as a first-class engineering requirement from the moment you start the design.

If you’re working through what that looks like for your environment, we’re happy to compare notes.