A retailer changed its returns policy on a Monday. The customer service assistant, an AI tool that had been live for six months and was praised in every steering committee, kept telling customers they had 60 days to return an item instead of the new 30. It took three weeks and a pile of complaints before anyone noticed. When the CIO asked who was responsible for AI model monitoring on the assistant, the honest answer was that nobody was. The project team had moved on, the vendor monitored uptime, and the service desk assumed IT was watching the answers.
This is the part of AI that rarely gets a budget line. Organisations spend months choosing and testing a use case, then treat go-live as the finish. In reality it is the moment the risk changes shape. Below we explain why AI model monitoring after go-live deserves its own owner, use the MetroMinds Library to show which kinds of AI are hardest to watch, and end with a monitoring routine you can start next month.
Go-live is where the risk changes shape
Before go-live, the main question is whether the AI works. After go-live, the question becomes whether it keeps working while the world around it changes. Policies are updated, products are renamed, customer behaviour shifts with the seasons and the data feeding the model slowly drifts away from the data it was trained or configured on. None of this shows up in an uptime report.
Many organisations know they are not ready for this. In a Gartner survey of 360 IT leaders, cited in a press release from April 2026, "only 23% said they are very confident" in their ability to manage security and governance for generative AI tools [3]. For high-risk systems, European law now makes ongoing oversight explicit. Under the EU AI Act, " shall monitor the operation of the high-risk AI system on the basis of the instructions for use" [1]. Even for AI that falls outside the high-risk categories, that principle is a sensible standard to work towards.
Some AI is much harder to watch than others
Every use case in the MetroMinds Library is scored on , and one of the dimensions is : how easily you can see what the AI is doing once it runs in production, measure its quality and spot when it goes wrong. Across the Library, 12 percent of use cases score low on this dimension, with a 1 or 2 out of 5. That average hides a sharp divide.
In content creation, 61 percent of use cases are hard to observe. In conversational AI it is 35 percent, and in knowledge intelligence 29 percent. In prediction, optimisation and agentic AI, the share is zero. The reason is simple once you see it. A forecast or an optimised schedule produces a number you can compare with reality the next day. A generated letter, a chatbot answer or a summary of a policy has no single correct value, so checking it takes judgement, sampling and time.
That has an uncomfortable consequence. The kinds of AI that spread fastest through organisations, the assistants and writing tools that teams adopt on their own, are exactly the ones that are hardest to monitor. If you have read our article on shadow AI, you will recognise the pattern.
The gap differs by industry
When you split the same score by industry, the differences become sharper. Content creation is hard to observe in most sectors, with shares of 70 to 90 percent in transport and logistics, retail, manufacturing, government, financial services, healthcare, energy, media and communications. Conversational AI sits around 45 to 50 percent in several of the same industries.
For a CIO, this works as a planning tool. If your sector sits in a dark cell, every new generative or conversational use case should arrive with a monitoring plan and a budget for human review. If that plan is missing, the business case is incomplete, because part of the cost of running the AI safely has been left out.
What good AI model monitoring covers
Monitoring AI is broader than watching a dashboard. The NIST AI Risk Management Framework describes it as a set of plans that start at deployment: "Post-deployment AI system monitoring plans are implemented, including mechanisms for capturing and evaluating input" from users, followed by arrangements for appeal and override, incident response, recovery and decommissioning [2]. In plain terms, you need to know what goes in, what comes out, when people overrule the AI, and what you will do when it fails.
The MetroMinds Library treats this as a domain of its own. Its Trust, Governance and Engineering domain holds 480 use cases, spread over 13 industries and eight kinds of work: runtime monitoring, drift detection, model evaluation, guardrail enforcement, audit trails, explainability, compliance controls and prompt and agent lifecycle management. Each has 60 use cases. That tells you something about scale. Governing AI after go-live is real work for real roles, and it deserves to be planned and funded like any other use case.
Who should own AI after go-live
The most common gap is ownership. IT tends to own the platform, the vendor owns the model and the business owns the process, which means everyone assumes someone else is watching the output. A workable split gives the business owner responsibility for the quality of the answers and decisions, IT responsibility for the technical signals and tooling, and a portfolio manager responsibility for checking that every live use case has both.
A few questions help you test your own position:
- For each AI use case in production, can you name the person who checks the quality of its output, not only its availability?
- Which business events, such as a policy, price or product change, automatically trigger a review?
- Do you record when people overrule the AI, and does anyone look at those overrides?
- Was the cost of monitoring and human review included in the business case before approval?
- Do you know which of your live use cases are generative or conversational, and therefore hardest to observe?
How MetroMinds helps you plan for life after go-live
MetroMinds sits before your monitoring tools: it helps you make sure monitoring is planned before a use case is approved. Every use case in the Library has a Technology Readiness profile that includes runtime observability, so you can see early which ideas will be hard to watch. In your portfolio, you can compare what your use cases demand with how ready your organisation is, and add monitoring use cases from the Library, such as drift detection or audit trails, alongside the use cases they protect.
That way the conversation about who checks the AI happens when the use case moves from accepted to , rather than three weeks after something went wrong.
How ready is your organisation to govern AI once it is live? Take the AI Readiness Snapshot
Sources
- European Union, Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 26(5), Official Journal of the European Union, 12 July 2024.
- National Institute of Standards and Technology (NIST), "Artificial Intelligence Risk Management Framework (AI RMF 1.0)", MANAGE 4.1, January 2023.
- Gartner, "Gartner Says Organizations with Successful AI Initiatives Invest Up to Four Times More in Data and Analytics Foundations", press release, 16 April 2026.
- MetroMinds Library, 4,980 AI use cases with benchmark ranges based on real-world implementations, August 2026.
