Human oversight: the AI advantage most organisations haven't claimed yet

The gap between organisations getting real value from AI and those still finding their way is widening. A recent AI performance study conducted by PwC found that the most AI-fit organisations achieve significantly greater performance from AI than their peers—and that the difference comes down to foundations, not just technology. Governance is one of the key factors that separate leaders from the rest. But governance is broad. This research, conducted by CIO.com in collaboration with PwC UK, focuses on one of its most critical and least resolved dimensions: where, when, and how humans should be involved in AI-supported decisions.

Most IT leaders agree that AI oversight matters. But as organisations rightly try to keep humans in the loop, it’s emerging that they often struggle to find the right place to put them. Human oversight is frequently applied to routine, low-risk decisions – delivering friction without benefit. Meanwhile, decisions that genuinely require judgement or accountability often pass through with limited or inconsistent scrutiny.

At a time when many businesses are reporting a value gap in scaling their AI pilots, solving this dilemma offers competitive and commercial advantage.

Research conducted by CIO.com for PwC shows that while 93% of decision-makers say AI oversight is important or critical, just 7% have it consistently embedded across the organisation. However, this statistic presents a significant opportunity for those that can figure this out before their competitors and capture AI benefits earlier. In fact, PwC's AI performance study found that the most AI-fit organisations now capture up to over seven times the value of others and show significantly better performance than their peers – with governance one of the nine foundational factors that separates leaders from laggards.

In practice, the organisations making progress are starting with what can be seen and measured. During our research, a Head of Data Analytics and AI in the insurance sector noted, “The initiatives getting the most executive attention right now are the ones that directly help our people be more productive. They’re simple to understand, relatively low risk, and the benefits are tangible.”

As AI takes on increasingly complex tasks, the risks and implications of getting the human-machine balance wrong will only grow. But for organisations that get it right, the benefits are significant.

Where and how organisations apply human oversight isn't a standalone decision. It’s shaped by where they are on their AI journey and where they're directing investment. Organisations focused on revenue-generating AI in the front office face different oversight challenges to those embedding AI into operations or using it to reduce back-office costs. As AI evolves from assistive tools to more autonomous agentic systems, the approach to human oversight needs to evolve with it. Organisations need a framework that can adapt over time, supports clear accountability and auditability, and recognises that generative AI is inherently less predictable than traditional systems.

What leaders get right

Those seeing the greatest value from ‘human-in-the-loop' aren't trying to oversee everything. They’re clear on the non-deterministic nature of generative AI and where human judgement is required versus data-driven automation AI. They’re then using these factors to make deliberate, considered interventions at the points of greatest impact, while stepping back from the routine, low-risk decisions where human review adds cost but not value.

The research reflects this. Human judgement is most concentrated around validating outputs, investigating anomalies, and monitoring risk or compliance issues, each cited by 45-48% of organisations. People are stepping in selectively, at the moments where judgement matters most, and letting the technology handle the rest. There is a clear payoff: 53% of leaders cite accuracy as the outcome most improved by human-in-the-loop, followed by customer outcomes (19%) and operational resilience (11%).

That balance is also critical for building trust. According to a CIO in the healthcare sector, “Strong governance helps build trust with clinicians, leadership, and the public, and reduces the risk of unintended consequences. In the long run, that discipline supports more sustainable adoption rather than hindering it.”

Accountability, confidence and governance: the three value gaps to close quickly

Three persistent gaps stand between most organisations and the full value of human oversight. Fortunately, each constraint is solvable, and can quickly be turned into opportunities:

• Accountability. In too many organisations, the person overseeing the AI technology is the same person responsible for ensuring its outcomes are correct. In the worse cases, there is no clear owner for the business consequence. This is ineffective as a check and balance. The research reveals this structural challenge: in 63% of organisations, accountability for AI-supported decisions sits with the technology function. But the decisions and consequences often sit with the business, who may, or many not, have sufficient controls in place to support this requirement. That disconnect between who owns the technology and who owns the outcome is where oversight breaks down. Organisations getting this right are building cross-functional structures around AI to build effective oversight at scale. Whether through a dedicated centre of excellence, a federated model, or a hybrid approach, the operating model determines whether human oversight is a principle or a practice. As one Chief Information Security Officer in geosciences explains: “Decisions around AI use are made collectively, with each function bringing its perspective to the table. That structure helps ensure that no single viewpoint dominates and that risks, compliance, and business value are all considered together.”
• Confidence. Even where accountability is well defined, organisations face a challenge around whether their workforce is ready to act on it. While 92% of IT leaders express high levels of confidence in their ability to override AI outputs, only 23% are completely confident in their workforce’s understanding of AI’s limitations. There's a meaningful difference between believing your team can intervene and knowing they'll recognise when they should. “Humans must always be able to step in, challenge outputs, and override AI-supported decisions when something doesn't look right. The key is making sure those mechanisms are not just documented, but understood by the people using the systems day to day,” says an insurance CIO. It's a challenge reflected more broadly across UK business. PwC's global AI performance study found that 83% of leaders acknowledge a structural skills divide between domain experts and AI technologists, yet only a quarter feel well prepared to bridge it. The gap isn't about talent, but about how organisations connect business knowledge with technical capability.
• Governance. Many organisations struggle to operationalise governance at the pace AI adoption demands, with many organisations reviewing human-in-the-loop controls quarterly or less. Meanwhile, AI capabilities, data sources, and use cases shift continuously. Governance that moves slower than the technology it governs is a problem—but the answer isn’t more bureaucracy. In fact, research suggests UK businesses may have a hidden edge here: the regulated, risk-aware nature of UK business means governance is an area where UK has strong foundations to build from. The opportunity is to turn that instinct for control into a genuine driver of competitive advantage, not just a compliance exercise. The same insurance CIO frames it well: “The challenge only arises when governance becomes a blanket ‘no.’ My approach is always: give me the rules, let's get them approved, and then we'll operate safely within them.”

What the best organisations are doing differently

The organisations pulling ahead aren't necessarily the most advanced in their AI adoption. What sets them apart is how they've designed the role of people around it:

1. They are selective about where humans add value. Rather than defaulting to blanket oversight, leading organisations make deliberate choices about where human intervention is necessary.
2. They are explicit about accountability. Leaders remove ambiguity around who owns AI-supported decisions. Instead of shared or implicit responsibility, they define clear ownership at each critical point in the workflow.
3. They have coherence in their AI deployment strategy. They are moving beyond individual use-cases to take a more holistic approach that has coherence across end-to-end measurable processes.
4. They review controls at the pace of change. As AI systems evolve, so too do the controls around them. Leading organisations adopt dynamic approaches, regularly reassessing where oversight is needed and adjusting controls as models, data, and use cases change.
5. They build judgement as well as technical understanding. Leaders invest in developing both AI literacy and decision-making capability across the workforce. Employees understand how models work and when to trust outputs, when to question them, and when to intervene. It’s a trend that reflects a broader shift in CEO priorities, with the UK CEO Survey also revealing that 61% of CEOs now cite reskilling and upskilling as a key response to transformation — up from 51% two years ago — with technology skills a common focus. This research supports those findings, suggesting a clear link between how organisations invest in people and how effectively they operationalise oversight. Those embedding AI coaches (68%) and pairing technologists with domain experts (64%) are building the cross-functional capability that makes this work in practice.

Most organisations aren't there yet, which is precisely why the opportunity exists now. The next phase of AI advantage won’t come down to better models so much as better judgement.

As one retail CTO reflects: “In many cases, the limitation isn't the technology itself, but the human element — changing workflows, building trust in the outputs, and making sure there's still appropriate oversight rather than full automation.”