The real lesson: AI governance must be designed, not declared
KPMG reportedly removed its report, Redefining excellence in the age of agentic AI, after several major organizations disputed claims attributed to them. GPTZero identified inaccuracies that appeared consistent with model hallucinations, and organizations including UBS, the UK NHS, Swiss Federal Railways, and Transport for London reportedly denied or challenged statements made about their AI use.
The easy reaction is to blame the model. That is too convenient.
The more important conclusion is this: the failure was not that AI produced unreliable content. The failure was that an enterprise-grade review process allowed unreliable content to become institutional output.
For executives, this is not a media story about one report. It is a preview of what happens when AI is adopted faster than the organization’s ability to govern it.
Human-in-the-loop is essential in critical processes. But if every AI action requires a human checkpoint, the organization has not automated anything. The objective is not more supervision. The objective is smarter supervision.
Why this case matters beyond KPMG
A Big Four report is not a casual blog post. Many boards, CFOs, CIOs, procurement teams, and strategy units use such reports as decision-support material. When a report contains fabricated or misleading examples, the damage can move downstream into investment cases, vendor selection, internal AI roadmaps, and board presentations.
This is why the incident should concern every enterprise leader. AI errors do not stay inside the document where they were created. They become assumptions. Assumptions become budgets. Budgets become projects.
The financial risk is obvious:
- Wrong benchmarks can distort ROI calculations.
- False market examples can create pressure to copy non-existent practices.
- Misleading claims can weaken board-level confidence in AI initiatives.
- Reputation damage can exceed the cost savings achieved by automation.
AI can dramatically improve operational efficiency, especially in research, drafting, summarization, knowledge work, customer operations, finance workflows, and internal support. But efficiency without control is not transformation. It is exposure.
The problem is not AI writing. The problem is AI publishing without accountability
There is nothing inherently wrong with using AI to assist in the creation of research reports. In fact, refusing to use AI in high-volume knowledge work is becoming difficult to justify.
AI can help teams:
- Map market themes.
- Summarize public filings.
- Compare policy documents.
- Draft first versions of analysis.
- Identify contradictions across sources.
- Generate interview questions.
- Structure complex material for executive audiences.
The issue begins when the organization confuses fluent language with verified knowledge.
Large language models are powerful reasoning and language systems, but they are not primary sources of truth. They can synthesize, infer, classify, and explain. They can also invent plausible details when the process around them fails to constrain, verify, and audit their output.
That distinction is critical. AI is not merely a technical tool. It sits at the intersection of technology, management, domain expertise, organizational process, risk, and human judgment. This is exactly why serious AI adoption requires more than prompt enthusiasm. It requires education, business experience, operational understanding, and disciplined governance.
Human-in-the-loop: yes, but only where it matters
The KPMG case is a perfect example of where human oversight is non-negotiable. A public report, issued by a trusted advisory brand, making factual claims about named organizations, should never be released without independent verification.
But the answer is not to place a human reviewer in front of every AI-generated paragraph across the company. That approach does not scale. It also destroys the productivity gains AI is supposed to create.
The right model is risk-based human-in-the-loop.
A practical risk model for AI-generated work
Organizations should classify AI-assisted processes into clear control levels:
- Low-risk internal productivity: summarizing internal meeting notes, drafting internal emails, formatting documents, brainstorming, personal research support.
- Medium-risk operational support: customer response drafts, internal policy summaries, financial commentary drafts, workflow recommendations, code suggestions.
- High-risk institutional output: published research, regulatory submissions, legal positions, financial disclosures, board materials, client-facing recommendations, HR decisions, medical or safety-related guidance.
- Critical autonomous actions: AI agents that execute transactions, change system configurations, approve payments, update customer records, trigger communications, or make decisions with financial, legal, or reputational impact.
Human review belongs primarily in the third and fourth categories. In the first category, broad literacy and good tooling may be enough. In the second, sampling, audit trails, and escalation rules may be appropriate. In the third and fourth, verification must be designed into the process.
The goal is to turn yesterday’s human operator into tomorrow’s supervisor of hundreds of AI-assisted processes. That is the operational promise of AI: not replacing judgment everywhere, but multiplying expert judgment where it creates the most value.
What should have happened before publication
A report citing real-world enterprise AI deployments should pass through a verification workflow that is boring, explicit, and documented.
At minimum, the process should include:
- Source locking: every factual claim about a named organization must be linked to a verifiable source.
- Claim extraction: the AI-assisted draft should be converted into a list of factual claims for review.
- Independent validation: a person or secondary system must verify claims against primary sources.
- Named-entity review: references to companies, public bodies, executives, products, and statistics require special scrutiny.
- Evidence grading: claims should be marked as primary source, secondary source, analyst interpretation, or unsupported.
- Final accountability: a named human owner must approve release, not merely the tool that produced the draft.
This is not bureaucracy. It is basic professional hygiene.
For technical teams, even a simple claim-checking layer can reduce risk substantially:
For each external factual claim:
1. Identify the named entity.
2. Identify the specific factual assertion.
3. Retrieve primary or trusted sources.
4. Compare the assertion against the source.
5. Flag unsupported, contradicted, or ambiguous claims.
6. Require human approval before publication.The sophistication of the model matters. The design of the process matters more.
The uncomfortable truth about AI expertise
This incident also highlights a wider market problem: too many organizations are receiving AI advice from people whose expertise is mostly opportunistic.
AI implementation is not just a matter of choosing a model or buying a license. It requires understanding how business processes actually work, where judgment is applied, where exceptions occur, how controls operate, how employees behave, and where automation can create measurable value without unacceptable risk.
Relevant education matters. Academic depth matters. Field experience matters. The strongest AI practitioners are often multidisciplinary: they understand the technology, but they also understand management, operations, finance, compliance, and the specific professional domain being transformed.
Small and mid-sized companies are especially vulnerable here. Large enterprises usually have procurement teams, legal review, security teams, and internal experts who can filter weak advice. Smaller organizations often move faster, trust louder voices, and pay for it later.
AI literacy and AI agents must advance together
There are two adoption paths organizations should pursue at the same time.
The first is AI literacy: employees must learn how to communicate with models, challenge outputs, structure context, protect data, and understand model limitations. The ability to work well with AI is becoming a core business skill.
The second is AI agent development: organizations need internal capability to build, deploy, monitor, and manage agents that perform recurring tasks across systems.
These two paths are not identical. AI tools often require employees to change habits. Agents, when designed well, can reduce friction because they operate inside existing workflows. Technically, agents may look more complex. Organizationally, they can sometimes be easier to adopt.
This is why enterprises need a serious platform for AI agent creation and management. Microsoft Copilot Studio is a reasonable option for organizations deeply invested in the Microsoft ecosystem. Tools such as n8n are also entering enterprise environments in ways that would have seemed unlikely a few years ago. Claude, including capabilities such as Claude Code, remains one of the most compelling enterprise AI environments, although security and governance must be handled carefully. Copilot is improving as well, even if Microsoft’s scale can sometimes slow the delivery of more radical innovation.
The broader point is not which tool wins. The point is that organizations need infrastructure, standards, and internal ownership.
In the near future, information systems departments will increasingly function as human resources departments for AI agents: onboarding them, assigning permissions, monitoring performance, retiring weak agents, and managing accountability.
What Israeli executives should take from this
For Israeli companies, the lesson is especially relevant. Many management teams rely on global consulting reports to justify strategic decisions, technology investments, and AI transformation programs. Those reports can be useful, but they should not be treated as unquestionable evidence.
Before using an external AI report as a basis for strategy, leaders should ask:
- Are the claims supported by primary sources?
- Are named organizations accurately represented?
- Is the report distinguishing evidence from interpretation?
- Was AI used in the research or drafting process?
- What validation process was applied before publication?
- Would we be comfortable presenting this evidence to the board, regulator, or audit committee?
The same applies to Israeli AI vendors building corporate content, research, compliance, or knowledge-management tools. Fact-checking, source attribution, audit logs, and human-in-the-loop controls for critical outputs should be part of the product architecture, not a premium feature added later.
The strategic takeaway
KPMG’s pulled report should not make enterprises afraid of AI. It should make them more mature.
AI will continue to replace and enhance non-deterministic processes that previously required human judgment. That is exactly where much of its value lies. But when the output affects public trust, client decisions, financial planning, legal exposure, or institutional credibility, human oversight is not optional.
The winning organizations will not be those that put humans everywhere. They will be those that know precisely where human judgment is worth the most.
That is the difference between using AI and operating AI professionally.
