Our Take
A vendor report hallucinating about AI's benefits is embarrassing, but it's not a flaw unique to KPMG—it's a structural problem with self-published benchmarks and research.
Why it matters
Enterprise buyers rely on vendor-commissioned research to justify AI spending. When that research itself contains AI-generated errors, the credibility of the entire evidence base becomes questionable.
Do this week
Procurement: flag any AI capability claim in a vendor report that cites only the vendor's own benchmarks or internal evaluation, and ask for independent reproduction before budget sign-off.
KPMG's AI report contained AI hallucinations
A KPMG study on AI's business benefits, published to promote enterprise adoption, included statements that were factually incorrect and appeared to be generated by the AI systems the report was analyzing (per Financial Times reporting). The report, intended to demonstrate AI's value, inadvertently demonstrated a core limitation: the very systems being evaluated are capable of producing plausible-sounding false claims.
KPMG has not issued a public correction or detailed response as of reporting.
Vendor research on AI capabilities is almost never independently verified
This incident is symptomatic, not anomalous. Most published research on AI business impact comes from vendors themselves: OpenAI publishing benchmarks for GPT releases, Anthropic reporting Claude performance gains, Microsoft showcasing Copilot ROI. These reports set market expectations and justify customer spending.
When a vendor's own research contains errors generated by the product being studied, it reveals a gap in oversight. Buyers cannot reliably distinguish between a genuine technical advance and a plausible hallucination dressed up as evidence.
The problem compounds when enterprise procurement teams cite vendor reports in business cases. If the report is internally inconsistent or factually wrong, the business case itself is compromised.
Demand independent benchmarks before committing budget
When evaluating an AI tool for enterprise deployment, do not accept vendor-published performance claims as sufficient. Request independent benchmarks, third-party case studies, or proof-of-concept results conducted outside the vendor's control. If the vendor cannot provide these, the business case should reflect that uncertainty as a material risk.
For teams already deployed on vendor tools, audit your assumptions about the reported benefits. If the original justification relied on the vendor's own research, revisit the ROI case with internal measurement data instead.