Question 1

Do enterprises actually use LLM benchmarks to choose a model?

Accepted Answer

Rarely as the deciding factor. Among the 93 leaders who described where benchmarks sat in their last model decision, 44% called them background context or ignored them and 41% said real-workload testing was the decisive evidence. Asked how much benchmark discourse helps, 55% of 92 said it has limited or background value and 42% prefer pilots and direct testing.

Question 2

How do enterprises evaluate LLMs before buying?

Accepted Answer

By testing them on their own workloads. 61% of 100 leaders are involved in hands-on model evaluation and selection. The criteria weighed most were task capability, output quality and accuracy (42% of 96), then security, privacy and enterprise integration (29%) and cost, speed and hands-on workflow testing (29%).

Question 3

Is cost per token the right way to measure LLM spend?

Accepted Answer

Buyers say no, but few have replaced it. 60% of 98 leaders compare models on downstream ROI and labor savings rather than price per million tokens, and only 23% track aggregate spend and token efficiency. Yet 53% of 96 have no formal cost-per-outcome metric; 29% measure per case, ticket, document or task and 18% count time saved.

Question 4

What blocks enterprises from adopting a new LLM?

Accepted Answer

Security and data exposure. 59% of the 75 leaders who named a near-dealbreaker cited security, privacy or data-exposure risk, ahead of failure on real-world task quality (28%) and unfavorable pricing (13%). Security, privacy, legal and compliance teams hold the veto for 44% of 77.

Question 5

Who approves LLM purchases in large companies?

Accepted Answer

Usually a shared group, with security or governance teams holding the veto. Decision rights are shared for 45% of 100 leaders, and the veto sits with security, privacy, legal or compliance for 44% of 77 and with a cross-functional governance group for 43%.

Question 6

Do LLM updates break production systems?

Accepted Answer

Sometimes, and quietly. 40% of 97 leaders said their last bad surprise in production was an output regression, hallucination or silent model change. Asked directly about provider updates, 21% of 78 said silent updates or version changes caused regressions, and 8% have testing, monitoring, rollback or fallback protections in place.

Question 7

Are DeepSeek and other non-US LLMs replacing US models in enterprises?

Accepted Answer

No, not in this sample. 75% of 101 leaders said non-US models such as DeepSeek, Kimi and Qwen are excluded or have never been tested, and 93% have limited or no comparative production evidence. Only 27 of the 102 interviewed have run one on a real workload.

Question 8

What percentage of companies use more than one LLM provider?

Accepted Answer

63% of 101 enterprises interviewed run multi-provider environments spanning several frontier labs. A further 20% are evaluating a second provider alongside their current one, and 17% run OpenAI or Microsoft Copilot alone. Routing is manual for 80% of the 84 who described it.

Question 9

Why do companies switch LLM providers?

Accepted Answer

Performance first, cost second. Among the 84 leaders who described their last model change, 57% said it was driven by performance or capability, 27% by an IT, engineering, business or executive decision, and 15% by cost, pricing or redundant spend. ---

Make Your Category the Next G2 Report.

State of AEO 2026: AI Outpaces Measurement

The Connected Mandate 2026: UK AI Readiness Gap

70% of Product Roadmaps Are Shifting Toward AI. Monetization Is Still Unresolved.

Make Your Category the Next G2 Report.

Keep Reading

State of AEO 2026: AI Outpaces Measurement

The Connected Mandate 2026: UK AI Readiness Gap

70% of Product Roadmaps Are Shifting Toward AI. Monetization Is Still Unresolved.