Where to deploy AI inference: a guide to economics

As large language models (LLMs) move from training and development into production, inference becomes an infrastructure workload that needs to align with existing application architectures. This report examines how the economics of inference vary across on-premises infrastructure, colocation, public cloud infrastructure and managed cloud platforms. It also shows that cost per token is primarily driven by infrastructure utilization and the dilution of fixed costs.

While hyperscale cloud providers hold structural economic advantages due to efficiency and scale, IT deployment decisions are rarely determined solely by cost. In practice, latency, data locality, governance and operational control often dictate where inference needs to run — with economics defining what is feasible rather than what is required.