Six AI infrastructure conundrums

Those who have attended a data center conference in 2024 will have noticed a new buoyancy in the collective mood of the industry as investors, designers and operators plan for radical innovation, backed up by a sudden surge in investments.

The reason, of course, is generative AI. The launch of OpenAI’s ChatGPT in late 2022 triggered a wave of investment in systems, data centers, software, services and skills — reported to be hundreds of billions of dollars. To support the high density, mostly Nvidia-based systems used for AI training, data center operators need to deliver power to the rack at five to 15 times the current industry average of 7 kW to 8 kW. This requires investment and scrutiny around current practices in terms of power availability, generation and distribution, as well as cooling requirements.