Software could be the easiest fix for hyperscalers' AI power squeeze, researchers say
The industry is spending billions on more efficient chips, cooling, and grid connections. But what about making computers do less work – or doing it at a better time?
Researchers suggest that optimizing software and algorithms could be a more effective solution than hardware upgrades for reducing energy consumption in AI data centers.
What happened
Researchers argue that optimizing software and algorithms can significantly reduce energy consumption in AI data centers, offering a more effective solution than hardware upgrades. Techniques such as using lower-precision formats, training optimizers, and workload shifting are being explored to improve efficiency.
Why it matters
For AI developers and data center engineers, optimizing software can lead to substantial energy savings without requiring costly hardware upgrades. This is crucial as data centers face increasing energy demands and grid constraints.
- Software optimization can significantly reduce energy consumption in AI data centers.
- Lower-precision formats and training optimizers are effective for energy savings.
- Workload shifting and grid interaction can help manage energy demand.
- Data sovereignty and network limitations pose challenges to workload migration.
- Efficiency gains must be balanced with the risk of Jevons' paradox, where increased efficiency leads to higher usage.
The bigger picture
Software optimization can help data centers reduce energy costs, improve efficiency, and better manage energy demand by shifting workloads to regions with lower-carbon electricity. This approach complements hardware improvements and grid infrastructure.
Optimizing software and algorithms offers a promising solution to reduce energy consumption in AI data centers, complementing hardware improvements and grid infrastructure. This approach is essential for addressing the growing energy demands of AI.
Technical details
- FP8 and bfloat16 precision formats for inference
- Perseus training optimizer for reducing unnecessary work
- Nvidia's Blackwell power profiles for fine-tuning GPU settings
- Day-ahead planning and real-time scheduling for workload shifting
- Data sovereignty and network limitations affecting workload migration
FAQ
Can software optimization reduce energy consumption in AI data centers?
Yes, software optimization such as using lower-precision formats and training optimizers can reduce energy consumption by up to 30% without sacrificing performance.
What are some examples of software optimization techniques?
Examples include using FP8 instead of bfloat16 for inference, employing training optimizers like Perseus, and workload shifting to regions with lower-carbon electricity.
How does workload shifting help manage energy demand?
Workload shifting allows tasks to be delayed until local demand falls or routed to regions with spare capacity and lower-carbon electricity, improving energy efficiency.
What challenges exist in moving workloads across borders?
Data sovereignty rules and the physical movement of large datasets across networks pose challenges to workload migration.
What is Jevons' paradox and how does it relate to AI data centers?
Jevons' paradox refers to the risk that increased efficiency leads to higher usage. In AI data centers, efficiency gains may result in more token generation, maintaining energy demand.
This summary and analysis were generated with AI from the original article and may contain errors. Read the original for the full story.
Read the full story
Originally published by Tom's Hardware AI on Oct 8, 2026.
Comments
Sign in to join the discussion.