The big cloud operators are ramping up investment in AI servers and infrastructure to meet demand for AI development and deployment, exacerbating the memory shortage caused by their insatiable growth.
Taiwan-based market watcher TrendForce estimates the world's eight biggest cloud providers – Google, Amazon, Meta, Microsoft, Oracle, Tencent, Alibaba, and Baidu – will lay out upwards of $710 billion in capex during 2026, about 61 percent more than last year.
According to figures disclosed earlier, the first four alone account for about $635 billion of that outlay, showing just how much the giant players dominate the market.
All of this spend – which adds up to more than the entire gross domestic product (GDP) of Ireland last year – is going on datacenters and the kit to fill them, including high-performance servers typically packed with GPU accelerators from Nvidia or AMD.
However, many increasingly invest in other accelerators such as custom-built application-specific integrated circuits (ASICs). These offer some advantages including better performance and energy efficiency for some workloads, but are less versatile than GPUs.
Google remains the only cloud biz that is adding more ASIC-based servers than GPU-based ones, according to TrendForce. It estimates Google's Tensor Processing Units (TPUs) will feature in about 78 percent of AI servers shipped to Google datacenters this year.
Amazon's build-out is expected to comprise 60 percent GPU servers, with systems based on its Trainium3 silicon set to ramp up later in the year. Meta will likewise rely primarily on Nvidia and AMD GPUs, which are likely to make up more than 80 percent of the servers it assimilates this year.
Microsoft continues to procure Nvidia rack-scale systems, and Oracle is also expanding its rack-scale deployments of GPU servers. Of the Chinese operators, Tencent also continues to roll out servers with Nvidia GPUs.
This demand for AI servers has led to rising memory prices and a shortage as the chipmakers switch manufacturing lines to favor high-margin products such as high-bandwidth memory (HBM) used in GPUs and server-grade memory chips.
Two of those memory chipmakers, SK Hynix and Sandisk, have today announced work on a standardization process for a new memory type aimed at boosting AI inferencing.
High-bandwidth flash (HBF) is a form of NAND flash intended to complement HBM by matching the latter's bandwidth while delivering 8-16 times the capacity for a similar cost.
HBM is commonly used for AI processing, but its capacity limits lead to lengthening inference times as models grow. Because it is flash, HBF is slower to access than HBM, but much faster than a flash solid-state drive (SSD). Combining the two could increase the size of workloads that can be processed without having to fetch data from SSDs.
See an explainer here over at Blocks & Files, while a fact sheet [PDF] is also available from Sandisk.
SK hynix describes HBF technology as a new memory layer between ultra-fast HBM and high-capacity SSDs, and says it will reduce the total cost of ownership (TCO) while increasing the scalability of AI systems. It forecasts that demand for complex memory solutions like HBF will pick up around 2030. ®
Source: The register