Korea's SK hynix revealed on Thursday that it had become the first chip manufacturer to mass produce the much-anticipated 36 GB 12-layer HBM3E chip.
The chips are slated to be in the hands of customers by the end of this year.
High-bandwidth memory (HBM) chips utilize a stacked design, where layers of DRAM are interconnected through through-silicon vias (TSVs), allowing for greater memory density in a more compact form factor. The chips are eagerly sought thanks to the recent surge in high-demand areas like AI, GPUs, and supercomputers.
SK hynix's previous max capacity was 24 GB HBM3E, achieved by stacking eight 3 GB DRAM chips vertically, as our sister publication Blocks & Files reported at the time. mass production began just six months ago in March, to the benefit of key customer Nvidia.
Demand for the product was so great that by May, market analyst firm Trendforce had warned that hunger for HBM capacity could cause a shortage of DRAM supply as companies ditched the latter to make the AI enablers on their limited number of production lines.
The layers of the new HBM3E are 40 percent thicker than the last generation. By stacking four extra layers of 3 GB chips, the company claims it has increased capacity by 50 percent while keeping the same thickness.
SK hynix 12-layer HBM3E – click to enlarge
To combat the frailty inherent in the chip's thin layers, SK hynix embarked on treating them with Advanced MR-MUF (Mass Reflow Molded Underfill) – a process where liquid protective materials are injected between stacked chips to protect the circuits and enhance heat dissipation then hardened.
The new chip product, according to SK hynix, has "increased the speed of memory operations to 9.6 Gbps, the highest memory speed available today."
"If 'Llama 3 70B', a Large Language Model (LLM), is driven by a single GPU equipped with four HBM3E products, it can read 70 billion total parameters 35 times within a second," bragged the company.
SK hynix shares rose 9 percent following today's announcement, surpassing the share growth of rivals Samsung Electronics and Micron, according to media reports.
The company's new memory may help South Korea to become one of the world's top three AI nations, a goal set the same day in an announcement by President Yoon Suk Yeol.
The government said it would achieve the goal through public-private collaboration. Yoon also pledged to build a National AI Computing Center and reform regulations to support SK hynix's efforts.
The country has put a "Presidential AI Committee" in charge of coordinating the R&D push – a committee that consists of 30 experts, 10 ministerial level government officials, and two presidential aides. ®
Source: The register