Cerebras CS-3: Revolutionizing AI with Wafer Scale Processors

Introduction to Cerebras Wafer Scale Engine

The Cerebras Wafer Scale Engine (WSE) stands as an exceptionally impressive AI chip solution and remains one of the few from a startup that has achieved commercial success. In March, Cerebras announced the third iteration of its chip, known as the Cerebras WSE-3. This latest version leverages further process miniaturization to significantly increase density compared to its predecessors.

Specifications of the Cerebras WSE-3

The Cerebras WSE-3 chip boasts some astounding specs. It features 4 trillion transistors, 900,000 AI-optimized cores, 125 petaflops of peak AI performance, and 44GB of on-chip SRAM, all fabricated using TSMC’s 5nm process. Although the company has not publicly disclosed the chip prices, multiple sources estimate them to be in the range of $2–3 million.

Comparison with NVIDIA’s H100

In direct comparison, the WSE-3's die measures 46,225mm², which is 57 times larger than NVIDIA’s current H100 AI GPU, which is 826mm². Additionally, the WSE-3 supports external memory options of 1.5TB, 12TB, or an astounding 1.2PB, enabling it to train AI models with up to 24 trillion parameters.

Advancements in Core and Memory Specifications

NVIDIA’s current H100 AI GPU has 16,896 cores and 528 Tensor Cores. In contrast, the new Cerebras WSE-3 boasts an incredible 900,000 AI-optimized cores per chip, 52 times more than the H100. This remarkable advancement is further underscored by its 21 petabytes per second of memory bandwidth, which is 7000 times more than the H100, and 214 petabits per second of fabric bandwidth, making it 3715 times faster. The chip includes 44GB of on-chip memory, which is 880 times more than NVIDIA’s H100 AI GPU.

Expert Opinions on Wafer-Based Chips

Anshuman Tripathi from NSAB believes that wafer-based chips will soon surpass Nvidia’s non-wafer GPUs, marking a significant shift in the industry.

Future Prospects for Wafer-Scale Chips

TSMC aims to introduce more powerful Wafer-Scale chips by 2027, which could herald a new wave of Wafer-Scale computers, pushing the boundaries of AI and computational performance.

Final Thoughts

Cerebras' WSE-3 represents a monumental leap in AI processing power and efficiency. With its unmatched specifications and performance, it sets a new benchmark for the industry, paving the way for future innovations in wafer-scale computing.