On Tuesday, during the Vision 2024 event in Phoenix, Intel unveiled a new AI accelerator named Gaudi 3, positioning it as a competitive option to Nvidia’s H100, a data center GPU known for its robust performance claims when handling large language models like ChatGPT.
In direct comparison to Nvidia’s H100, Intel forecasts a 50 percent faster training time with Gaudi 3 for both OpenAI’s GPT-3 175B LLM and Meta’s Llama 2 with 7 billion features. When it comes to inference tasks, Intel asserts that Gaudi 3 outperforms the H100 by 50 percent for popular models like Llama 2 and Falcon 180B.
Intel’s focus on challenging the H100 stems from its significant market share, despite Nvidia having more advanced AI accelerator chips like the upcoming H200 and Blackwell B200, which have surpassed the H100 on paper but are yet to be released.
While tech companies and AI researchers have faced supply challenges with the H100, Intel’s Gaudi 3 emerges as a promising alternative if priced competitively. Intel highlights Gaudi 3’s design enhancements over its predecessor, Gaudi 2, such as utilizing two parallel silicon dies connected by a high-bandwidth link, boasting 64 cores in total with 48 megabytes of central memory per die.
Gaudi 3 leverages 8-bit floating-point infrastructure for enhanced performance in training transformer models and offers a triple increase in calculations using the BFloat 16 format. With 128GB of HBMe2 memory and 3.7TB memory bandwidth, Intel emphasizes Gaudi 3’s energy efficiency, claiming a 40% improvement in conclusion energy-effectiveness compared to Nvidia’s H100.
In a comparison between Gaudi 3 and Nvidia’s Blackwell architecture, Intel’s use of TSMC’s N5 process technology narrows the fabrication technology gap. Gaudi 3’s use of HBM2e memory underscores Intel’s focus on balancing performance and cost-efficiency, positioning itself as a strong contender in the AI accelerator market.
As the competition intensifies in AI computation, the future release of Intel’s Falcon Shores chip remains an area of interest, raising questions about Intel’s future strategies in semiconductor fabrication and technological advancements to stay competitive in the AI accelerator landscape.
