Intel is piling pressure on Nvidia with the launch of a new AI accelerator that’s faster and cheaper than the H100 – but will it be enough to keep up with the blisteringly fast H200?
At Intel Vision 2024, Intel launched its Gaudi 3 AI accelerator, which the company positions as a direct competitor to Nvidia’s H100, claiming it offers faster training and inference performance on leading GenAI models.
The Gaudi 3 is expected to perform up to 50% better than the H100 on several tasks, including training time, inference throughput and energy efficiency.
Building on the performance and efficiency of the Gaudi 2 AI accelerator, Gaudi 3 reportedly features 4x AI computing for BF16, 1.5x greater memory bandwidth and 2x network bandwidth for massive system expansion, compared to its predecessor.
Superior performance
Manufactured on a 5nm process, Gaudi 3 features 64 AI-customized and programmable TPCs and eight MMEs that can perform 64,000 parallel operations. It offers 128 GB of memory (HBM2e not HBM3E), 3.7 TB of memory bandwidth and 96 MB of on-board SRAM for efficiently processing large data sets. With 24 integrated 200Gb Ethernet ports, it enables flexible system scaling and open-standard networking.
Intel claims that Gaudi 3 is superior to H100 on several models, including a 50% faster training time on Llama 7B and 13B parameters, as well as GPT-3 175B models. Additionally, there is a 50% increase in inference throughput and 40% greater energy efficiency on the Llama 7B and 70B parameters and Falcon 180B models. Intel says Gaudi 3 also outperforms H200 in inferring speed on Llama 7B and 70B parameters, and Falcon 180B parameter models by 30%. Since these are Intel benchmarks, feel free to take them with a grain of salt.
Tom’s hardware notes: “Ultimately, the key to dominating today’s AI training and inference workloads lies in the ability to scale out accelerators to larger clusters. Intel’s Gaudi takes a different approach than Nvidia’s emerging B200 NVL72 systems, using high-speed 200 Gbps Ethernet connections between the Gaudi 3 accelerators and linking the servers with leaf and spine switches to create clusters.”
Justin Hotard, Executive Vice President and General Manager of Intel’s Data Center and AI Group, said: “In the ever-evolving AI market landscape, a significant gap remains in current offerings. Feedback from our customers and the wider market underlines the desire for more choice. Companies are weighing considerations such as availability, scalability, performance, cost and energy efficiency. Intel Gaudi 3 stands out as the GenAI alternative, offering an attractive combination of price-performance, system scalability and time-to-value advantage.”
Gaudi 3 will be available to OEMs in the second quarter of 2024, with general availability expected in the third quarter.