We knew Nvidia’s new GH200 Grace Hopper processor was fast, but the first benchmark test results show exactly how fast it is. GPTshop.ai, which has built an incredibly powerful desktop computer based on the Grace Hopper processor, gave Phoronix access to the chip so it could benchmark it.
NVIDIA’s GH200 is a powerful combination of the 72-core Grace CPU and the H100 Tensor Core GPU. It supports up to 480 GB of LPDDR5 memory and 96 GB of HBM3 or 144 GB of HBM3e memory. The Grace CPU is built on Arm Neoverse-V2 cores, each with 1 MB of L2 cache and a total of 117 MB of L3 cache.
The NVIDIA GH200 runs standard AArch64 Linux distributions. For testing purposes, Phoronix used Ubuntu 23.10 with Linux 6.5, which provides an industry-leading view of the NVIDIA GH200 Linux performance against other Intel Xeon Scalable, AMD EPYC, and Ampere Altra Max processors.
CPU performance
The GPTshop.ai GH200 system was tested with 72 cores, a Quanta S74G motherboard, 480 GB RAM and 960 GB + 1920 GB SAMSUNG SSD drives. All tested server processors ran at the top rated memory frequencies and the maximum number of supported memory channels.
The first benchmarks focused on CPU performance, and GPU benchmarks would follow. Unfortunately, there are no power consumption figures yet, because the NVIDIA GH200 currently does not have any interface under Linux available for reading the power/energy consumption of the GH200. However, the first raw benchmark figures for CPU performance are promising and show the NVIDIA GH200 as a beast in the processor department.
You can view all test results on the Phoronix site here, but an average of all results can be seen in the graph below. Although the EPYC 9754 somehow came out on top, the GH200 processor came out on top in some tests.
In summary, Phoronix says: “On a geo-averaged basis of all benchmarks run, the performance of the GH200 Grace CPU nearly matched that of the Intel Xeon Platinum 8592+ Emerald Rapids processor. The Arm Neoverse-V2-based Grace CPU was generally much faster than the 128-core Ampere Altra Max AArch64 server. It will be interesting to see how AmpereOne can compete, even if no hardware is available for testing yet. (Unfortunately, no AMD MI300A hardware to test at this time either.) NVIDIA ARM CPU performance has certainly come a long way since the NVIDIA Tegra benchmarking early days for ARM performance.
More CPU benchmark numbers are available at this result file. There are also some other benchmarks here of some preliminary tests.