NVIDIA H200: The Pinnacle of AI Performance
At SabrePC, our goal is to provide the entire range of high-performance GPU-accelerated solutions to foster your research innovations. Over the past three years, artificial intelligence has become deeply integrated into everyday tasks. Generative AI, in particular, has made these advancements accessible to a wider audience. This surge in AI applications underscores the need for cutting-edge hardware.
We are thrilled to announce that we will soon offer products featuring the new NVIDIA H200 Tensor Core GPU in the HGX form factor. This latest development in NVIDIA's Hopper Architecture for AI accelerators will empower users to push the boundaries of what is possible with AI technology. NVIDIA H200 will be made available in HGX servers in Q4 2024.
NVIDIA H200 vs NVIDIA H100 Changes
NVIDIA H200 is the newest addition to NVIDIA’s high-performance data center GPU lineup that enables the training and deployment of our favorite LLMs and generative AI models. It will replace the NVIDIA H100 as the flagship GPU part of the Hopper architecture. NVIDIA H200 comes with upgraded HBM3e memory running at 4.8TB/s of memory bandwidth, a 43% increase over H100, and expands GPU memory capacity to 141GB, nearly double the H100’s 80GB. Faster and larger HBM memory accelerates the performance of computationally intensive generative AI and HPC applications while meeting the increased demands of growing LLM and AI model sizes.
Since the silicon and the architecture of the GPU die have not changed from H100 and H200, the raw GPU performance remains the same. All the improvements to the H200 are the introduction of the faster and higher capacity 141GB of HBM3e.
| Specification | H100 SXM | H200 SXM |
|---|---|---|
| FP64 teraFLOPS | 34 | 34 |
| FP64 Tensor core teraFLOPS | 67 | 67 |
| FP32 teraFLOPS | 67 | 67 |
| TF32 Tensor core teraFLOPS | 989 | 989 |
| BFLOAT16 Tensor core teraFLOPS | 1,979 | 1,979 |
| FP16 Tensor core teraFLOPS | 1,979 | 1,979 |
| FP8 Tensor core teraFLOPS | 3,958 | 3,958 |
| INT8 Tensor Core TOPS | 3,958 | 3,958 |
| GPU Memory | 80GB HBM3 | 141GB HBM3e |
| GPU Memory Bandwidth | 3.35TB/s | 4.8TB/s |
| Max Thermal Design power (TDP) | UP to 700W (configurable) | up to 700W (configurable) |
| Form Factor | SXM | SXM |
| Interconnect | NVLink: 900GB/s PCIe Gen5: 128GB/s | NVLink: 900GB/s PCIe Gen 5 128GB/s |
| Server Options | NVIDIA HGX H100 | NVIDIA HGX H200 |
| NVIDIA Al Enterprise | Add-on | Add-on |
The H200’s larger and faster memory accelerates generative AI and LLMs while advancing scientific computing for HPC workloads with better energy efficiency. Compared to the H100, the H200 has up to 2x the LLM inference performance when handling models like Llama 2 and GPT-3.
How to Get NVIDIA H200
NVIDIA H200 will be offered in a couple of different form factors/deployments: NVIDIA HGX H200, NVIDIA DGX H200, and NVIDIA H200 NVL.
The NVIDIA HGX H200 is an 8-GPU board that is slotted into custom server configurations. These deployments let the user choose their platform, storage, memory, and more. HGX H200 solutions will start shipping end of 2024.
The NVIDIA DGX H200 will utilize the 8-GPU HGX system board but in an NVIDIA pre-built, configured, and validated solution. It is fully turnkey and only purchasable through NVIDIA partners. DGX H200 will start to ship in August 2024.
The NVIDIA H200 NVL is the PCIe version of the NVIDIA H200 for mainstream servers and deployments. It has all the benefits of the Hopper architecture without the node-to-node or all-GPU interconnect; only pairs of H200s can be NVLinked together.
SabrePC will offer NVIDIA H200 in the HGX form factor and the PCIe form factor. An Eight-Way HGX H200 offers full GPU-to-GPU bandwidth through NVIDIA NVSwitch delivering over 32 petaFLOPS of FP8 deep learning compute and over 1.1TB of aggregate HBM memory.
When to expect NVIDIA H200
Expect the availability of the NVIDIA HGX H200 in Q4 2024. Talk to a SabrePC representative today for updates on solutions featuring NVIDIA H200 (HGX and NVL). Accelerate your computing infrastructure, increase your business’s productivity, and develop AI models with the highest-performing AI accelerator.
Have any questions? Looking for an NVIDIA HGX H200 solution or an alternative to power the most demanding workloads? Contact SabrePC today or explore our various customizable Deep Learning Training server platforms.