NVIDIA AI Enterprise Adds Support for NVIDIA H200 NVL

NVIDIA AI Enterprise is the cloud-native software platform for the development and deployment of production-grade AI solutions.

The latest release of the NVIDIA AI Enterprise infrastructure software collection adds support for the latest NVIDIA data center GPU, NVIDIA H200 NVL, giving your enterprise new options for powering cutting-edge use cases such as agentic and generative AI with some of the most powerful and efficient hardware ever released.  

The rest of this post goes into more detail about what’s new, including ways in which NVIDIA makes it easier for organizations to speed time to value with these solutions.

NVIDIA AI Enterprise software platform

NVIDIA AI Enterprise has evolved quite a bit from its initial introduction. 

The software platform consists of numerous software components, which can be downloaded and run on your own servers, edge systems, and workstations, as well as in cloud instances from leading global and regional cloud service providers. This software can be divided into two categories:

• AI and Data Science software catalog
• Infrastructure software collection

The AI and Data Science software catalog consists of NVIDIA NIM microservices, as well as the following frameworks:

• NVIDIA NeMo for building and customizing agentic and generative AI
• NVIDIA Riva for speech AI
• NVIDIA Metropolis for vision AI
• …and more

 It also includes tools for building AI and data science workflows:

• NVIDIA Triton Inference Server
• NVIDIA TensorRT and Tensor-LLM
• PyTorch 
• RAPIDS

Each software package is delivered as a container for easy cloud-native deployment.  For more information about how these are published in several branch versions, each designed to address a particular usage scenario, see Enhanced Security and Streamlined Deployment of AI Agents with NVIDIA AI Enterprise.

The infrastructure software collection provides components needed to support AI and data science software on accelerated systems. It includes drivers for GPU, networking, and virtualization, Kubernetes operators, and Base Command Manager Essentials for cluster management. 

NVIDIA also partners with leading companies to ensure seamless integration with their management and orchestration platforms. This latest release is a new version of the infrastructure software, and the primary new feature is support for the H200 NVL GPU.

H200 NVL GPU

During the Supercomputing 2024 conference, NVIDIA announced the availability of the NVIDIA H200 NVL PCIe GPU—the latest addition to the Hopper family.

H200 NVL is ideal for organizations with data centers looking for lower-power, air-cooled, enterprise rack designs with flexible configurations to deliver acceleration for every AI workload, regardless of size.

Enterprises can use H200 NVL to accelerate AI applications, while also improving energy efficiency through reduced power consumption.

With a 1.5x memory increase and 1.2x bandwidth increase over NVIDIA H100 NVL, you can use H200 NVL to fine-tune LLMs within a few hours and deliver up to 1.7x faster inference performance. 

NVIDIA AI Enterprise adds support for H200 NVL in two phases. 

• NVIDIA AI Enterprise infrastructure collection 6.0, available today, is supported for bare-metal applications as well as virtualization using GPU pass-through.  
• NVIDIA AI Enterprise infrastructure collection 6.1, expected later this quarter, adds support for virtualization with vGPU.

NVIDIA AI Enterprise Software reference architecture

Along with this release, we have also created a guide to help you quickly deploy and configure your on-premises servers to run nearly any AI production application.

The NVIDIA AI Enterprise reference architecture details a flexible infrastructure stack for OEMs and partners, tailored for AI inference workloads such as fine-tuning. It ensures consistent software components, adaptable hardware, and up-to-date guidance for each major release. 

This reference architecture supports scalable, secure, high-performance AI deployments, providing a robust foundation whether you build a complete data center or customize node configurations. Enterprise customers can deliver AI solutions faster, focusing on business rather than deployment issues.

Availability

If you purchase a server with H200 NVL, you can start taking advantage of NVIDIA AI Enterprise right away.  All systems sold with this GPU come with a five-year NVIDIA AI Enterprise subscription.  

Otherwise, there are several ways to get started, including free access to NVIDIA-hosted versions of NIM microservices for testing and a 90-day free evaluation license. Both new and existing customers can download NVIDIA AI Enterprise Infrastructure Collection 6.0 today from the NVIDIA NGC Catalog.