NVIDIA BlueField Bringing Performance and Cost Optimization to the JBOF Architecture

The development of AI has driven exponential growth in computing power, with network speeds doubling every few years. Less well-known is that it also imposes new demands on storage.

Training new models typically requires high-bandwidth network access to petabyte-scale data, while inference using novel retrieval-augmented generation techniques (RAG) requires low-latency access to hundreds of terabytes of storage. New models can also train, index, and retrieve rich image and video data. Many new AI inference indexing and search tools rely on vector databases and must retain large amounts of metadata about all embedded content.

Traditional file storage is often not suitable for these new workloads, and traditional storage hardware is not the most efficient or performance-oriented way to support AI. Usually, object storage is the best architecture for handling large volumes of data. New storage solutions are needed that provide better performance and efficiency relative to price and power consumption.

DPU-based JBOF Architecture

Supermicro has long provided numerous JBOF (Just a Bunch of Flash) solutions, deploying them as direct-attached storage or network file or object storage. Now, they are introducing a new JBOF supported by the NVIDIA BlueField Data Processing Unit (DPU).

The new JBOF no longer uses traditional storage server designs with separate CPUs, memory, and NICs but instead leverages the DPU to run software, connect to the network, support remote management, and accelerate critical networking, storage, and security functions within a custom chip.

The DPU card can replace CPUs, NICs, DRAM modules, CPU PCIe switches, encryption accelerators, BMCs, and remote management ports. Connecting SSDs might still require an external PCIe switch.

BlueField is NVIDIA’s market-leading DPU, optimized to offload and accelerate networking, storage, security, and management functions. Since the DPU integrates networking, CPU, memory controllers, PCIe switches, and traffic accelerators into a single chip, data can move directly between SSDs and high-speed network ports without being processed by an external CPU or crossing separate PCIe buses multiple times, enabling low-latency storage access and making the system more efficient in terms of price, performance, and energy efficiency.

The BlueField DPU storage controller card supports network traffic up to 400 Gb/s and can accelerate NVMe over Fabrics (NVMe-oF) storage protocols and other RDMA-based storage traffic. It also acts as a PCIe root complex to manage SSDs and uses its Arm cores to run storage software, allowing the JBOF to be presented as block, file, or object storage and deployed as part of an extended storage solution.

BlueField also provides security offload and remote management capabilities, including its own Baseboard Management Controller (BMC) and a separate management port, which are typically required for large-scale data center deployments and cloud operations.

The New Supermicro JBOF

Supermicro’s new 2RU JBOF features a flexible design, supporting either 36 E3.S SSDs or 24 U.2 SSDs, with raw capacity up to 1.44 PB, and with the newer 60-TB SSDs, it can support up to 2 PB. It comes with two controller canisters, each capable of supporting up to two BlueField-3 DPUs and one NVIDIA GPU.

The JBOF can deploy two canisters within each JBOF chassis for active-active or active-passive high availability; it can also use just one canister to achieve higher efficiency in cloud storage scenarios where redundancy and failover are handled by software across multiple JBOFs.

Each controller canister can house up to two NVIDIA BlueField-3 DPUs and one NVIDIA GPU card. Customers can deploy two canisters per JBOF for high availability or one canister to improve cost efficiency and reduce power consumption.

The dense design is well-suited for both vertical scaling and horizontal scaling of file and object storage, while the high network throughput (up to 800Gb/s per JBOF) supports the demands of AI training and high-performance computing workloads.

Testing of the Supermicro JBOF has shown that under storage workloads with one BlueField DPU installed, it saturates the 400-Gb/s network connection. The BlueField-based JBOF also demonstrates a 13% reduction in latency for small-block (4 KB) random read workloads, with the new JBOF clocking in at 86 μs compared to 100 μs for traditional x86-based JBOFs.
Replacing CPUs, memory, network cards, and BMCs with a single DPU card can save up to 50% of the power for the non-SSD subsystem or up to 10% to 15% of the total power for the entire JBOF (including SSDs). In large-scale horizontally scaled storage deployments, this translates to kilowatts of power savings for data centers.