DPUs in GPU Clusters: BlueField & DOCA

Overview

Why this topic

GPU servers are expensive, and every host CPU cycle spent on virtual switching, storage protocols, encryption or firewall rules is a cycle the training job does not get. A DPU (data processing unit) is a network adapter with its own Arm computer on the card. It runs that infrastructure work beside the host instead of on it, and because it has its own operating system, it can enforce policy even when the host is not trusted.

GPU cloud operators use DPUs to give tenants bare-metal servers while keeping the network and storage under the provider's control. Interviewers ask what a DPU actually offloads, how it differs from a SuperNIC, how its software is installed and upgraded, and what breaks when a fleet of them is managed badly.

Interview questions

• Trace a packet from a VM on a DPU-equipped host to the wire. Where is the policy enforced?
• Plan a BFB and firmware upgrade across 2,000 GPU servers with BlueField DPUs.
• What does host-isolated (zero-trust) control on a DPU protect against, and what does it not?
• Design the network cards for a 64-node GPU cluster with 8 GPUs per node. Where do SuperNICs go, where do DPUs go, and why?
• How does DOCA SNAP give bare-metal GPU servers storage, and why would a cloud choose it?
• After switching a BlueField to NIC mode, a host shows no BlueField network interfaces and you cannot reach the Arm OS. Debug it.

More AI Infrastructure topics

• Bare-metal Provisioning & Fleet Firmware
• Batch GPU Scheduling on Kubernetes: Kueue, Volcano & Gang Scheduling
• CUDA Kernel Profiling, Occupancy & the Roofline
• CUDA Memory Hierarchy & Access Patterns
• Disaggregated Prefill-Decode Architecture
• Distributed Training: NCCL, Collectives & Parallelism
• EDA Licenses & Compute Farms
• Emulation & FPGA Prototyping Pipelines
• Firmware CI with Hardware in the Loop
• GPU Architecture & AI Hardware
• GPU Cluster Fabrics: InfiniBand & RoCE
• GPU Driver & Software Stack Release Validation