Multi-tenant GPU Clusters & Confidential Computing
Sharing GPU clusters between tenants safely: whole GPUs, MIG, time-slicing, MPS and vGPU, noisy neighbors, network and storage isolation, Hopper and Blackwell confidential computing with attestation, and what each level protects.
Overview
Why this topic
One cluster, many tenants: product teams, outside customers, a research group with sensitive data. Each tenant wants the GPUs to feel private. How private they really are depends on the isolation level, and every level is a trade between utilization and protection. Confidential computing adds a level the operator itself cannot see into.
> [!IMPORTANT] Name the threat before picking the level. Keeping a neighbor from slowing you down and keeping the cluster operator from reading your model weights need very different controls.
Interview questions
- A tenant asks "is my workload isolated?" Explain what each sharing level does and does not protect.
- How do you find and fix noisy neighbors on shared GPUs?
- Walk through bringing up a confidential GPU workload on Hopper, including attestation.
- What does GPU confidential computing protect against, what does it not, and what does it cost?
- Design a multi-tenant GPU platform for internal teams plus one external customer with sensitive data.
- Isolate tenants on the network and storage of a GPU cluster.
Sign up free to read the full topic
More AI Infrastructure topics
- GPU Quotas, Utilization & Chargeback On-prem
- GPU Virtualization: Passthrough, vGPU, SR-IOV & MIG in VMs
- GPUs on Kubernetes: GPU Operator, Device Plugin, MIG & Time-slicing
- LLM Inference: KV Cache, Batching & Scheduling
- Manufacturing Test Data & Yield Pipelines
- Mixture of Experts (MoE) Inference
- Post-silicon Validation & Bring-up Labs
- Power & Cooling for GPU Racks
- PyTorch torch.compile, FSDP2 & Training Optimization
- Quantization & Model Compression
- Rack-scale NVLink: Operating GB200/GB300 NVL72
- ROCm & the AMD GPU Stack
Start free on Cariara · Coding problems · System design studio