AI Infrastructure interview prep

• CUDA Kernel Profiling, Occupancy & the Roofline — Nsight Systems and Nsight Compute workflow, occupancy tuning, warp stall analysis, ILP, warp divergence, and C
• CUDA Memory Hierarchy & Access Patterns — Coalesced global memory access, tiling with shared memory, bank conflicts, warp shuffle intrinsics, and asynch
• Disaggregated Prefill-Decode Architecture — Separating compute-bound prefill and memory-bound decode onto independent GPU pools for independent scaling. K
• Distributed Training: NCCL, Collectives & Parallelism — AllReduce, AllGather, ReduceScatter, and Broadcast collective operations. NCCL topology awareness, ring vs tre
• GPU Architecture & AI Hardware — NVIDIA GPU architecture from streaming multiprocessors through tensor cores, HBM memory, NVLink, and Blackwell
• LLM Inference: KV Cache, Batching & Scheduling — KV cache mechanics, memory management, and paging (PagedAttention). Continuous batching (Orca), chunked prefil
• Mixture of Experts (MoE) Inference — MoE architecture, top-K expert routing, expert parallelism, load balancing, all-to-all communication, and Deep
• PyTorch torch.compile, FSDP2 & Training Optimization — torch.compile internals (TorchDynamo, TorchInductor), FSDP2 sharding strategies, activation checkpointing, mix
• Quantization & Model Compression — FP8, INT8, INT4, and FP4 quantization. Post-training quantization (PTQ) vs quantization-aware training (QAT).
• ROCm & the AMD GPU Stack — A bottom-to-top tour of the AMD ROCm stack — firmware, amdgpu, amdkfd, ROCr, HIP, and math/comm libraries — an
• Speculative Decoding & Parallel Token Generation — Draft-then-verify speculative decoding, Medusa multi-head prediction, EAGLE speculative decoding, lookahead de

Start free on Cariara · Coding problems · System design studio