GPUs on Kubernetes: What Actually Happens When You Request Nvidia... Gulcan Topcu & Daniele Polencic

This talk explores the intricacies of GPU scheduling in Kubernetes, specifically addressing what occurs when a user requests Nvidia.com/gpu: 1 in their pod specification. The speakers, Gulcan Topcu and Daniele Polencic from LearnKube, trace a GPU workload from start to finish, detailing how device plugins communicate with the scheduler and how the container runtime integrates GPU resources. Attendees will gain insights into the challenges of sharing a single GPU among multiple pods, examining solutions such as time-slicing, MIG hardware partitioning, and software enforcement. This session is designed for those curious about the technical underpinnings of Kubernetes and GPU utilization, no prior GPU knowledge is required.