Running AI Workloads in Containers and Kubernetes - Kevin Klues

This talk explores the efficient management of machine learning and AI workloads in the cloud using containers. The speaker covers how GPUs are utilized with both standalone containers and Kubernetes, examining techniques for GPU sharing such as time-slicing, Multi-Process Service (MPS), and Multi-Instance GPUs (MIG). Attendees will gain a thorough understanding of GPU support in these environments, enabling them to optimize GPU use in their applications. The session concludes with a practical demonstration. Kevin Klues, a distinguished engineer at NVIDIA’s Cloud Native team, brings his expertise in Kubernetes technologies to this discussion.