Virtualizing Large Scale GPU Cluster for Sovereign AI: Petasus AI Cloud Journey with Kube... Jian Li

This talk focuses on virtualizing large-scale GPU clusters for sovereign AI, presented by Jian Li from SK Telecom. The speaker discusses the architecture of the Haein Supercluster, Korea's largest AI cluster equipped with over 1000 Blackwell GPUs, which plays a vital role in Korea's Sovereign AI initiative and is built on CNCF projects. The session highlights the use of Kubernetes and KubeVirt to achieve minimal performance overhead, along with techniques like NVLink/NVSwitch and GPUDirect RDMA virtualization that outperform traditional PCIe and TCP-based solutions. Attendees will explore resource partitioning through namespaces for on-demand GPU cluster creation, the use of automation tools to streamline provisioning, and the role of DCGM and Prometheus in improving observability and minimizing service downtime. The discussion also addresses challenges in multi-tenancy for sensitive workloads and shares best practices for cloud-native AI data centers.