Project Lightning Talk: Hami: Dynamic, Smart, Stable GPU-Sharing Middleware In Kubern... Mengxuan Li

This talk covers HAMi, a dynamic middleware designed for efficient GPU-sharing in Kubernetes environments, presented by Mengxuan Li, a maintainer of the project. In the context of AI and machine learning, the speaker discusses the challenges of managing diverse GPU workloads across various hardware vendors and how HAMi addresses key issues such as exclusive allocation, device topology awareness, and observability. The session highlights HAMi's capabilities, including safe device sharing among containers, scheduling optimization through inter-topology analysis, and a unified monitoring system for heterogeneous devices. With support for over 10 hardware vendors and adoption by more than 100 enterprises globally, HAMi has been instrumental in improving GPU utilization across extensive deployments.