Breaking the Monolith: Decomposing and Governing Giant LLM Jobs Across Clusters - Kevin Wang, Huawei

This session features Kevin Wang from Huawei discussing the challenges and solutions for managing large language model (LLM) jobs across multiple clusters. The talk introduces the concepts of multi-cluster architecture, with a focus on adaptive cross-cluster scheduling using Volcano Global and Karmada. Key topics include the implementation of a universal global scheduling control plane, the creation of a higher-level job abstraction for intelligent job decomposition, and the establishment of a centralized global queue. This approach aims to enhance resource utilization and flexibility while ensuring fair allocation of resources and preventing bottlenecks in shared environments.