To Swap or Not To Swap: Memory Management Design Patterns for AI Workloads in Kuber... Nic Vermande

This talk by Nic Vermande from ScaleOps covers memory management design patterns for AI workloads in Kubernetes 1.34 and above, focusing on the implications of swap support now that it is stable. The speaker examines the complex debate surrounding swap in the context of AI and ML workloads that exceed 100GB memory footprints. The session features three real production scenarios, including strategies for overcommitting memory, handling burst traffic, and situations where swap can negatively impact training workloads and real-time inference. By the end of this talk, attendees will gain insights on when to enable or disable swap for optimal performance and receive production-tested configurations.