Make GenAI Production-Ready With Kubernetes Patterns - Roland Huss, Red Hat & Bilgin Ibryam, Diagrid

This talk focuses on making generative AI (GenAI) production-ready using established Kubernetes patterns, presented by Roland Huss from Red Hat and Bilgin Ibryam from Diagrid. The session demonstrates how to utilize controllers and custom resources to create stable model endpoints, as well as how to implement sound startup hygiene for consistent model rollouts. It emphasizes the importance of GPU-aware placement to optimize resource allocation and explores token-based traffic signals for improving latency management. By the end, attendees will gain a comprehensive understanding of how to integrate a retrieval-augmented generation workflow with proper orchestration and background data ingestion, equipping them with actionable insights for applying Kubernetes in AI environments.