Lessons Learned Orchestrating Multi-Tenant GPUs on OpenShift AI with NVIDIA KAI (G/H2... Luca Berton

This talk covers the orchestration of multi-tenant GPUs on OpenShift AI utilizing NVIDIA KAI and G/H200 hardware. The speaker, Luca Berton from Dell Technologies, shares valuable lessons learned from running shared production-grade GPUs for AI and machine learning safely and efficiently. Key topics include tenant isolation patterns, scheduling on heterogeneous nodes, and tuning for throughput versus latency, along with strategies for upgrades, rollbacks, and essential operational considerations such as observability and autoscaling. Attendees can expect practical insights and manifests to implement in their own setups immediately.