Towards Building an Open Source AI Reference Stack for EU Sov... Madhav Bhargava & Sanjay Chatterjee

This talk covers the initiative to build an open source AI reference stack aimed at enhancing Europe's digital sovereignty through the NeoNephos foundation and the IPCEI-CIS initiative. The speakers, Madhav Bhargava from SAP Labs and Sanjay Chatterjee from NVIDIA, present an end-to-end open source AI inference stack developed on the Gardener Kubernetes platform. Key components include community container native Linux images with integrated NVIDIA drivers, seamless integration of the NVIDIA Kubernetes GPU operator, and the Grove project, which facilitates efficient multi-GPU multi-node distributed inference for generative AI. The session also demonstrates the stack's capabilities using the NVIDIA Dynamo inference platform to run disaggregated multi-node inference workloads.