Portable High‑Performance LLM Serving: A Triton Backend for... Burkhard Ringlein & Jan van Lunteren

This talk covers the introduction of a Triton backend for vLLM, which is becoming the industry standard for serving Large Language Models in production. The speakers, Burkhard Ringlein and Jan van Lunteren from IBM Research, explain how traditional performance reliant on hand-written CUDA or HIP kernels can limit portability across different hardware. They showcase how the Triton attention backend offers competitive performance across GPU platforms using a single code base, eliminating the need for specialized kernels. The session includes insights into the engineering, system aspects, kernel enhancements, and optimizations that enable this Triton-only solution to deliver consistent high performance on both NVIDIA and AMD GPUs.