Lightning Talk: Combo Kernels: Horizontal Fusion Optimizat... Karthick Panner Selvam & Elias Ellison

This talk by Karthick Panner Selvam and Elias Ellison from Meta focuses on combo kernels, a compiler optimization in PyTorch Inductor that fuses multiple independent operations into a single Triton kernel launch. The discussion highlights the issue of GPU kernel launch overhead caused by numerous small operations in models, such as weight preprocessing and tensor copies, which can create performance bottlenecks. The speaker explains how combo kernels address this by employing a dispatch mechanism to combine operations while maintaining correctness. Key innovations include optimized per-subkernel block dimensions and a flattened grid dispatch that enhances performance. The results showcase significant speedups on H100 GPUs, with improvements seen in benchmarks like HuggingFace and TorchBench.