Lightning Talk: Faster Than SOTA Kernels in Torch.compile With Subgrap... Elias Ellison & Paul Zhang

This talk discusses how subgraph optimization and custom operator autotuning in torch.compile can achieve performance improvements that surpass state-of-the-art results for matrix multiplication and distributed collective operations. The speakers, Elias Ellison and Paul Zhang from Meta, explain the innovative DecomposeK optimization, which enhances matrix multiplication speed, particularly when the inner dimension is large, offering up to a 28% speedup with activation fusion. They also introduce the Custom Op Autotuning feature, which benchmarks and identifies the fastest kernel implementations for custom operations, alongside a range-based dispatch autotuning method that dynamically selects optimal implementations based on input shapes. The results from their demo showcase significant performance gains, outperforming existing kernels by notable margins.