Model-Changing Transforms With Torch.compile - Thomas Viehmann, Lightning AI

This talk covers the capabilities of torch.compile, a tool designed to enhance the performance of PyTorch models through various model-changing transformations, such as quantization and distribution. The speaker, Thomas Viehmann from Lightning AI, explores the internals of torch.compile, emphasizing how transformations can be applied without altering the model itself. He discusses the integration of PyTorch's Python and C++ components within the Dynamo frontend of torch.compile, demonstrating how to leverage these systems to optimize models originally designed for single-GPU environments. The session also highlights the development of an innovative transformation mechanism informed by the experiences gained with Lightning Thunder, an experimental compiler for PyTorch.