Deploying PyTorch Models To the Browser and Beyond With Transformers.js - Joshua Lochner

This talk covers the engineering roadmap for running Hugging Face Transformers locally in web browsers using Transformers.js. The speaker discusses the end-to-end pipeline necessary for exporting, optimizing, and deploying PyTorch models to the web, utilizing emerging web technologies like WebGPU for efficient inference. The session dives into converting PyTorch models to ONNX using torch.export (Dynamo) and employing runtime-specific optimizations with ONNX Runtime GenAI. The workflow presented allows for the creation of highly efficient, quantized model artifacts that are suitable for browser execution. Finally, the speaker demonstrates the deployment of these optimized models with Transformers.js to build engaging WebAI experiences.