Lightning Talk: TorchJD: Jacobian Descent in PyTorch - Pierre Quinton & Valérian Rey

This talk covers Jacobian descent (JD), an advanced optimization technique that extends gradient descent to optimize vector-valued functions, making it particularly useful for multi-task learning in neural networks. The speakers, Pierre Quinton from EPFL and Valérian Rey from Simplex Lab, discuss how JD utilizes the Jacobian matrix of losses to iteratively update model parameters, allowing for better performance compared to traditional gradient descent approaches. They also introduce the TorchJD library, designed to simplify the computation of Jacobians and facilitate more effective updates for multiple objectives. The session includes an overview of the theory behind Jacobian descent and practical demonstrations of using TorchJD in various applications.