TorchStore: What We Learned Building Distributed Storage Sol... Lucas P, Danielle P, Allen W, Amir A

This talk explores the challenges and solutions in building distributed storage systems for Asynchronous Reinforcement Learning (AsyncRL) workloads, emphasizing the efficient exchange of large tensors across processes and nodes. The speakers present Torchstore, an open-source distributed tensor storage system that utilizes Monarch actors to address continuous weight synchronization and varying sharding configurations during training. Key lessons are shared regarding the design of pluggable transport backends such as RDMA and shared memory, as well as the implementation of live DTensor resharding. The discussion further highlights the friction encountered while integrating with inference engines like vLLM, providing valuable insights for those involved in actor-based training systems or disaggregated training-inference architectures.