Beyond JSON-RPC: Scaling Model Context Protocols With gRPC in the Py... Ashesh Vidyut & Madhav Bissa

This talk by Ashesh Vidyut and Madhav Bissa from Google addresses the limitations of Model Context Protocols (MCP) that currently rely on HTTP and STDIO, particularly for high-performance PyTorch models. They introduce SEP-1352, which integrates gRPC as a native transport for MCP, offering significant improvements in throughput and latency by leveraging Protobuf instead of JSON. The session explores the advantages of using gRPC's streaming capabilities for handling long-running model outputs, production-ready features such as authentication and load balancing, and provides insights on upgrading from PyTorch MCP HTTP services to gRPC services while preserving existing infrastructure.