The Hyperscale Uncertainty Principle: Debugging Tail Latency in a Trillion-Object... Yashraj Kakkad

This talk, presented by Yashraj Kakkad from Google, focuses on the challenges of debugging tail latency in a trillion-object system at Google Photos. It highlights the integrity pipeline designed to maintain data reliability and how it encountered unexpected latency issues despite a well-structured partitioning strategy. The speaker explores the non-linear effects of P99 latency and discusses three competing solutions, including traffic shaping, request hedging, and the innovative "Partition Alignment Principle." This session provides valuable insights into the complexities of hyperscale Site Reliability Engineering and emphasizes the importance of predictability in system performance.