Our journey to horizontally scalabe controllers - Jan-Gerrit Göbel

In this talk, Jan-Gerrit Göbel, a core maintainer of the open-source learning platform hobbyfarm, discusses the challenges of scalable Kubernetes controllers. He explains how hobbyfarm, which enhances training programs at SVA by providing pre-configured environments, faced performance issues as its user base expanded. To address slow-reconciling controllers that caused longer wait times and occasional outages, Jan proposes using horizontally scalable controllers through sharding, a method typically applied in large databases for data distribution. He compares this innovative approach to existing scalable solutions like leader-election and external message-queues, sharing insights from his experience implementing these horizontally scalable controllers.