Sponsored Keynote: From Pilot to Production: Scaling and Optimizing Age... I. Levine & K. Babo (ASL)

This talk discusses the challenges and solutions for deploying AI agents in production environments. The speaker introduces K Agent, a project designed to enhance Kubernetes by treating agents, skills, and tools as first-class entities. Additionally, they present the Agent Gateway, which facilitates the deployment of agents with features like memory management and session optimization, while reducing the operational burden on platform teams. The talk highlights the importance of an efficient and community-driven gateway, as well as the Agent Registry project, which offers a comprehensive catalog for managing agentic artifacts. This registry supports multiple runtimes and integrates with metrics and telemetry tools to provide visibility and security across deployments. The speaker emphasizes the momentum of these projects within the open-source community, particularly their integration with the Linux Foundation and CNCF.

So, with the crazy pace of innovation that happened today in the AI ecosystem, the developer, the AI developer in the organization, we would love to run everything. And they would love to run it everywhere. Right. Okay, so creating with anything, deploying anywhere is great for pilots and POCs, right? But like not production. Right. And that's exactly the that's exactly what Solo is aiming to solve. All right.

So, filling this production gap, let's start with everybody's favorite runtime, Kubernetes, right? Um unlike the other agentic runtimes you see in this picture, Kubernetes lacks a direct representation of agents. And that's exactly why we created the K agent project to treat agents, skills, and tools as first-class entities in the platform. K agent acts as a general-purpose agent harness like Open Claw with declarative agents, but also supports

deploying agents you develop in other frameworks like LangChain and ADK and CrewAI. Agent Gateway helps these developers that are targeting Kubernetes with agents with features they need like memory management, session optimization, human-in-the-loop controls, but it also takes away all the undifferentiated heavy lifting that platform teams need to host production-grade agentic infrastructure. So, next, let's talk about networking and security. When you're looking for a gateway, what are

you looking for? So, the first thing you need, it need to be a good gateway, right? Needs to have all the features that you want to run today, but even more important, you need to be able to run the feature that we will need in the future. We built Agent Gateway architected for the down up to basically pick catch up with the AI ecosystem. Second of all,

it need to be efficient. You need to run at scale without taking crazy amount of your infrastructure resources. We could publish a public benchmark that you will be able to take it yourself and run it in your infrastructure and assess how efficient Agent Gateway is. Then, you want community. And even better, an open government community. So, Agent Gateway is a project under the Linux Foundation. There's tons

of companies and users that are donating it and contributing it already right now. And last but not least, you need users. You need users that will run it scale with a real-life use case. And hopefully, those users are happy. So, the next problem we see everywhere is visibility. When teams are hosting agentic platforms, visibility into what's actually deployed, right? This is a major problem in the industry.

We created the Agent Registry project to solve this by providing a single catalog and pane of glass for all of the agents and tools, skills, prompts, evaluators that you're going to use across your environments in agentic applications. But, it's much more than a static catalog. It supports import and federation from multiple private and public registries with a curation workflow to make sure that you're only exposing approved

agentic artifacts to your developers. It even supports being exposed as an MCP tool server. So, if you want to hook it up to like a general-purpose agent framework like Cloud Code or K Agent, you can do that with semantic search. And right after you will have a central registry, you will want to hook it up to one of the runtime. Probably more than one runtime. With Agent

Registry, you can actually register agentic runtime like K Agent, Vertex, Agent Core, AI Foundry, and even Nimo Club from uh from Nvidia. Once you will register it, agent agent registry will be continuously discover and drift detect drift of your environment. That will help you a lot with your shadow IT. Next, we created a central and unified deployment that you will be able to deploy all your agentic

workflow to those platform. We also going to aggregate all your metrics uh and uh and tracing with open telemetry. The rain is nice. With open telemetry and uh and uh and uh hotel pipeline all the way to your favorite dashboard. And integrated with agent gateway, you basically getting a central and unified way place to run all your to enforce all your security as well as your access

policy. Okay, so with agent gateway in the Linux Foundation, K Asian in the CNCF, over the last year we've seen tremendous momentum in these communities of use and contribution. That's just been incredible to watch over the last year. So, Edi, we were just talking about registry and how that is this critical gap in the industry that should probably be in an open source foundation as well, right?

Yeah, so let's fix it right now. So, let's donate agent registry to the CNCF, right? Yeah.