About this talk
In this talk, Luke, a solutions engineer at Cloudflare, discusses the advantages of building agents using the Cloudflare network. He explains how Cloudflare serves a vast amount of internet traffic efficiently, thanks to its extensive network and technological infrastructure, which developers can leverage without the complexity of traditional cloud services. Luke emphasizes the developer-centric approach of Cloudflare, which abstracts away many operational concerns, allowing developers to focus on coding. A main feature covered is Cloudflare Workers, a serverless compute platform that eliminates cold starts and scales seamlessly, ideal for running agents. He also introduces durable objects, which provide built-in state management for agents by integrating storage solutions effortlessly. Furthermore, he outlines essential components and future trends impacting the development of intelligent agents, such as persistent memory and managed OAuth.
Full transcript
Can you all hear me? Great, good to hear. So, my name is Luke. I'm a solution at engineer at Cloudflare and today I'm going to talk about building agents on Cloudflare and hopefully convince you that a that Cloudflare is a great home for your agents. So, for those that don't know, I quickly want to um take a step back and explain to you what Cloudflare actually is.
So, the way I like to usually explain this to our customers is that Cloudflare is the biggest computer network in the world. Uh we currently have 3,000 about globally. Now, in itself that is kind of like a meaningless fact, but what that allows us to do is serve 95% of internet traffic uh within 50 ms of latency. Now, as a reference point, um blinking with your eyes
is about 2 to 300 ms. So, we are actually about four times faster than that. Um this network has direct network peering with over 13,000 other networks and the total network capacity is 500 terabits per second. Now, what is relevant to know for agents is that um some of these pops, currently 250 215 pops, they also have GPU capacity and we're actually expanding this um as we
go, right? Now, a lot of people might know Cloudflare from our CDN capability where we use this network to deliver content or about our DDoS um network uh DDoS protection services where we use this network to absorb volumetric attacks attacks. But, um what is maybe less well-known is that uh we also give developers the power to directly build on top of this network. And these products that
allow you to do that the diff they typically fall in our developer product suite of products. it's good to know that, you know, with our developer platform, we are not trying to be like the next AWS or the next Google Cloud. Because if you would try to do that, then we would probably end up just being a lesser version of that. So, there's a very distinct vision
for the Cloud for Developer platform, which is really centered around the developer experience. And the developer platform currently already exists for 10 years. So, it's already a mature offering. So, instead of talking, you know, about the things that you can do when you build on Cloudflare, I would like to just take a moment and tell you the things that you don't have to think about when you
build on Cloudflare, because we abstract that away from you, right? So, when you build on Cloudflare, you don't need to configure instances. You don't need to select and manage operating your operating system. You don't need to issue SSL certificates or be concerned about setting DNS records. You also don't need to think about configuring a multi-instance setup. You don't need to configure a You don't need to think
about multi-region setups, or you don't need to think about auto scaling. And also the CDN that we have kind of like works out of the box. So, these are all things that we abstract away from you developers to make your life easier, so you can focus on the things that, you know, are most important, which is shipping code. Now, the secret is already out for the developer
platform, and currently we have over 4.5 million developers actively developing on Cloudflare. And this was as per the last measurement of April 2026. >> [clears throat] >> Now, the main programming primitive when you're building on Cloudflare is going to be Workers. And Workers are a serverless compute type, similar to Azure Functions or AW Lambda or AWS Lambda, except they're not actually similar. Um the former uh serverless
offerings that I just mentioned, they rely on underlying VMs. The Cloudflare Workers, they run on a different runtime called V8 isolates. Now, what you need to know about V8 isolates is that there is a lot less process overhead for running your user So, a single process can actually spin up hundreds and thousands of workers. Um and that has some nice benefits as well, being that when you
build with a worker, there are essentially no cold starts. So, the moment you make a request is almost immediately fulfilled. And because there are no cold starts, we are able to do some commercially interesting things, We are able to only make you pay for the active CPU time when you are running a process on a worker. So, in the case of agents, you know, a lot of
things a lot of times you are making an LLM an API call to an LLM provider, and then you're waiting for the response. When you're building with a worker when you're waiting for the response, the worker kind of like hibernates. So, you're not actively using CPU, so you're paying for that as well. And that's different from hyperscalers where you are actually paying for the whole wall time,
which is the moment a request comes in until the moment a request is fulfilled, right? And then a little bit some as well. Another interesting thing about workers is that you'd employ them once and they become globally available of in every single Cloudflare point of presence in the world. And to demonstrate that, I have embedded a worker in this presentation. Now, I'm going to invoke this worker
and it's going to show you how long it took to fulfill the request, and it's going to show you which Cloudflare data center fulfilled this request. Now, we're in Amsterdam right now, so the worker the worker got sent to the nearest Cloudflare data center here in Amsterdam. But next week I'm going to be in Stockholm. If I'm going to run this worker again, it will say Stockholm
right here because the request will be fulfilled by the Cloudflare point of presence in Stockholm. And you can actually get that out of the box. So this is high availability without you having to think about that. Another cool primitive that we have on Cloudflare is durable objects. And durable objects, they are built on top of Cloudflare workers. Um what they essentially are is they are a stateful
worker. And there are some nice um characteristics for the durable object. First of all is that they are single-threaded. So that means that one durable object instance handles all requests for a given durable object. That means that there are not going to be race conditions, there's no distributed locks, and that coordination for durable object is kind of built in. On top of that, each durable object is
globally unique. And that means that any request from the world to that durable object will be immediately routed to that unique instance. And then what's the most important thing to take into account with durable object is that it's a stateful worker. This means that the storage comes in and kind of like built in the worker itself. And there are different storage layers that you can choose from.
There's the key-value store or there is a D1 SQLite database, but you don't need to actually manage that storage. It's included in the in the durable object itself. And to show you that, here is a a durable object. It's uh a worker, but it has a state and the state is reflected by a simple counter. Currently, the counter is 16. If I'm going to increment the count,
it will stay it will stay 17. Now, I can refresh this page, I can come back here later, and the count will still say 17. So this is a neat thing about durable objects. And if you would want to recreate something like this on a traditional hyperscaler, you would need several components. You need a database, you need a caching layer, you need some distributed locking mechanism, and
something related to queues to manage all the requests, right? And with durable objects, you kind of get that in a single class and you don't need to think about the underlying architecture at all. So, what I just showed you is that uh workers, they're highly scalable and they can scale to millions of instances quite easily because of that V8 runtime. I also showed you that workers, they
scale to zero and that there is And that when you build with durable objects, which is a stateful worker, that you kind of like get the state of a worker included by default. Now, who do we know that benefits from this? It's this guy, right? It's an agent and I'll explain to you why. So, it's important for the infrastructure to scale to millions of because we as
Cloudflare think that agents are going to be one-to-one. Meaning that every person will have its own agent or maybe multiple agents running at the same time. And that's different from traditional architecture where, you know, multiple users, they use one single application. So, if we all have our own agent that scales uh that changes the scaling laws dramatically, right? Then, when it comes to scale to zero and
no no cold starts, a lot of these agents, they're going to sit idle and you're not going to be actively uh computing things uh all the time, right? So, what scaling to zero and no cold starts kind of gives you the best of both worlds. It gives you don't have to pay for the underlying containers all the time, but at the same time, you're also not compromising
because, you know, workers, they spin up like this. And then with durable objects, of course, again, built on top of workers, is that the state is included by default. So, for an agent uh in that case, you want to keep the conversation history going across multiple sessions. And that's why durable object is super cool primitive for agents. to build production production ready agents, you need a couple
components and we're not trying to pretend here that we we know best, but what we've seen when talking to customers is that these are generally the components that are required for a production ready agent. You kind of want to have a UI user interface for people to interact with the agent. Then you need to have an agent runtime. This is either a container or in the case
of Cloudflare durable object. Then there needs to be an agent orchestration framework. So this is either when I started building with LLMs, this was llama index or LangChain, but currently it's the AI SDK from Vercel, but also modern tools or harnesses, they also fall under this like open claw and um Hermes for example. Then there needs to be a memory layer, a way for the agent to
store memory and conversation over time. And you also want the agent to have an ability to execute code. Now, you even want to have this ability even when you're not building a coding agent per se because what we found is every computer has a terminal and if a agent can write code against that terminal, that means that an agent can control computer. And if a an agent
can control a computer, it's kind of like a gateway to information in the world. It's it's access to information. So that is why it's very important for an agent to be able to run code. On top of that, you want to have some browser access for the agent to get information that's up-to-date from the public internet. And then you might want to have an ability for an
agent to run sub agents to you know, execute those long-running research tasks. If you have MCP servers, these will be the entry points for an AI to interact with your application, Um you want to have those as well. if you are building a production-ready agent, then you might also want to think about how are we going to authenticate users against those MCP servers. And you might want
to set up different policies for different users. And finally, of course, you need the LLM. These are going to be the brains of the operation for the agent to interact with. And you might want to have a gateway sitting in between that, so you can cache responses or you can route uh between different model providers in case a model provider goes down. And then, there are several
components that are also important. Um, and those are, of course, observability, security, and if you want an agent to interact with your on-prem infrastructure, you also want to consider network access. Now, the good thing to know is that on Cloudflare, we offer all these components in a single platform. If you want to have a UI, you can use Pages of Workers. For the runtime, there's Durable Objects.
And for orchestration, we have the AI Agent SDK. Memory kind of like comes out of the box with a Durable Object, as we discussed earlier. And for code code execution, you can use um dynamic workers, which is basically an empty shell worker that you deploy without any code, and then the agent can dynamically write code and execute it in that worker. But we also offer the Sandboxed
SDK, which is a way for an agent or a human programmer to interact with an with an underlying container that's hosted on the Cloudflare network For browser access, we have a tool called browser rendering, which is an amazing API for interacting with browsers. And then, if you want to spawn sub-agents, we recently released this offering called Facets. And Facets is I mean, kind of like an inception
concept, but it's a way for a Durable Object to spawn other durable objects. Um apparently, it's a great way to uh run sub agent as well. Then for MCP servers, you can host these natively of Cloudflare as well in our MCP server portal. And then if you want to authenticate users against those MCP portals, we have a offering called Cloudflare Access. And with Cloudflare Access, you can
define different groups and different policies based on those groups, and you can put users in various groups as well. And then for the LLM, what we have is a platform called Workers AI. And Workers AI is our um AI platform that allows our customers to run LLM models on the GPUs that are in the Cloudflare network. So, these models run natively on uh the Cloudflare And then
as a gateway, we offer another product called AI Gateway. So, I guess the key takeaway from this is that um Cloudflare offers all the components and observability and security, they are included. And we also have a product for uh network access, which is Cloudflare Mesh. And I'm going to talk about that a little bit later as well. Now, we are at a developer conference, so want to
quickly show you some code on what it looks like if you're actually building an agent on Cloudflare. one of the first things you're going to configure when you're building an agent is the wrangler.jsonc file. So, this is the config file that you kind of have with every Cloudflare durable with every Cloudflare primitive on our developer platform. Uh and this is the config file for our research agent.
What we do here is we set certain bindings. So, here we create a binding for this agent to interact with our Workers AI platform. Here we kind of like map the agent class. We set up a binding between the agent class and in durable object. And then here we bind the agent with the browser rendering service. And here we bind another Here we set up another binding
for the agent to interact with a dynamic worker. And I'll show you how that works later. kind of like what you do is you import the packages here. And then the first thing you want to do is you want to expose this research agent class that extends the agent And then first thing you do is you want to use the sessions API to create an agent's memory
on top of the durable object. So in this case we have a uh session. And we create it by first setting a soul. And the soul is kind of like the system prompts. The agent cannot change this. And then we include another memory section. It's actually called memory. And what this does is this allows the agent to write facts and preferences that it learned over time to
the agent memory. And then we say, "Okay, we want to spend about 2,000 tokens on this specific memory component." And then there is another thing that we do. We cache the the memory. So the system kind of like uh the memory survives when the agent hibernates. And then we want to automatically compact the memory after uh 100K tokens. So the memory stays up to date when and
it doesn't get too long that it overflows the context. Then we can like expose certain hooks. So here we have an on start hook. So when this durable object awaits and the agent awaits in that case, then we automatically connect the agent to our GitHub MCP server here. Another hook that we have is an on connect hook. So when the client connects to this durable object, we
send over the history and the memory so that this can immediately be rendered in the client UI. Then finally we have a on message hook. So, this is when somebody talks to the agent. The first thing we do is we append the latest message to our history. Then we expose certain session tools for the agent to actively control that memory. So, this exposes the set context and
search context tools that the agent can call to actively search in its conversation history. And what we do next is we use this binding that we set up in the Wrangler JSON C file to set up an RPC binding with Workers AI. So, uh what that allows us to do is to automat- automatically connect to the Workers AI service without having to configure, you know, API keys.
So, another benefit of that is also if you're worried and you're doing a lot of uh agentic coding, that your API keys won't be leaked because there's no notion of API keys when you're building with Cloudflare. If you want to connect multiple services, you're using bindings, and this will set up an RPC binding between the agent and in this case Workers AI. And you'll see that again
here in the tools that we supply to this agent. So, in this case, we add a browser tool to the agent for allowing it to use the internet. And we again, we do this with environment variable and then the browser binding that we set up that is exposed through that environment variable. And we kind of do the same uh for code execution, right? So, we've actually now
we've built a pretty cool agent. We have an agent that has access to the internet. We have an agent that has ability to write code is able to remember things across multiple sessions. 5 minutes, got it. Now, if you want to deploy this, again, we focus on the developer experience. So, it's very easy for you to deploy this. All it takes is one command. You change into
the agent directory, and you call a Wrangler CLI tool, and you just call MPX Wrangler deploy, and just like that, your agent is now live on Cloudflare. So, everything that I showed you right now are components that you can use today to build agents on Cloudflare. But, we also think about, you know, what's coming next. And there are some industry trends that we see that are relevant
and we're going to build some of our future products based on these trends. One of them being that agents are going to run longer. And long-running tests, they need um persistent memory and reliable model access. So, we're going to release a primitive called agent memory. Now, I already showed you agent memory via the sessions API, but if you want to have a longer-running agent that stays with
you maybe for weeks, uh months, or maybe even years, you need a more intelligent storage layer. And this is what agent memory will be. It will actively extract facts, events, and instructions, and store it in a easily to searchable format so that the the context um stays relevant for a longer period of time, and there's a more efficient way for the agent to search through that context
as well. Then about our AI platform, we currently have 70 models models running natively on our Cloudflare network, but we're going to focus a lot on the stability and the speed at which we produce tokens. So, we're going to implement stream buffering for reliable token delivery, and later this year, we're also going to release the ability for customers to bring their own model and host it natively
on the Cloudflare network in all those edge locations. A second trend that we see is that agents are going to need their own identity and access. That's why we're going to release managed OAuth for agents. And what it kind of what this will allow the agent to do is to authenticate at itself and not piggyback on existing user credentials. scope tokens with limited permissions that are easily
auditable as well. And the way that would work is that a human would authorize on the agent's behalf once and then the agent acts on that user's behalf. we're also going to release cloud for mesh. So, this is a tailscale kind of setup where you can make an agent part of a mesh network and you can use that for the agent to interact with your on-prem applications.
And making an agent part part of a mesh network is again as easily as setting up a RPC binding via the regular config file with the the mesh network. And then you can use the bindings just like this to connect with your on-prem applications. And the benefit of that is of course that you don't have to configure VPN, no firewall rules, or no annoying net configurations. And
the last trend is that agents are going to need their own place to work. So, there has been a lot of issues and talk about GitHub lately because apparently the GitHub servers they're getting overloaded by all this agentic coding and they can't really handle the load. So, what we're going to release is we're going to release artifacts. And this will be a version storage that speaks Git
and it will be a repo per session or per agent or per sandbox and it will be a an agent native um play working environment. And we're going to also going to release an open source variant of this. It's going to be called artifacts FS. If you want to learn more about this, I've linked to the blog post here. And final thing and my colleague confidence is
also going to talk about this tomorrow is we're going to give every agent its own email inbox. So, each agent gets his own email inbox on your own domain and it can use this to receive tasks, to process tasks in Async, and reply you or your customers when the process is done. And access to the email service will be exposed via the on email um hook in
the agent SDK. the key takeaways for you from this talk I hope that I've convinced you that agent that Cloudflare is a great place for your agents. And it's also important that you know that we dog food this. So, we use our own technology to also build our own internal Cloudflare OS, which is going to be our agentic native operating system that we're going to release later
this year. And that the industry is moving fast, insanely fast I would say, and that, you we kind of like shaping it as we go. that's it. If you want to get started on with building cloud with agents on Cloudflare, talk to me or any of my colleagues who are around here about the booth and we're happy to help you get started on this journey. And um
thank you for your time. >> [applause]
