Your API Is Not Ready for AI (Lifecycle Readiness Guide) - Rohit Bhardwaj

This talk focuses on optimizing API design for better machine readability and usability in projects. The speaker discusses the importance of developing APIs that not only serve human needs but are also comprehensible to machines, thus facilitating smoother integration with AI and automated systems. He emphasizes the need for implementing reliable patterns, such as idempotency, error handling, and adherence to OpenAPI specifications for effective API lifecycle management. Additionally, the presentation introduces a framework called 'machine' to aid in building robust APIs while addressing common pitfalls such as schema drift and non-deterministic responses. The discussion also touches on the challenges faced by agents and the necessity of creating predictable behaviors for seamless user experiences. Finally, the importance of developing APIs that are trustworthy, efficient, and easy to test and validate is highlighted, giving attendees actionable insights to enhance their own API projects.

All right, thank you so much. So, first of all, like you know, I always ask the question, like you know, hey, welcome everyone. So, it's it's the last day and I know we are we have learned a lot here and you know, one of the thing which I always look for is what are the practical things you can take it from on the API side. The talk

is really for that. How can you prioritize? What are the things I should do now in my project? Because you have to write a story, yes? We all need to write a story. We need to tell product manager what to do and how to do it and that's what this talk is all about. This talk has to do with life cycle. So, when I say life cycle,

so API, we have developed API, post, you know, REST API, we're all developing that. But really speaking, have we made it for a machine to learn it? That's what the real question is. So, what happens is that we have built these APIs for long time, but it's really human readable. Yes, we we're reading the specs and we're trying to understand and that's where this talk is coming

in. Now, the thing which you might want to look at is this tiny URL. Uh you want to keep that in your back pocket, which is a iapi ready. So, if you have this, the talk is there, all the information and also a checklist is also there which you should apply while creating any APIs moving forward. Okay? So, if you have if you're building any API, take

a look at that, that'll really help you build that API for yourself. Okay, so that's what we are going to start on. Why does this human friendly is needed? How does this really affect, you know? That's where the practical guides. How can I do it in one, like you know, let's say in 4 weeks, come to a better API. Now, API design is needed so that the

human can understand it. How the agent understand that's the transformation journey we're going to do today. So, this whole 1 hour is dedicated towards making our life easier. Not our life easier, but a life easier from the API strict. So, machines is able to look at the schema, understand it, and polymorphically make a change what needs to be done. So, why this talk really exists? This talk

exists because you're starting from detecting a system issue. The notification is sent and then what happens right now? If you have a defect, how do you find a defect? Currently? Customer customer calls? Customer is calling, "Hey, there is a problem in the API." Yeah? And then we look at, "Okay, there is a problem in the API. Okay, issues in there in the data, issues there in the

authentication, issues there in the billing, and integration, and then the API usage." And then an API is prepared for this usage or not. If it is not prepared, now we need to apply mitigation strategies to really make your API in the right format, you know? Ultimately, why do we do all these things? Because because we want to make sure you follow the right patterns to to create

these APIs. So, if something goes wrong, you come to know before the customer knows. Because if customer is finding out something is going on wrong, that's a little bit late. So, what really changed? The change is human is reading the documentation, agent agent reading emails. So, this is where the old world is. You have a API created for a human to read it. Completely changed now. Now,

we have an LLM schema. Agents are coming in and API needs to be read by a machine. So, machine is reading this API, not the human. So, human is reading a Swagger documentation. Instead of that, we need to read this schema definition. Adhere to the schema definition and then create our perspective. How do we do that? How do we make this thing happen? Now, there could be

a lot of problems which can happen, you know, server behavior, um Adam potency keys, checks, poor data validation. Most of the problem really is if I I don't want to read the whole thing here. Most of the problem is in the data. You know, the heart of the problem is where? Data analysis is the real problem. So, go back and check how many duplicate how many have

duplicate problems. How many have bad data? Okay, you say that, "Hey, your data is there which is secure data is there." Um um it could be there also. Uh you you can you can build build build the build the perspective for it. So, the perspective is that data can change anytime. So, it's not static to begin with. Hey, you have a customer's address, email is there. Is

it static? No. Every month, people are changing jobs. Every month, the in the the company is changing their location. So, that's where the D&B comes in, Dun & Bradstreet's comes in, where you have to check for a business Anybody here in business to business? You need to find out the business actually changed the address. Do Do the business tell you you they have changed the address? No,

how do you do that nowadays? You have to have some service, some API to validate if those data is there. So, those things we should do, and then instead of human-centric, how can I make it the trustworthy API for us uh so that I can have AI efficiency, predictability coming in for any of the projects which is coming on. So, this is a new mantra coming in.

How can I determine through schema or real predictivity in the system? So, what that means, LLM is not predictable, yes? LLM is What is LLM? Large language model doing? Yeah? Non-deterministic. But, you know, the ask is to make it deterministic. Our job as an architect is to make it You know, not non-deterministic. That means if I'm asking a question, it returns back the answer which I'm expecting.

You should not hallucinate in that one. So, that's where that's where these three pillars are are coming in. So, what that means is you need to have a API scorecard. So, by the end of the talk today, you will get a scorecard. You will be able to discover what the problems are there and create a determinism. What's the output should look like? Normally, sometime you have an

API, sometime the API returns this. Sometime based on the condition, you return something else. Is that a good thing or a bad thing? It's a good thing for the developer because job security, yeah? >> [laughter] >> Because that developer only knows good luck. >> What really is happening? But, it's really not good for us. So, that's where the that's where the problem is coming in. So, what

do we need to do? The new customer has come up. This new customer is the autonomous agent. This agent is there which is making efficient decision. So, he's making all these decisions coming for us and how these decisions are made and reliable performance needs to be there for this agent and you want to scale this application. You know, we're building an API, an agent is there who's

solving all these problems. But, I need to understand the psyche of this agent, yes? Like, I need to like the render, you know? I need to understand the person's motivation, like, you know, why he went to other place and, you know, what is he what's the real purpose of of of the problem which is coming in. So, that's what we will be looking at today. And if

I don't do that, they don't have intuition. So, there's something I'm working on on on on my side like you know, I have a patent working on that is how can I make an agent a human intuition? Give them intuition. Still working on that. Not there yet. So, we tried to do it. It's It's still a lot of work to do that part of work. So, over-reliance

on the schema. So, that These are all the problems which are there in the agent side. So, what is an agent really doing? Think about an agent is a is is a customer. Customer support agent is there. Customer support agent is a front line customer is calling and then it's calling the front line service agent which is calling inventory agent and other agents in this case. Now,

these are all different different types of agents are there which are working on it. And our job is to predict what the those agents are doing. So, in this strategic area of of of of our usage, what's really happening is that you know, we are creating monographic agents. That means you're having one agent to solve a problem. You're creating APIs to support that particular agent and they

have repeated tasks to do that. Now, the multi-agent orchestration is also coming in. That means one agent is calling another agent. By the way, the API remains the same. Rohit, where are the APIs? Where do you think are the APIs here? I didn't talk about API. But where do you think the API is? Anyone? Or you have an ensemble agent also in this case. So, what's really

happening is that you were trying to have repetitive tasks, knowledge retrieval, summarization, ticket management. So, these are the different different tasks which are there which you going to go through through the evolution process. Now, think about an agent service agent. Let's just talk about service agent. What are the jobs the service agent does? Actual service agent does. Actual human does. Anyone? What are the jobs they do?

>> Customer service >> Huh? >> Customer issue coming in, they're trying to solve that customer issue. Huh? Actually human agent, yeah. No. >> Uh the work he or she will do the work on behalf of us like Amazon. >> Yeah. Yeah, so they will be solving a problem. Yes, this is somebody's asking a question, "Hey, how does this product works? How do How do we make the

changes up in this case?" So all those things are possible, you know, in in this one. So how can I take that and create an automation for it as a microservice? That's the goal which we have got. We are all here to create those microservices or APIs to solve that problem. So that means find out all the jobs to be done in different stages of the customer

journey. Customer wants to buy the product, they have question on the product. Yes? Customer want to decide which one is the best product for them. Customer wants to have already bought the product, but they don't know where is the product is. Customer have bought the product and they are happy with this or they are sad with us. So those are That's called customer journey. Based on the

customer journey, now we identify these tasks for us and we create these orchestration agents to really solve end-to-end for that particular customer, you know, how how the journey is going. So that's what we will be looking at uh today as part of our uh our work. And when you're doing that, think about it. It's really batch processing. You're transforming and you're embedding and you're trying to come

up and store the data so you can answer these questions over the time, okay? So as you can see, it LangChain is like, you know, you get source from multiple sources, okay? You get this data in, you transform this this and you put in the embedding. So what does that really means? Let's talk about it. Customer is not happy. How do you say in floating point numbers?

Anyone? How would you say that in floating by >> Wait. >> One to five rating. >> One to five rating is good, but the thing is that they're not even giving one to five rating. They are just making a call, okay? They're making a call and then they say, "Hey, this product is not working for me." Or in the social media they are having some place like

they say, "Hey, this product doesn't work." or something like that. Now, we need to ingest that data, transform that data into a format which I can embed and then once I embed I can see that hey, this customer is happy or sad. Sentiment analysis I'm trying to do. Based on that sentiment analysis now I can work on it and create my perspective. Just like marketing specialist is

there, is is trying to what's the job description for it? And for this job description, why do we have him? Because it helps in high net promoter score, increase customer satisfaction, ROI increasing. Now, these are actual micro marketing specialist. How about I create an agent for it? An agent is created which has this outcome coming in. Till the time you have an outcome, does it is there

any value of that job if there's no outcome there? No. So, you need to find out what the outcome of that is and then kind of work towards it. Make sense? That's the reason this job is coming in. So, you need to have a role defined. An agent has a role defined. So, APIs, whenever uh somebody's calling an API, you need to know which is who's calling

my API. Is an agent making a call or someone else is making a call? Yeah? And then same thing is true with the trusted data. What data, what knowledge articles, what customer data this agent has access to and not access to everything else. So, you need to restrict the the agent to have only small amount of data access to. What What actions can I perform for this

job? For example, I'm performing a job right now. Uh this this problem There is a fault code coming in in in in the engine there is a fault code coming or in the machine the fault code is coming. Somebody's calling me. I look into that fault code and then I'm trying to find out what the answer is from the knowledge articles and what should I do next?

Yeah? Those are the three things I'm doing. If an agent need to do that how would he do it? What is the problem? So first of all I need to find out what job needs to be Once I find out what job needs to be done I'll call an action which is an API I'm making a call to and then that API will tell me the grounded

data what I need to return back to the customer. Yes? And that's where the this API really comes comes in picture. So let's take a look at this in a little bit more detail what really is happening One second. Okay. So I'm going to just show you. So this is this is one of the one of the agent. I'm looking at one of the agent right now.

Let's see if the agent is coming up. Yeah, so this is the agent which is there. Now based on this agent which is which is coming here I need to be able to find out, you know, what are the jobs this agent do. we we need to know what these job what the jobs the agent is doing and based on that job what the agent is doing

I need to predict what kind of API calls I need to make. Yes? And that is something which we need to because we we are doing that manually right now. You know, we used to do that same thing manually. So I'll just go in and I just provide this details. Hey, I want to find out for this resort what is the full moon party going on. Now

take a look at this. What are the what are the what are the actions which are there? I can do account management, case management, delivery management, experience management. Okay, let's click on experience management here. Take a look at this. Now, this experience management is the topic or the sub agent which is there. Sub agent which has a which has a scope, what things this sub agent can

do. And instructions on how to call the get experience details, which is an API call which I'm trying to make here, and then bring that experience for myself. And these are the actions I want to do. So, the way the way the whole thing works is through jobs. I need to know what actions I need to take, and for those actions, what restrictions I need to have

to perform this operation. And then I have a planner here which is going to help me out plan what what do I need to make a call to. So, I can say that, "Hey, I am this user and provide this access to this user, and based on that, I need to find out I need to go for the full moon boot party." Now, a email has been

sent out. Okay, and then I'm able to interact with the agent at this time. So, the reason I showed you all these things is because that's exactly what's happening here in In this case, I have defined the actions, I have defined the guardrails. I'll talk about much more on the guardrails in one of my other talk in the security side. We'll discuss a lot on that, but

really speaking, you need to have guardrails for everything you're doing. For API you are creating, what are the guardrails for the API also you need to have in mind. And the channel where the data is coming from. And one other thing is like, you know, you need to know what challenge channel it the data is coming from. So, you if something goes wrong, you can protect yourself

from that blast radius. So, that's the that's the part which is important for you to do. Okay? Now, you may ask a question that, "Hey, what I showed you is the collaborative agent which is the orchestrator which is calling the sub agents to solve the problem." Okay? Now, one thing to keep in mind is that whenever you're making an API call, API call API is making another

call to another API which is making call to another API which is calling back the same API. Is it possible? It's possible, yes? Probably not a good idea, yes? So, that's what we want to avoid any loops in the API. So, that's another part of the puzzle we need to solve. Do not have loops in the API. So, that's agents going to be having more cost coming

in. So, that's something you have to avoid So, that said, you can also create ensemble agents, you know, when you are working on it. So, let's take a look at what are the different types of agents can be there. So, there you can have multiple agents like sales agent, service agent and they're all working towards and you need to create APIs for all of them. But, what

I want to do is that that, you know, let's see if I can show you this quickly and then we can we can move on from there, okay? So, what's really happening is and let me make a little bigger here. So, it's not one agent. You have like multiple of these agents are there. Sales agent is here, service agent is there and they need to all interact

with each other. When I'm creating an API, now this one is agent force. You think about Microsoft has a similar thing. And multiple products have the similar things there. Now, we are making a call in here to find out where is my order and I get the order back from there. So, this order is actually an API call which is getting in behind the scene. Now, how

do machine know that this is the order coming to me? And what are the problems in that? We're going to discuss that in the next 45 minutes, okay? So, you can have a SDR agent. SDR agent is sales development agent. Now, why I'm talking about these things is because you can just say this is not one API call. This is a state management going on here. State

management means you will find out this is the lead. Okay, somebody wants to get this lead and work on that lead. Yes? So, let me give you an example on the SDR agent. What's happening is that I want to reach out to these people to buy my product. Okay? Basically, they I want to say that buy this product and then because I want to buy this product,

I want to send an email out to them, answer the questions they have got, and schedule me. After the scheduling is done, then a human touches this. Okay? So, what all that means? All the API calls are automated through state management. So, there is something called state management also there, which also kicks in with agents. So, when the agent is making a call, you always have a

state to manage. What state am I in? Based on that state, your response changes. Based Based on the state, your jobs to be done done changes. Yes? Customer is discovering the what what your product is. Are your jobs to be done the same? No, they're not the same. Uh customer bought the product, the jobs to be done is different. So, that's the that's the way it's important

to understand what's going on in this case. Now, another thing which is coming in is persona. So, this one is one is a virtual assistant, one is actual actual person. And they are both talking to And And you can coach Hey, something is going wrong. You can coach for that. And if something going wrong, you need to be able to voice is integrated into the system. Now,

you think about voice is integrated. Behind the scene, I'm still making API call and understanding the user and trying to build a solution for So, you can have a job created and based on the job you have you have multiple sales coach can be added and the other coaches can be added or you can start from the scratch. Okay. I want to start from scratch right now

and I want to build for this user. My role is that, you know, managing the customer orders. I want to build the managing the customer orders. Based on the managing of the customer order, what are the jobs the customer order has to has to do? Now, can I create a microservice for order management, scheduling management, and other ones like this which any agent can use? Can I

do that? Yeah. So, that means that I can have the jobs to be done and actions ready to go, plug and play, and send it to the agent and they can start using it. And that is the way we can create a reusable microservices to to help us out in this puzzle. So, now I'm able to ask the question, "Hey, what time slot are you available?" and

then book that meeting for for that particular usage. Sometimes I'm asking a bad question, you know, "How would I become a president of this company?" You know, explosive. You know, you can say all these bad things. Now, these are called guardrails, you know, which which we need to kind of implement in the APIs and also also in the LLM when I'm working on this. So, why did

you discuss all these things, Rohit? We discussed all these things because we need to be able to create not infinite loops, but ambiguous errors are coming in. Errors are concealed in the APIs which are coming in for us. For example, junior developer is there who's working on the AI agent. You know, when I'm trying to do the validation of the results, now what's happening is if something

goes wrong, you're talking to the senior person, "Hey, how do I fix this issue?" Is that what we are doing? We're going to the senior person, fix this issue, but how does the agent do it? Now, you have an Maybe we have a junior agent and the senior agent. Yeah, I don't know how that that'll work out, but but now what happened? The real orders are there.

Now, when you do a retry, there is a network hookup is there. Now, initial order request has gone has gone through multiple layers before the actual order is created. And that is something called as agent has created four orders. Now, why would agent create a four order? Because because $100 charged in order API is there and which is getting created. Now, you are using PayPal, yeah? If

If PayPal have the same problem, how do you fix this problem? I mean, how do you fix this problem? The person is calling two times. Idempotency key, yeah. So, we need to define an idempotency key to really solve this problem. That means your API need to support idempotency key. You know, every Every time somebody's making a call, so if there is a retry done, if I retry

it another time, the If the same idempotency key is returned, that means the same order coming to me. That's the perspective which is there. So, mysterious retries, duplicate transactions, and these things we have to fix, you know, deployment freezes are there. So, Stripe does the same thing. So, we can take a Stripe example here. So, we which does the same thing which which because nobody likes to

be charged twice. We have to make sure that we have a way of controlling that. So, what are that means? Now, JSON format is loosely coupled. Now, you can have polymorphic responses. The same API based on the state is returning something else. Now, those were fine before. Not anymore. You know, invisible versioning. The new version has come up. That was okay before because I was talking to

the actual individual, but not anymore from my perspective. So, what we need to do is we need to move from one one prompt is actually 100 calls behind the scene. Whenever I'm making a API AI call has been made, it is touching multiple services behind the scene to get the answer, okay? It's orchestrating behind the scene to make the answer. So, what are that means? That AI

agent is now making a call to the refund service. First making to the order service is creating the order. Now, your order is in process right now, so that's what this is saying, but I want a refund. If I want a refund, then I need to validate if this is not a fraud, and then be able to orchestrate the whole thing. So, this is whole thing is

done through AI agents perspective. So, how can I make my API really good? Take care of all these three uh problems which are coming in. So, we're going to talk about all three uh in our in our loop. So, what are that Agent is trying to do meaningful work. Like, you know, this is the agent um employment is done. Now, whenever there is a problem there, you

know, we need to make sure that that that problem is resolved before it comes. That's where the LLM optimized APIs created, okay? So, your API needs to be LLM optimized, that's number one. Number two is Adam potency, we talked on this. And reliability improves with that. Number three is Number three is to make sure that the API life cycle is managed, you know? So, implement machine readable

API life cycle management. If you do this step, these three steps if you do, you'll be in a good shape. Now, that's where I created machine. Machine is a framework which I created from the systematic framework to repeatable readiness. If you follow this machine framework, you'll be good to go. That's the That's the perspective. So, machine is model the contracts. That's the number one. I should be

able to model the contract, automate the discoverability, and constrain the behavior, okay? And hardening is needed in our solution. Observe what's going on, navigate, and evolve. That's called machine framework. We're going to talk about this machine framework in next, you know, 25 minutes or so, 30 minutes, okay? So, machine means like AI when I'm trying to drive human human first API I want to make it in

machine first API. How can I have AI-driven automation from machine first API done? So, that's what we'll be discussing as part of today. whenever a human used to come in, they used to negotiate, manual challenge was done. Now, it's all done through machine. So, that's that's what discoverable automatic discovery of any new API change through machine has to come through. That's what we'll be looking at now

as part of our puzzle. So, there are these are the thing which we'll be looking at. Number one is open API 3.1. So, there there is a standard open API 3.1. If I implement that standard, machine will be able to read it. Number two, it item potency key. Number three is RSC 9457. This is the structured error processing. And number four is sunset headers. Number five is

OAS top 10. If you if you do these things these five you are able to create the APIs really well, okay? We'll go through each one of them in detail, you know, how to do that. But, if you implement this the API standards, you're good to go. Let's take a look at the manual contract and the and then the model contract, how we define this contract and

so uh solution that. So, this is where open API 3.1 JSON schema comes in play. Open API JSON schema states, let's go to that Uh let me go here. So, open API Open API 3.1. So, what API Open API 3.1 does? So, Open API 3.1 is coming up with with the specification, and it's actually September 2025. It it actually released 3.2 is also released here. Now, why

they are coming up with this is because the machine should be able to read all the data which is coming through. That means if I want to support new query methods, you know, so get, post, delete, options. So, these things are possible. So, machines are able to now work through and create it. So, what does that means? You have a field there. Okay, that field is social

security number. Everybody know about that, yeah? Social security number or credit card number, yeah? Somebody types in the credit card number. What do I do in the APIs? What do we do in the API? We validate that is correct or not. That is okay. We can do that, but is that the only approach? No. In that case, if I put it in the machine open API 3.1

specification, I have to write zero code and it'll be done for me. Including the, you know, regular expression. For email, there is a regular expression for that. What's the maximum value? What's the minimum value? If you do not do that, what happens? You are vulnerable for what? SQL injection attacks, you know, and they they could be other things which can be done to make sure that API

is not working after some time. So, that's what you gain when you are when when you kind of working towards this this particular principle. Okay? So, I suggest look into this and kind of work through that as as part of the the work, you know? Now, if you need to specify, you know, a state management, you know, state management should be through enums. A code like, you

know, I talked about SDR agent. In SDR agent, I have different different states. I Initially, I reached out to the user. First time I reached out to the user and I sent the email out to them. Okay, they did not respond. Oops, what do I should I do? I'm going to call it the next time my state change. Second time I'm making a call. How many nudges

can I have? Maximum four nudges. I'm making up like you know, four or five nudges. So, what are these? These are states. We I'm going to go through certain states and you need to employ that as part of enums, you know? Uh so, and then and then make sure the response is very predictable. Exact same response coming back. If I don't have that, you know, I don't

exactly know how uh you know, the data is really really coming for me. So, what all that means? Whenever before I do it like you know, the post order is created. If you have a order API and if it's not supporting item potency, it's a Because earlier what we used to do, tell me what we used to do earlier. You go to a website and you say

buy or check out. What have What used to happen? What happened now also? Most of the uh UI is like this. Do you see that do not click the back button? You see that? Yeah? Yeah, don't refresh. You know, don't do anything. Why are they doing that? >> It will get a lot >> A lot will not will lose the state that >> It's not just not

lose the state like because there's a network behind the scene which is running, there's a possibility their API does not support item potency. You see, that's the main reason. Don't click the back button, you can say to a How can you say the same thing to to a No, not to a website, you to agent. Agent is not human. It doesn't understand don't click the back button.

You know what they're going to do? Agent is going to say that okay, I'm going to retry. I don't care. That is what we are able to achieve by just you know, providing uh providing our support to to solve the solve the puzzle for us. So, we need to look at the resolve resolve the contract schema, make sure that the order ID is correct and it's it's

a order ID is correct and it's able to look at this the data which is coming to us and then and then implement that particular solution. So what are that means after post machine first order is created. Now what I'm doing in this case is that let me see if Oh yeah, this is the Okay, this does not work. Okay, the client is making a call to

the database. Now first time I create the order, insert the order. Now at that point of time, I get the order key for with me. Now next time I make the same call even if that order is already there in the database, I will not create another order. This is called as like you know idempotency key along with certain duration of time, 30 minutes. So it's not

there for all the time. Just like queues, like you know first in first out queue, you are getting the same value again, don't process it. That's the same thing we need to do for our case. And that kind of builds the autonomy like where the customer is now making a call multiple times, it should just work without any problem. Okay? So that's where the agent friendly information

is coming in. So what are we trying to do? Operation ID. Now somebody is create new customer order for the item. Now this is the one successful order and one out of stock failure. So they So you can have multiple states created in the enum. You can either have auto stock coming in. If you have auto stock there, then you need a compensation pattern to support that.

So So should you should you charge the money if somebody ask for an order? Or or wait for Should you charge for money or not charge for money? >> I can't tell. >> You're saying don't don't charge for money. Okay. So what how does Amazon do amazon.com Flipkart do it. You know the first thing they do? Thank you for your order. >> I'll charge your credit card

first. I don't care whether I have it or not. And that's it. So that's what you need to do. You need to provide compensation APIs if something goes wrong. What is a compensation API? Anyone? One is you give back the order. You say, "Sorry, I can't do that." Another is huh?" Alternative choices means that like, "Hey, it will arrive in 3 days or 4 days from now.

Do you want to cancel it still?" That's another thing. Or alternative like, "Hey, another order is available for you you can do you want to get that?" And that is the compensation pattern. Actually, that works better than you know actually implementing the solution. So, that's what we do. Contract drift prevention happens. So, now what happens? We are trying to do CICD. Now, proactive contract identity. Now, here

let me ask you a question. You product manager you created the API. Finally, you created the API. Shoo, you put it in the production. What product manager say I got new requirements, yes? You I have a new another requirement. You need to solve this requirement. We were like, "Wait, I just put it in production. You want me to put another requirement now?" We all face this problem.

So, what happened? The contract is what? Drifting while it is in production. How many here are scared to make a change? Who's calling me and whether it really works or you know it's it's it's a big problem which comes in. And that is where like you know we're going to talk about more like you know API discovery is very important. API should be discoverable when I'm working

on it. Now, poorly designed API to actual API coming in, it has to be discoverable while working on the solution. That means that what things you need to do? You need to provide these things. Number one, operation ID. You should always have an operation ID. Now, operation is create order. Now, I know that like you know whenever we are working on it, we create post method, yeah?

Post method is the way to do that. Is post method machine readable? It is to a certain extent, but if I say create order, I know I'm creating an If I'm just having a post, then you have to see the entity and then find this information. So, operation ID is important. Adding the tags, like you know, say to categorize the operation, each is discoverable. This is orders

and checkout. I'm doing a checkout right now. So, adding tags also help understand, "Hey, what what are you really talking about?" Number three is write the summary. Summary means you have an action, what is this action for? Now, LLM can understand what's going on, like, "Oh, this is the action I'm trying to do here." Because if I go in right now to ChatGPT, ChatGPT goes in and

calls all these APIs behind the scene and get me the answer. If the APIs are written in the open specification format, it will discover it. Otherwise, it won't do that. So, that's the problem which is there which we need to solve for. And describe the operation and possible examples. Now, this is another one. If you have a failure, what do you do? Out of out of out

of failure exception is there. So, if you do these three things, four things, you are in a good shape. So, anti-patterns. So, what are the anti-patterns in this one? Now, operation ID values is post V2 service X. not really a good way of doing that. So, this the problem is that, you know, internal routing keys not a good idea to do. Have a operation ID, proper operation

ID should be there when you're trying to create that. Missing or copy-paste, like you know, you're trying to do copy-paste the summary, like control C, control V, we do that a lot. Not a good idea um in this case. No examples are provided. See, this is the output coming. What is the example for that output? I don't know that. You know, that's another problem which is there.

One endpoint does many things. That's another problem. You have one endpoint, that endpoint sometimes get you some other information, sometimes some other one. That also is not a good thing to do. And uh you know, behavior changes based on on flags. Okay, how many here have like a get method and that get method increment a counter how many time the customer calls us? How many here have

that? Some of us do that like there is a get method is there and we are changing some state behind the scene. That is also not a good thing to do. So, what do we need to do now? Let's take a look at the constraint the behavior. Now, we need to understand the API behavior, implement Adam potency and you know, ensure determinism along with this. So, let's

talk about how do we do this thing, you know? So, error handling, how do I do the error handling in this case? So, Adam potency key, I think we already talked on this one, but now the problem is problem is problem comes in is that if I do not provide here, you know, you know, you know, error clear error message which is coming to me. So, that's

something we need to kind of avoid building the solution. So, check out double double check out charge. I think we already talked about Adam potency keys for this one. So, I'm going to skip this one. So, so that's something we need to avoid and of course in that case, you know, PayPal. So, PayPal can also be used. So, PayPal has got like, you know, the perspective here.

So, you can actually see how the PayPal is doing it. PayPal APIs are which you can kind of build for your solution. Now, what it provides is like, you know, whenever I make a it provides me Adam potency key. So, next time I'm trying to make a order I should be able to do it without any problem. So, that's the reason the PayPal is good to good

to look at as a Adam potency key perspective. Next is, you know, every response has the same value coming back to me, not a different value. So, that is the JSON should be predictable JSON should be Now, error codes 409 is conflict, You know, you know, concurrent modification. You know, item potency conflict is there. So, two people modified the same thing. So, that's 409. Unprocessed entities, you

know, validation business rule failure is there. So, you need to provide specific error code to really solve for, you know, any of these problems. So, semantic needs to be maintained also. Now, another thing here is that, you know, error handling. Now, for error handling, there is there is there is RFC 9457 is there. So, you cannot have free form text error message. That's a bad idea to

do, you know. You need to have a code error code needs to be there. You need to have error defined for that processing to be done. So, let's take a look at RFC 9457. This is for HTTP APIs, like which are which are coming in. So, once you apply this particular principle, for example, detailed JSON object can be there. You you know, you do not have enough

credit, for example. So, you're saying 403, why is more important, huh? No? No access because you don't have enough credit. Which is different. Like if 403 no access is fine, but if I don't bro Oh, I don't have credit. Your current balance is this, but your cost should cost is 50, which is the different than saying that hey, you don't have access to it. So, so we

should be able to define the proper error message, you know, when we are trying to must be positive integer, must be green, red, or blue. So, we are trying to say what should be the actual value for that validation coming in. You know, and this is this is also, you know, 422 422 which is there in this case. So, I would say that if I follow this

principle, like, you know, out of stock out of stock is there. Status, title, and try to implement that solution. Now you have a machine be able to read this solution. Everybody with me? Machine can read this solution and understand what the error code is and either retry or tell that hey, you don't have balance, you know? How chat GPT will know these things? They will not know

till the time we are able to do that. So, anti-pattern is I'm having a get method that logs the visit, number of people who are visiting that. Not a good idea. I talked about this. So, don't have number of visit in the get method. That's That's the wrong approach to do Now, there is a hidden side effect is there is triggered whenever whenever retry is done, unintended

state is created. Retry should be exactly the If somebody retry the same Now, if you go back to AWS, Amazon, any server you're trying to run on, if you say start the same server again, well, you know what happens? It does not start another server. It is item potent. Every command is item potent there. So, you have to make sure all the APIs are item potent. You

know, misleading error messages. Now, now if you're having error error returned with HTTP 200, now agent misinterprets the failure as success. That's a problem like because you're returning 200 message. That's also not good in this case. Another one is the response includes hidden flags. Now, if within that you have a hidden flag is there Now, because it's hidden flag, agent don't know how to parse So, if

I just take care of these elements of the practices, we should be in a good shape. Now, you may ask a question, Rohit, what else can I do to protect myself? So, there is something you know, APIsecurity.io. What is it called? APIsecurity.io. So, take a look at that. It actually talks about all the new entries which are coming in in from the API perspective. Unsafe consumption of

APIs. You know, you know, include the injection which just came up in 20 2019. So, these are all the APIs you know, perspective, you know, OWASP top 10 API vulnerabilities. Okay, from the API perspective which is there. So, API API top 10 It's not security top 10. It's only specific to the APIs which are which are which we are talking here um are coming in. Broken object

level authentication, you know, that's that's still still still one of them. That an agent has only access to do these things. Some API is there. It It needs to know who the person authorization is there and only provide that. No impersonation is needed. You cannot impersonate that that value. OAuth is a is a great great great way to do that. So, I'm not here for this this

here. I'm here for something else, the tools. So, if you go to API security.io, this is the tool available for both Visual Studio Code, IntelliJ, as well as uh you know, Eclipse. Now, what it does and I'll I'll I'll just take a look at this tool here. So, what it does is that now you can actually apply this tool and you can actually scan, you know, uh

scan the code which is coming in. So, so you can actually scan uh scan and audit your API. So, you can actually provide an audit so for for this for this particular API. So, that's the advantage which we get from this uh when we are when we are working on. So, what does that means? It means that when I'm working on this Take a Let's take a

look at this for a second. Okay. So, what I can do is that I can look into the um Visual Studio Code and then find out from this you know, for a given field for a given field See this here. for a given field whether there is a security vulnerability or not. So, you don't do anything. Just put this in your Swagger documentation, in your OpenAPI documentation.

You will be able to find out the schema is no pattern is defined. security is not properly maintained. Now, for each one of them, they will suggest the example where the vulnerability will take place. And based on that vulnerability for that particular schema, now I want to put the max length for it with a pattern attached to it. You see what I'm saying? Small thing but bigger

impact with zero code written. So, this is just the action you can take right now. Take this to the next level and you know, you can start using it. Very simple. I'm not doing anything, but I'm able to now able to create integration and security is added when I'm kind of working on this particular Simple fix for a big problem which we all have faced. Now, this

is actually an um, you know, audit audit is there which you can kind of start using when you're working on on this particular solution. Right? So, simple simple and easy thing to do to implement this. Now, we talked about a lot of these things. Now, the next thing is harden harden the safety. So, out of this we already discussed model, automate, constrain. Now, I'm in the harden

phase. Let's take a look at the How do we harden for safety? What do I do for harden for safety? Normal use is well-behaved agent. Constrain constrain behavior so is there. Misuse happened. Misuse of the prompt happened. Prompt injection is happening. What do I need to do to protect myself? That's where the hardening really comes in play. So, what happens is that you know, airport security. So,

airport has a security, What what does the airport do? It look at anything which is coming in and then try to say that, "Hey, if something comes in if it is not a good thing, if you're bringing explosive Oh, okay. Let's say if you're bringing liquid with you, they say no, it's not allowed. Same thing is here. You have a rate limit, you have quotas there. That

means you she was talking about one use case where like, you know, hey, I did the rate limit, that doesn't work for me. No, rate limit is not going to work for you because you need to also do shuffle sharding, for example. Shuffle sharding means that you need to find out if you have a VIP customer, give them VIP treatment. If you have a regular customer, give

them a regular treatment. And then also look at the history of the user, how they have used your system. I'll give you an example on this one. Um it might help you understand what's going on. So, I had one customer like we'll name that customer. They were doing the charges, you know, and these were credit card charges coming in. And you know what the charge was for?

Um it was a big equipment was there, a million dollars. 950,000 dollars was like, you know, was coming in. So, and then we said not one, they were like multiple of them were there like, you know, every 1 minute one credit card was charged. Then we realized somebody was just testing whether this credit card works or not because you're getting here sorry, that you don't have balance

or something. So, so you it was just kind of doing that. So, we had to come up some other mechanism to fix that because the bot was doing it. So, we put a CAPTCHA in between. See, CAPTCHA can be used to really do that. But how do you put a CAPTCHA for an agent? So, that's the challenge. I don't think there is any CAPTCHA for it. If

there is any, let me know because I have to put it. Huh? >> Agent can CAPTCHA defeats the purpose. So, agent can't have a CAPTCHA. >> Can't have a CAPTCHA. >> He can't have a CAPTCHA. So, but but if I need to challenge the agent, hey, are you the right agent or not? How do I do >> I'm telling I fixed it for human, but can I

fix it for the agent? I don't know. Like, you know, that's where the So, mal So, hallucination call has been made. If a hallucination call has been made, I need to now contain that blast radius. Contain the blast radius means, you know what? If you If you're on the security line, and they find something suspicious on you. You know, it happened with me. I I I booked

the flight and shouldn't say that. I booked a flight to uh to to India from US, and I booked that on the same day, 2 hour before. So, and I got a real frisking done for But, it happens in like, you know, What What do they do? They take you to the corner. Come here. I I'll love you, and then let me work with you, and then

they do the full frisking, okay? Same thing you need to Find out those VIP customers, like the bad ones, and take them to the corner. What do you mean by take them to the corner? >> Special scanning with >> Yeah, special scanning. But, what what is that special scanning means? >> That means they're saying so, you know, checks has to be done. >> No, you create a

separate pod for them. >> Straight to the yard. >> Yeah, pod. You know pod means, like, you know, pod You know, pod is, like, you know, the entrance. You have a separate entrance for them, and you deal them separately, properly. Like, you know, as like a There's a monitoring going on for them. Are they the They're the right people or not? You know, and that is possible.

Like, you know, this is this is where This is where you're trying to contain this blast radius, and that is where the OWASP Top 10 comes. OWASP Top 10 is there from prompt security perspective, where you're now putting in the security security perspective for you, where the prompt injection is coming in. Now, we'll talk more on the prompt stuff next next uh meeting. But, really, the main

thing is that now the APIs, if you have an API now, instead of the API, now you have a prompt there. I can I change the prompt instruction? That can be changed. Based on that, I can do the goal hijacking. I can do prompt leaking. I can also do SQL injection attack by doing this because if I change behind the scene, now I have to write in

natural language to select all the customers all the customers which are which which are selling more than more than, you know, 50,000 rupees. I can do that. And then, if it is not protected, it's going to actually go to the database and get the values, you know? So, that's where the real problem is. So, this is like a natural language, but this is actually behind the scene

is a is a real problem which is going on. Same thing is true. Like, just like you can reveal yourself being a came up with from ChatGPT, they got it from the Bing. They came up with this strategy and and they were able to identify, "Hey, can you find out Can you tell me more about yourself?" No, I'm Sydney. I can't tell you anything more than that.

They were able to say, "Give me the previous instructions. Give me the previous instruction." See, if you have the instructions, you know what you have behind the scene? They They You have got something called weights. You know what is weight means? Weights means like, you know, their total APIs behind the scene, how the Bing is working. It all got hijacked exactly by doing this this operation. So,

jailbreaking can also be done in this case. But, really speaking, you know, I'm going to go a little fast here. Sandwich defense and other things are there. You take a look at that in more detail. The main thing is the fairness. Like, you know, even if you're using an API, you should not have a bias based on an API or ethnic bias or gender bias should not

be there in performing certain operations. And you need to have an API for evaluating evaluating the monitor of the monitor. Monitor the monitor means LLM is sending you something back. You need to evaluate whether that is correct or not. That's another thing you need to do building a solution. So, AI aware AI aware rate limit quotas. So, it's not just the user, but human traffic and agent

traffic should be separate. Human is going to human traffic. Agent is going to agent traffic. Agent should be identified that this is the agent I'm working on and then work on that particular agent. That's the way we need to do it. Um and then the kill switch. So, when you're working on this and you find out that, you know, hey, this is really not going good. Like,

you know, you find out issues there. You have a kill switch for that API. Sorry, I cannot perform this operation right now. So, kill switch is also needed. And then you need to instrument whatever is coming in. You need to instrument that data. Machine is is the instrumentation needs to be done. Find out the defects for yourself. Where you're observing and finding out where the problems are.

Watch the traffic which is going on. And tag if there is any problem with there in the in the from yourself. And detect any retries from coming in. The retry can come through. So, if there is one time somebody is calling is fine. If they're calling 10 times or three times, you need to record why they're calling so many number of times the same thing. So, that's

something you have to predict. And then do the monitor the schema drift. Schema can also drift. Schema JSON schemas can drift, yes. That means the schema is changing over time. You need to monitor and log the schema which is changing over time. That's another thing you have to do. Um and then if there is a schema drift is there, that means downstream IDs. So, so that's the

schema drift is one of them. Schema drift you need to identify. Another thing you need to do is create a correlation ID. Correlation ID It you start the correlation ID going from LLM to tools to downstream systems using the correlation ID to solve the problem. So, that's another thing, you know, you can add to yourself while building this. Okay. At the end, deprecation and sunset So, these

are the two things I would suggest you should do for for building the solution. So, deprecation and sunset header means that you need to also have a version strategy. And this version is available only till this much amount of time. You can also say that where where you're trying to say in this case, deprecation sunset header is RFC 8594, which states that, you know, hey, this sunset

will be there on Wednesday, 30th April, 2026. >> [snorts] >> And before that, end of life is there. That is RFC 9 8594. So, that means the agents are given time to really react towards it. Okay? And then you do the last part you're doing is the versioning strategy. At the end, you're building in the versioning when you when you when when you when you're providing the

solution for yourself. So, that's another thing you can do. So, what are the missing mitigation time frame for you? The mitigation time frame for you is 90-day, 60-day, and 30-day model you should apply to really apply the sunset for any any data which is coming in. So, make sure that the change the field of the schema without versioning, you know, which is coming in. So, you want

to make sure that versioning is done. You should not hurt the agents. Never sunset old versions right away. So, those things also, you know, we need to apply when we're working on this. Okay? And at the end, you know, have a continuous readiness life cycle should be there when you're working on it as a solution. Okay? All right. So, that said, let me go through homework for

you because homework is important. So, let me go through the homework for today because hey, without homework nothing will work, yeah? So, I have a scorecard, okay? And by the way, if anybody wants to know exactly what the ID what what am I talking about is this URL. This is the tiny URL you need to have access to AI API ready. AI API ready is the one,

okay? So, I'm going to go through the checklist because that's something important for you to know. So, model you need to support what? Open API 3.1. Second, strict JSON schema typing needs to be done. No polymorphism, one to many relation should not be there. Canonical response envelope should be and for everything and error modeling should be done. And there is a scoring is there for each one

of them. If you are doing observability, in that case, you need to provide the operation ID, add metadata, and schema should be clear. And look into the scorecard for yourself and see where you are. For example, if you are don't have Adam potency keys, apply that. If you don't have deterministic key, apply that principle. If you have if you are not concurrency save, make it concurrency save

for yourself. Hardening is another one where you need to do schema validation done. And these are by the way the scores. Are you ready for rate limit aware? Are you ready for high skill prompt injection attack which is going on? So, just by this, you will be able to understand what things you can apply from observability perspective. Agent tagging needs to be done, we talked on this

one. Schema drift, you need to apply the principles for that. And same thing is to be the navigate the versions which are coming in. Explicit versioning needs to be done. And sunset header needs to be done also. And successor links, if you have a successor, you should have a link for the successor. If that API is not working, go there and then apply the evolve. That means

now you have five space mitigation strategy with backward compatible testing needs to be done. You apply this score for yourself and now you know what you need to do to apply this thing. I have also added here for North Star for Amazon. If you are if you actually build this properly, how to implement this solution for all these methods. So take a look at this and on

your own time. This will get you actionable steps So I hope you enjoy this session. I know it's a it's a it's a quick 1 hour, but this is all action. You can take it right now for your project and start to implement it. Thanks all for joining in. >> [music]