Dynamic Mocking for Event-Driven APIs: A Cloud Native Approa... Harshvardhan Parmar & Anushka Saxena

This talk explores the implementation of dynamic mocking techniques for event-driven APIs using the cloud-native tool Micros. The speaker explains the challenges associated with static mocking in event-driven systems, highlighting the complex nature of distributed architectures compared to REST APIs. To tackle these challenges, the session introduces the usage of AsyncAPI specifications that allow for the dynamic generation of mock data that mimics real-time system behavior. The speaker provides a demo showing how Micros integrates with Kubernetes, streamlining the process of generating live mocks based on defined specifications. The goal is to enhance testing capabilities for cloud-native workloads while maintaining fluid interactions between producers and consumers in the API ecosystem.

Hello everyone. Um, a very good afternoon to all of you. I hope you all had a good lunch and um, I suppose no one's going to sleep midway. Even if you do, that's fine. I'm not going to tell anyone. Um, so today I'm here to talk about how you can do dynamic mocking for event-driven APIs using our own very cloudnative um tool called micros micro and um

so this talk was supposed to be with my co-speaker Harshwadin but unfortunately he couldn't um be here in person due to some visa issues and stuff um but so it's just me today but Let's get going. Uh, I actually have something from Harshwaden. Hey everyone, Hashwad on this side. I hope you guys are enjoying CubeCon and I'm already missing it too much. So that's why I'm wearing

my last CubeCon t-shirt. Uh thank you so much for coming out to listen our talk and hopefully you guys will like it and please feel free to share your thoughts uh with me with Anushka and with the Microsoft community. I hope we'll see you soon again. >> All right. So um we definitely missing him but the show must go on. So um Harward is a um am

I audible with this? Okay. So um Harshwaran is a maintainer at the micro um project and um he uh he was about to give a demo um about a few things that we'll be discussing um in a few minutes um and a little bit about me. I am Anushka Sakina. I am a software application developer uh developer apprentice at Google. And uh just to let you know

that uh this is nothing related to what I do at my job or um um my employer has to do nothing with this. So um just a disclaimer that we have to give all the time and I have been um a mentee at the async API project um which uh which helps us to to provide the API spec for event-driven APIs and I have been a contributor

to micro as well and uh I've done latest um my LFX mentorship with the cloud native PG projects. So let's get going. All right. Um, so we see messages everywhere. How many of you use uh event-driven architecture with your um uh production systems? Can I just have a raise? My god, that's a lot. Okay, so that that pretty much explains why we are here. So um how

how often do you think that um um it's uh how easy it is for you? Um okay, uh let me put it in a different way. Do you think it's very easy for you to to mock your uh um event-driven APIs um and like just just test them? Is it that quick and easy? Do you think anyone? And uh if you have a solution to this problem,

uh I'd love to discuss more about it later on. But yeah, so um we see where the problem is, right? Um you you see rest it it's just simple. is just straightforward. You do what you see. But with event driven, it's all different. We we have got a lot of different complex things to handle all together at once. So this leads to the rise of event-driven API.

We see distributed systems everywhere like everywhere. Um so why are we shifting? Let's say um we we think of rest as a vending machine. You you want a soda. It it just gives you it's it's simple. It's straightforward like that. But but um with event-driven systems we are building something like a WhatsApp group chat because here multiple services are trying to type send uh messages via the

Kafka and MQTT all at once which which gets clumsy at times. Let's accept the fact. So um since all of these um messaging stuff is uh is the new backbone of our microservices uh and our ribbon driven architectures we have to have something to test it right because I mean we can always test on the broad uh but but that's not what we are here to discuss

today um so uh to find a way that is suitable for everyone and makes you um test your cloud native workloads easily that are running on event driven uh event- driven systems. We've got something for you. Oh, all right. Yeah, I uh so yeah um I I I saw uh no raise of hand when I said is it easy to test your um your event driven architectures

in uh before you you roll them to the prod. Um would anyone like to share your story or or just something you broke or something you tried that worked out? It's all right. It's all right. Uh okay. So um let's discuss the challenge a bit more. Um why why is it challenging? Can't we just like copy paste a JSON response like we do for REST? We just

go to our open API or or swagger or Postman whatever. We just put our response what we're expecting and it's it's done. It's that easy to test. Why can't we do that here? Because here whatever payloads we're getting they're timed. they might they may get stale and you cannot just refresh your uh messaging queue altogether anytime right um it's it's not practical it's not possible and also

um I would say that if we are trying to mock a system that is event driven it it feels like uh let's say I my mock is sending um order made at 12 and it will keep sending it repeatedly. What would my downstream service would like think in such a situation? It will think that my data has got stale and this may lead to other problems and

stuff like that. So, so this this kind of feels like um you're just practicing to play tennis against a wall because it will never return the ball in a different direction. it will just do uh repeat the same thing again and again which is not what we see in the real-time systems do we none of us do right it's very unpredict it it's almost unpredictable we we

cannot see what's so we see the problem static mocks cannot replicate the real-time behavior for um solving the problem of the um the static mocks. We have come up with something called the dynamic mocks. And uh for the normal API specs, we we we can have a defined schema. We can as we discussed earlier, we can have just a simple JSON schema and that works out. But

in the case of event-driven APIs, we need something more. we need a better schema uh an enhanced one where we can see how how we're expecting the the the messages and the payloads the the structure has to be defined. So for that we have async API. If you know open API you you pretty much know it already. Um so async API provides you the API spec for

your event ribbon APIs and uh these API specs are something that are going to be very useful to us. Okay, so we are going to go beyond the payloads and we are going to simulate the realtime behavior of our system using the dynamic mocks and we will reproduce the producer and the consumer interaction how the stateful message flow happens inside the system and then enable the integration

testing um before we roll it out to prod. But as I said, you can always test on pro. I think we've pretty much discussed the slides. Yeah. Okay. So, here comes our savior. Introducing to you Microox, which is your own Kubernetative the the the cloudnative tool that sits on your Kubernetes cluster. It's a very lightweight solution that you don't have to uh repeatedly spin up or do

stuff uh that will um make your testing experience problematic. So how do we do this? So you see as we discussed that these won't be any random examples of um of your messages that will be in your um in your system. it will always have some executable specs that um that you can have in your mocks and whenever you try to uh test your APIs uh test

your they'll just magically replicate um the different situation that you might have and at Micros we support a lot of different um a uh a lot of different protocols as you can see here so you don't have to worry about um which one you're using. We have a solution for all of them. And so yeah, as I said that um Microsoft is u is kind of it

it it will have your Kubernetes native um feeling and and that that flavor of uh cloud native setup that you can do. It will spin up lightweight containers and uh won't won't cause any issues to you um while you try to mock um mock your um I'm sorry the event driven APIs. Okay. Uh so how do we do this? We've got a lot of things in house

a lot of tech jarens that uh we are yet to know about. So let's get on to our demo. >> So like as you guys already got the gist about today's topic. So I am not going to waste too much time. Let's directly jump onto the today's demo. So uh for today's demo I have created one uh sync API file which is a spec file where I

have mentioned some of the ordering events. This is a uh like basically order uh based uh event driven API where I have some order creation event then payment processing event then >> so so we can assume the the case that we were considering earlier let's say uh we're creating something uh we we are replicating an e-commerce setup and here we have our API >> checking event and

order shipping event and I have added some of the examples for there as well so I have created the examples for each of the events Uh so now let's see uh like how Microsoft can convert this thing this file into the mock. So uh for this uh I have created one uh like I have I'm running Kubernetes using kind and uh I have installed the micros uh

on the on the cubernetes. So if I go cubectl get pods and in the micro namespace I have this uh this pods running for those uh micros. So we have I have the main pod for the micro then the singing minion which is basically a minion for the singing in the micros and then I have some kafka pods and supporting pods for kafka as well and then

key clock mongod is something like this. Uh also uh this all things is sitting behind the ingress. So we have to get the uh ingress URL as well. So like uh basically using using those we can like uh access this uh this things on local. So here are the here is the URL mostly for the micro and u micro cpc for the key clock and the websocket.

So let's go to the m let's go ahead and explore the micro instance first. So uh yeah see here the micro instance is running uh uh it's it's the web interface of the micro. So if I go to this uh APIs and services here basically we can import the uh API spec uh like whether it's uh like open API or the sync API anyone. So uh as

for the demo like we are importing the sync API file which I've shown earlier. So let me go to the upload. Let me select the file. Uh let me go to the sync API uh 1.0.1 one I'm uploading it and see. >> So you can see here that we just took our API spec the async APIL file which had all of our schemas all of uh all

of how the structure should look like and how Micros is making it so easy for us to do the specdriven development. it's uh added into the macros and created as a service. Now if I go over here see every event has uh two or one examples and if I if I see this is created as a mock like living mocks from directly from your async APS pack.

So this is how micro generating your uh live mocks from the uh APS specs. uh if you can see here all the uh event uh or the kafka topics so which you basically your consumer can consume. So let's try to consume one of them. So I'm uh I'm going to consume this uh the send order create created uh endpoint uh this event. So let me copy it.

I copied to clipboard. Let me go to the console. Uh let me create the cat and I'm using this see the micros is emitting those events from the uh askp spec which we have created and now it's behaving like a mock over there. uh but now you can see this uh this values from the this event is coming like uh hardcoded values like as we mentioned in

the API spec over here it's coming like this so it's like same values which we have mentioned over here but it's not a dynamic yet so we have to make it dynamic so now let's go to the next uh the upgraded version of this SNKspec here in 1.0.2 two we have this uh this async API uh with the dynamic mocking support. So now I can I I'll

explain like how this things is happening. So in Macbooks we have one term called mock templates where basically is a kind of function which basically tries to uh get the values get the random or the generate the random values and for that let's see here this is like mostly the mock templates are defined and like between the two curly braces. So here you can see there's two

curly and inside this there is a function called random it and then we have the values like from where to where like from which uh which basically a range like from where to where we have to generate this random value and using this we are uh basically dynamic uh dynamically generating the value of this payloads order id field and this is how we are like I have

created all the mock templates for the all the uh events which are present over here and uh see you can directly uh do that. So now let's try to import this uh this uh this pack file and then try to consume it from the Kafka consumer. So we can see how the dynamically mocks are getting generated using this mock templates. So let me go to the micro

Uh let me go to the spec service again. Uh upload uh 1.0.2 uploading it. Yeah. Uh currently it's due to the versioning like uh this all this both are getting separated but like if we have the same version this auto like both the uh like this the second AP spec works as a uh secondary file and then like it basically just merge the examples from the second

file. But now let's go to over here see we have some of the examples. We have only one examples because we have mocked it or sorry we have dynamically mocked it basically. And now if I'm going to uh consume this uh simple order created. So you can see this all are the randomly generated. And now if I'm copying this and going to the going to the my

console and try to consume it. Now let me clear it first. Okay. Get and now I'm copying the new event which I'm going to consume. So see now it's getting generated dynamically. So how easy it is to like generate the mocks dynamically using your spec files with micros and that even uh sitting inside your kubernetes environment. So this is why like micro uh people are using micro

and like uh preferring micro because it's uh totally combines with your cloud native environment and makes uh APIs or the specdriven u uh development much easier I hope you like the demo. Uh like there are many more things to talk about but uh I'm keeping it uh very short just because like I'm not able to present it uh physically and I don't want to like uh bore

you guys with this uh this uh remote demo. So yeah, thank you so much and I hope you like the the demo and please feel free to share the the feedbacks which you have and yeah my postpand is there so feel free to reach out to her and u like you will got the you you'll got from of me from her and try to reach out to

me as well. Yeah and keep starring Microsoft as well. Thank you so much. So we see how easily we were able to just just just have your dynamic mocks ready within within a few clicks and and that just goes on like that. So um if you have any kind of doubts related to what we just discussed, we have Yasim, we have Lauren from our community. You can

please feel free to reach out to them. Have any doubts uh cleared up and we'd be happy to answer them. Uh oh. Okay, just to have a quick recap of what all we discussed. The static mocks do not replicate the real world systems. So we need a contract for which in our case we used async API and we use the behavioral and temporal mocking for realism. So

you you saw earlier how the events were being uh like they were just looking um stunnables of each other just similar ones which are not real. So we created them realistically then again um we we we use the micro um we use the micro uh tool to have our mock uh done dynamically and they sit on your cloud native environment. This micro will just spin up the

lightweight uh containers on your cloud native dev uh environment and make it easy for you um to dynamically test your uh APIs. Um if you want to uh see what Hward has used in the demo, please feel free to uh check out this GitHub repo. And there are a couple of more demos and stuff uh that we'll be sharing. I hope. And with that, um, Microsoft is

a lovely community. Uh, we are a very welcoming set of people who would love to have you as contributors, as as as adopters. And um um whichever way you would like to be involved, please feel free uh to check out the link in this QR. And um let's get connected and uh build some great stuff together. Um and please make sure to to just start our repo

um fork it, play around with it, and um help us make make our project better. Yeah, I won't do the mic drop literally. But yeah, that was it from my side. Thank you so much for uh being here and we're open to questions now.