Breaking the Monolith: Tesco’s Journey to Federated GraphQL with xAPI - Vishwas Chandrashekar

This talk by Vishwas explores the evolution of Tesco's orchestrator platform from a monolithic configuration to a fully federated system utilizing GraphQL. He begins by describing Tesco's vast retail operations and its commitment to innovation and sustainability. The speaker explains the fundamental concepts of GraphQL, including schemas, resolvers, and data sources, emphasizing their importance in creating a flexible orchestration layer. He outlines the challenges faced with the original monolith, such as codebase complexity and the need for rapid feature delivery, which led to the adoption of GraphQL Federation. Vishwas discusses the implementation of microkernel architecture and various strategies, such as modular monolith and strangler pattern, to incrementally migrate services while minimizing customer impact. He concludes with insights on improving operational autonomy, scaling, and introducing AI for process automation and monitoring.

Hello everyone. So, I'm Vishwas and today I'll be taking you through how we evolved our orchestrator platform from a monolithic setup into a fully federated system. So, let me begin with introducing Tesco to you. So, we are a massive retail organization contributing to around 28% of UK's food supply and have over 335,000 colleagues globally. And we are committed to reaching net zero carbon net zero by 2050

and we are the number one retailer in UK. We have operating at a revenue of 66 billion and we have more than 70 million shopping trips per week that we have. At Tesco, we have Tesco technology as a division. We have 4,000 technologists across different geographies. From UK, Ireland, we have Czech Republic, we have India as well. let me briefly explain about what GraphQL is. On a

show of hands, how many of you have used GraphQL either either personally or in enterprise? Okay, there are some so let me briefly explain what GraphQL is, right? So, a GraphQL is a query language a server-side runtime to execute those queries and usually comprises of uh schema a resolver and data sources. A schema is like a contract to the client. Client can pick and choose exactly what

they want to execute from this and whatever has been selected by the clients to execute, those will be having similar resolvers and they are functions that are executed. So, unlike rest, your schema is ever evolving. Which means you can keep on adding to that schema and you don't have to have versions for that. Like rest. Right? And data sources are the ones which power the resolvers. Which

means it can get your data from either files or APIs or even database. So, it could be any data source that power your resolvers. Now, this is what orchestration is and we use this as our orchestration platform. At Tesco, we've been using GraphQL from quite long time. And uh to be exact, we have started it at early 2016. And we've been using it for a decade now.

We've had many different transformations, optimizations on top of GraphQL service. And uh we are one of the early adopters of it. It actually released in 2015 and we adopted in 2016. We call our GraphQL service as experience API because we provide multiple different experience from that service. So, it's a gateway onto our retail services. So, since we are in retail domain, we have multiple different APIs that

power uh the retail services that we provide. So, all our APIs, we have X API as the gateway. It's the X experience API. Now, we provide experiences on both online and in store. And we have variety of clients, customers, retail customers. We have customer support executives, partners, vendors and many other stakeholders. All of them, if they need any information from retail, X API is the gateway to

that, entry point to it. So, if I give you an example of uh how people want to use GraphQL or with X API is some of the capabilities like on website, if you browse tesco.com/groceries, what you would see is search for a product. You can search for a product, get some details about a grocery product. You can add or remove the items from your basket. You can

identify how to fulfill these And we provide different types of fulfillments like home delivery, click and collect. Uh which is similar to you know view customer booking a slot, going to a store, and they pick and pack your items, and you just pick them up and go to your home. So that's what click and collect is from our end. And we have immediate delivery as well. Very

similar to how in India we have Blinkit and Zepto. That's another type of fulfillment that we support. And obviously we have checkout and payment as well. So these are some of the different types of capabilities that we support from X-API. what we also do is orchestrate all of this information via the retail let me speak about how we handle this at Tesco for online shopping. Right? So

last Christmas during our peak hours we had more than 1.6 million transactions per minute. And amounting to a million orders last Christmas. And it is around 8,000 orders per minute. This is the scale that we handle. And we had more than 4.4 million unique customers and ever growing as well. So this is the scale what we handle at Tesco and in turn at X-API. Now let me

dive a little bit deep deeper into how X-API looks like. So this is our monolith setup. So it's a typical graph QL system that we saw before. We have schema, we have resolvers, we have data sources as well. If you see data source a bit plump because we have used it from over a decade and all the business logic enrichment has been added on to that. So

what we did we had to split that up into two different uh layers. Resources was where our business logic lived of how we enrich information. Because we had to orchestrate between different services, we had to add all this business logic and that's where resources helped us out. And we had ski service which is basically a wrapper on top of our retail services or domains. So, what we

had to do was although monolith helped us a lot for orchestrating across different domains or different APIs, but there was one underlying problem for us because it has been there from a decade, we had a lot of business logic and code in a single code base. And also it was a one big deployment unit across different domains. Now, this was not a problem before, but as traffic

grew and also domain expertise grew, we had to look at how to improve this. That's where we went to a whiteboard, understood what our goals were, and figured out what exactly we needed and how can we support our customers better. That's where we came up with these important goals that we wanted to achieve. So, we wanted operational autonomy where to make sure that we do not have

a single point of failure. That was our first and foremost goal that we identified, and then we also wanted independent scaling so that large volume that we receive, let's say for peak uh during let's say Black Friday, we have a lot of traffic coming through on slots where customer can book their own fulfillment. At what time do they want delivery coming to their service, right? So, this

is where we get a lot of traffic during Black Friday. So, we could only scale that, but that was not a uh issue if we had independent scaling available. Now, over the time we also built up a lot of client transformations. So, we had version clients like mobile applications. So, these had their had their own transformation that was required from both request and response side. So, that

was another goal that we wanted to achieve. And we also wanted rapid feature delivery, make sure that we have reduced blast radius, so that any change to any part of the system does not bring the entire system down. And mostly, we wanted extensibility. So, right now with monolith, if you had seen, there was no opportunity for anyone to contribute additional information, which means any new feature, they

had to contribute to the same large code base. So, we cannot onboard any new teams for our features that we wanted to bring in or reuse the capability that we had. what we did was we identified what tooling or process could help us with this. So, we tried out many different solutions, and as you can see, we finally went with GraphQL Federation. We tried out GraphQL schema

stitching, we tried out GraphQL mesh, and then finally GraphQL Federation, which is just an improved way of schema stitching. Uh if you see GraphQL is a is a GraphQL architecture having multiple GraphQL services, which they call as subgraph. And each of these subgraph has schema, which is a contract, like we saw before, right? And composition of all of these schemas is what a supergraph would look like.

So, it's one big graph having multiple different subgraph within it. Right? And the place where the supergraph resides is what we call as So, gateway is what takes in request from the clients, orchestrates, identify where the request should reach for each subgraph, and then make sure it gets that information, aggregates it, sends it back to client. So, from a client standpoint, they do not understand exactly what

the underlying system do. All they care is they need to make a request to a GraphQL system, get the response back. So, this is what we wanted. Now, if I look back at our monolith architecture along with our goals, the first thing we wanted to understand how do we help our customers get more features as soon as possible. Right? And with monolith, it was very slow because

it was cross domain and very huge releases. So, we wanted to have extensibility. So, that's where we started with And we called it X API gateway, so why not? Right? So, we called it X API gateway. It had many different tooling supporting it. We had observability baked in as part of that. We had schema management baked in as well. And this gateway was responsible for orchestrating different

requests across different sub-graphs. The first thing that we did was converted our monolith into its own and had all our requests routing through That's what we did initially. Now, what this allowed us was have a platform where anybody can onboard onto. Which means, as you can see in the diagram, if we wanted to, we could onboard loyalty as an experience and have their own sub-graphs come in.

Now, this enabled us to have reusable reuse of capabilities and also extensibility of bringing in more features. Sorry about that. So, that's what helped us with X API Now, the tool that we used in this was a Polo router. So, router uh written in Rust was a very good tool that we could use for orchestration. This helped us very uh smoothly have any onboarding of new sub-graphs.

This helped and it had very low latency. It was very good for us from an extensibility standpoint. The only thing was we could there was a limitation for this. thing that we needed was also client-aware transformations, which means we had version clients, which was mobile applications, which needed backwards compatibility. So, you have, let's say, clients with Android OS 17 or Android OS 16. So, all of these

needed different changes or transformations per device. And this is something that gateway used to handle before or our old monolith. So, how do we handle this? Now, router was not that flexible. We needed to write in i-scripts, and it was not that performant when we had some additional custom transformation on top of it. So, we figured that out. And then what we did was we added in

a node proxy. And this is something that we ran in our lower environment and figured out, you know, this does not scale well. This takes in a request, intercepts the request, added any transformation, sent it to the downstream services, which is your subgraphs, and then to API. And once it gets back, apply any transformation back. So, from feature point of view, this worked really well. But the

problem was, if you already this was a problem with having synchronous requests coming through, and it does not scale well. So, we tried out to see whether this works. Does not It's not efficient. Although it helped with transformation, so then what we looked at was Apollo Gateway. And this has been there for quite some time, right? So, Apollo Gateway had started with schema stitching, but it it's

not something that we wanted. But when we were starting to look at how this helps, the Apollo team started to use the router's capabilities and added on to the gateway. And they had a preview version available. So, we started working with them, tried to see how they can improve this and get all the features that we wanted, and it worked out well. So, what we did was

we used something called as microkernel architecture. Basically, you have a core kernel or a core set of uh implementation, and then you have plugins across that. So, that's what we went ahead with. So, we used Apollo Gateway and we had plugins across that. So, one of the plugins is client transformations. So, we took in request, intercepted them, added any transformation, and then sent it back to the

clients. So, that worked well. So, another example that I could give for plugins is basically we had query allow listing. So, we wanted to make sure that clients request only exactly what they wanted and not something else. So, that's another thing that we added as plugin. So, this tool that we used for plugin was Anwala uh by Guild Systems. So, that worked really well for us as

well. So, now this provides us client over transformation. It scales well. Performance is also really good. But, there was there was a still a bigger challenge that's remaining, right? So, we had still monolith. How do we break that down? So, this is where we figured how can we migrate without affecting our customers and also have less developer toil because that's a problem. So, if you have seen

how GraphQL Federation works or how normalization works, so what happens is all the articles that you see always tells about you start with scratch from scratch, identify your domains, and split out your schema into subgraphs. That's what you always see. But, with enterprise and decade worth of it's very difficult for us to do that exactly. And our domains has not been split up that way. If you

see our data source, it was very plump, right? Very big. so we had to rely on few migration patterns. And we found out a very good complementary pattern for us to do it. And the first one was modular monolith. So, this is a pattern which helped us a lot to identify what boundaries were from a domain standpoint. So, the first thing that we did, identified our domain

like profile, basket, fulfillment. Profile being users account information or any preferences or you know, anything about the user. And then basket being whatever the user had information from a basket standpoint adding your products into basket. All of that become one domain and this is how we identify different domains. And then we structured them into modules. That's what modular monolith is. So this was our pathway onto migrating

it as its own subgraph. This is a very good approach that we had and this also meant that we could incrementally migrate stuff. And this was a very important for us because this is a tech driven initiative. We had to make sure that we bring down the cost and also churn the features, right? So this was a pathway for us to make sure we do it incrementally.

And along with that we went with strangler pattern. So you as you already as assumed what this is. So once we have the domain split out the next thing was how do you strangle them out into its own subgraph, into its own service? Right? So since it was already split out as a separate module, it was very easy for us to move it into its own subgraph

and that's what strangler pattern is. So these modules were separated out into its own service and monolith would store the rest of the schema. So this is how we work and these two patterns helped us really well. Had no issues with how we scale. It was a smooth and safe transition for us domain by domain migration path. So this is our old architecture, right? So we had

gateway, we had monolith. So what we did, we used those two complementary patterns and moved each of these subgraphs into its own domain. So they became their own subgraph. While monolith still served the rest of the schema that was there. So no downtime to customers, no problems with clients because they don't have to worry about schema changing or you know, their contract being changed. So client was

unaware of this. The only thing that we wanted to do, make sure was before we take it to production, we test it out whether it works fine. And those two patterns that we said that helped us a lot here. Great. So, now let us look at how we achieved the rest of the goals, right? So, as you can imagine, this basically helped us with operational autonomy. We

had sub graphs having their own life cycle, having their own schema, having their own KPIs. So, this worked well for us. And then rapid feature delivery. With monolith, we could not deliver as soon as possible because it's cross domain, right? So, any change had to run through all big changes with testing, with see their own CI, and we had to wait for another release to finish. So,

with this, they had their own release cycles. Same thing goes for reduced blast radius. So, we had an incident quite some time back when we were in monolith architecture where some change to discovering a product discovering is basically you search for a product, you will get to know what are the details of a grocery product, right? So, there was some issue with that and the entire system

brought down. And that's what reduced with this new architecture. So, if something happens with discovery, you still customers are able to add products to their basket, you know, go ahead with their checkout journey, and pay for their order. So, this was very good for us to continue with. And also the independent scaling as you can know, uh with the load, we can scale only basket or only

fulfillment. This helped us a lot. So, we achieved all our goals that we wanted to. But there was one additional thing that we wanted to make sure we do. Because we saw the scale at which we work, we this is a some process that we want to bring in to make sure this works smoothly without having to intervene too much. So, what we did was we came

up with a few set of processes or tools to make sure that it runs very smoothly and autonomous, right? So, the first thing that we did was whenever there's a change to schema with either sub graphs or with the super graph, which is your complete C schema that is there from a gateway standpoint, we brought in something called schema stewardship, and this is an improvement on top

of schema governance. I'm not sure if you've heard of governance. It's basically there's a group of people, SMEs for GraphQL. They observe how a schema should change, and they dictate how exactly this change might should happen. So, that's your governance. But, stewardship is where it's a contribution model where you have to collectively decide whether this change is valid, and then you pass on the knowledge to rest

of the teams. So, there's no isolated governance team. It's more of a combined effort from the SMEs as well as the team proposing the change. So, that's what we went with. We had schema stewardship where one or two members from each of these sub graph or teams came in, pitched in along with the graph product owner. They all decided how the standard should look like, conveyed it

to the rest of the team, and got everyone together whenever there is a change. So, that's what stewardship helped us with. And we also used to rotate the team members so that rest of the team also get the knowledge, unlike governance. That's not what happens with governance. The other thing was with monolith we had multiple different environments, obviously. So, we had development environment, we had pre-production, we

have production as So, how do we do this with multiple sub graphs and gateways as well? So, we had to bring in something called a schema manager, where it identifies which sub graph and which version of sub graph belongs to which environment. And how do we serve them efficiently? Right? This is serving to both sub graph as well as the gateway. So, we had to manager and

a schema server. And obviously we had to make sure that we test things properly, right? So we had different testing at different layers. So uh notable thing here is we had to bring in integration tests at different levels. So we had integration test to make sure that gateway is able to speak rightly with the sub graphs and also sub graphs have the request rightly come in, transform

it and it basically acts as a black box, right? A system in test. We also had composition test to make sure that whatever has been composed from a schema standpoint is valid for the specific client. So this is some something that we brought in as well. And the other thing was contract test and this helped a lot. Because this is an orchestration layer, the contracts between the

sub graph and the APIs need to be valid. So whenever there's a change in let's say APIs contract, REST API or any different data source, so that should also reflect back to the consumer, which is your sub graph. So we had some contract test set up there and same thing we had set it up between the gateway and sub graphs as well because these are all different

teams working on this. So we had used Conway's Law where basically we made sure that whatever architecture we went with also reflects with our organization structure as well. So we had multiple So we made sure that the testing between the gateway and sub graph is also managed, the contracts between them and the same case had been done with clients and gateway as well. with monolith it was

very simple for us. We had observability within a single system, it was very simple. But now with the distributed systems of multiple sub graphs, we had to think about a different tool. So we used something called OpenTelemetry or OTEL for distributed tracing and management and this helped us a lot with observability to make sure where exactly is the request in, how do we identify if there is

a problem in any part of the system. So, we had to change our infrastructure to make sure observability worked well And lastly, with all of these different processes and there I've not included many of them, but with all of these processes and tooling that we brought in, we wanted to make sure any new subgraph that comes in or the subgraph that we already have, bringing these features

are scaffold them into their own subgraph whenever there's an improvement. So, let's say if I want to bring in a latest version of node or I have improved the schema management service. So, how do I bring in these features? Or I have a new upgrade instead of using helm for deployment, I'm using something else. Right? So, how do we get all of these changes scaffold into the

subgraphs? That's where we brought in something called subgraph template and we built it in-house where any changes to the template, they'll get notified and all they have all the subgraph team or consumers had to do was run a CLI command and they get all of these features scaffold into that. this helped us achieve the process to run at scale. And obviously I've not included many of them

here, but some of these processes helped us run a long time. Now, let's see how this fits in here. So, this is what our new architecture looks like, right? So, we have observability built in like we saw, we had uh the template also added in, we had observability at both the different But there was one thing still remaining and I'm sure if some of you have already

used GraphQL, you might have seen there are some normalization even after applying normalization, you would still have some common code which might be needed across different Right? So, and these are not something that you expose to clients, which means client for example, if I take uh for searching for a product or looking for a product details, you in behind the scenes you would actually have to check

which store you're getting this for product from, right? Which store does this product belong to? That's a store information. And if you're adding this product to your basket, you also need to identify store information there as well. Which store are you getting this product from and whether it's available in that store. Right? And if you're fulfilling that product or you know, delivering it to the customer, you

need to make sure while fulfilling is it available? You know, at this store. So, this store information is is something that's common across different subgraphs. So, what we did, we brought in something called a shared SDK. And the shared SDK is is basically a wrapper which uh on top of all the which allowed us to have common logic moved there and backed by distributed caching and some

of the federated concept. So, basically whenever a request comes in from the clients, it goes to a subgraph and subgraph then decides that using some federation concept, we pass in the request to shared SDK, get any common information back, cache them and send it over to the gateway. So, that's what we did. So, this helped us achieve same KPIs or same latency that we had before with

XAPM only. And this was very important for us to achieve, right? We because this was tech driven initiative to reduce the toil from from our developers, this is something that we wanted to achieve as early as possible. So, shared SDK was the one which helped us achieve this. Right. So, now we had the complete federation setup. It was working for us. Now, there are some more nuances

that have not explained here. So, just to name a few, right? So, we had process setup for schema stewardship. But, this was still a manual effort, which means any new propositions, the schema team, the stewardship team had to manually interview and review all of the schema, make sure it composes well, make sure it follows the guideline. So, what we did was in this age of where I

brought in AI to help us automate most of it, right? So, I would say almost 70 to 80% of the time we don't have to come up with manual intervention. It automatically follows some knowledge base having some guidelines, make sure it adheres to that and applies or acts on those guidelines and you know, let's say promotes to different environments. If it's not If it's not breaking the

schema guidelines that we have. So, that's another one and the another one was observability. So, we brought in AI there as well and we are still working through it where it identifies a pattern if any issue had occurred occurred before, it goes through the knowledge base and sees whether a pattern of similar kind has been happening. And it acts on that. It either notifies or takes in

review from the on-call person or it acts on it. Let's say it needs to clear some cash, right? Every night what we had to do before was if there is any change to the induction of products, we had to clear our cash. And this was a manual effort and we would have to get run support team at night, you get a ring and they would have to

sit, clear the cash, observe, monitor, see it's all working fine. But what we did was we automated that. We brought in AI which actually understands the pattern anything happens on there, it automatically looks at that, monitors it, clears the cash and does it does it really well and it sends a notification as well. So, these are these are where are the some of the places that we

have brought in AI to help us and there are some more things that we have still working on. So, if you want if you're interested to understand a little bit more, probably we can sit after the session and discuss, but otherwise, thank you for listening and you can connect to me on LinkedIn and we can discuss a bit more about your experiences. These are our references that

I've used over the slides. Thank you so much. >> [music]