Debugging Microservices with AI + Neo4j: Find the Root Cause Faster - Ganesh Das

This talk focuses on the challenges of finding the root cause of issues in microservices architectures. The speaker shares personal experiences of debugging production issues by utilizing various logging tools such as CloudWatch, DataDog, and Splunk, but notes the difficulties of correlating logs across multiple services. The speaker introduces a new approach that leverages graph structures, particularly using Neo4j, to model service dependencies and propagate failure information in a more effective manner. By creating a knowledge graph that includes telemetry data and service relationships, the speaker demonstrates how this method can significantly reduce mean time to resolution (MTTR) by providing clearer insights into failure paths. Additionally, the integration of AI reasoning to generate root cause analysis reports is discussed as a means to further enhance the debugging process.

Today I'm going to talk about something very real which we all face in you know our day-to-day lives. So that is nothing but finding root cause in microservices, right? So everybody might have faced this challenge before uh in your previous experience or currently you might be facing this. what is happening, right? So from my experience, there were a time I was debugging production issues. So I I

I actually whenever I root I mean the service get failed, I know I look into the logs and start, you know, finding out what exactly is happening, right? So look at for example, I was looking at one point of time cloud watch and filtering it and uh you know, trying to correlate what exactly happened on the microservices, so so one point of time when I faced this

challenge, right? So I I started debugging it. 1 hour passed, 2 hours passed. It It ended up in a complete full day to you know find out what what is happening. So over a period of time I'm only not only looking one tool. I was looking at you know uh uh data dog. I was looking at Splunk. I also looking at cloud watch, right? So these are

uh different different logging systems. what I understood from this uh if I you know look only in one logs, I'll be able I'll be not able to find out the root cause because it take time. I'll be able to find it, but it take a longer period. Right? So what I did is uh I I looked into the old logs, find the root cause. Finally, I came

to know that logs are, you know, technically uh telling you what is happening, but it's not telling you why did exactly it happen. Right? So, the thought is mainly how we can improve here, right? How we can improve the time that is consuming to find out the root cause in microservices. let's let's think this way, right? Let's take a system that has uh that has an gateway

that is calling order and order, you know, uh do the payment service and payment to inventory. So, when a payment get failed, right? It actually impact order, order impact uh you know, gateway. So, there are multiple services get failed. So, as soon as it get failed, we will start looking at all three logs, right? All three uh services logs, what what is happening. I I I'm I'm

I'll see all three logs are showing the error. But, uh we we are not able to find at exactly where did it happen, right? So, this is where the challenge. So, let's uh Yeah, let's uh let's instead of thinking in uh logs, let's see uh flatten structure. Let's see how we can look this problem into a graph structure, right? So, basically, what we are missing here is

we are missing the correlation between these three uh services because one services is dependent on another one, other is dependent on third third one and fourth one. It's It's go like a chain, right? So, if we have a relationship between all these services and if we have a key telemetry information, so we will be able to find out where uh, the problem is right? So, to improve

that to to find a problem, I mean, solution for that I model that uh, services services in this uh, this way in Neo4j. If you see here the nodes I put as a service as a node. And that service calls another service and the the the second service called another service. So, we will be able to figure out the, you know, propagation path. Right? So, if you

consider the system that I discussed just before is uh, gateway calls order. So, call and uh, order calls payment, payment calls inventory. So, if you see the if I can able to set up a relationship between so, this is easy to mention I mean, find out the propagation path, right? Here service generate event event trigger alert, alert trigger incident finally we'll be able to, you know, uh,

find out a root cause and tag to a proper service which is the cause of the problem, right? So, let's see how did I um, model this in Neo4j. So, if if you see these are the services like service uh, gateway, order, payment and inventory. These are interconnected using the relationship called and this gateway is generating event that will contain a minimum required elementary information. And that

event will trigger an alert if it is an error, right? And alert will create an incident. So, this way we will be able to, you know, set up entire uh, failure with enough, you know, telemetry information into a graph. So, now this complete problem become a graph problem. So, we we can extract the context from the graph what was the you know, root cause of this problem.

So, if you see a small cipher I've written there, so which is actually trying to find out the propagation path that, you know, where is this entry point and where is the ending point, which is nothing but the failure point. Right? And pick up a list of nodes and try to that give you an information that this is the beginning of the problem. This is the service

that actually get failed and others are the cascading impact. Right? So, now uh what we have is but we have the context. We have the context information in the sense when there is a failure, we got the time we will be able to search the fetch the time stamp information. We'll be able to search the error rate. We can, you know, we have the And we we

we have the additional information like what is the service health and what what are the, you know, other parameters that are showing the the the ratio of the failure. So, once we have enough graph context, so we can use this graph context for a reasoning, right? So, so what I'm trying to say is we already have this detailed information. We already tried I mean, saved enough time

for the engineer, but additional as an information or converting to a natural language we can use AI reasoning here and finally prepare a you know RCA report. So, if you see here, this is the sample context that we can provide that we provide to LLM for reasoning. So, here you see the details of the and level of what is the error level and the you know what

what exactly it failed from the logs logs side, right? What what what was the cause of the failure from the log side? And the timestamp error rate and if you see here, I we have an additional property that is called order, right? So, the technically this service is calling order and the order calling payment and payment calling inventory. So, it's giving the propagation path to >> [clears

throat] >> to find the RCA to LLM. And at the at the as a resultant, it it will return back the details about the in this format, right? So, you can see the root cause is actually caused by the and the impacted service are order and gateway. Reasoning is payment is shows the strongest failure signal because of the you know, additional parameters that we passed through uh

passed to the LLM. what what where we improved, right? What we can see. So, that I mean, generally we know that when there is an alert fires, a lot of people get involved and the teams will start looking at the logs and multiple incident call. So, we will be able to overcome all of this because as soon as alert fires, we can you know, ingest the information

to graph query and become a good knowledge graph and use that it for final reasoning, right? So, this reduces lot of MTTR from hours from the second hours from seconds, right? This wait Yeah. How we can differentiate in our logs in this from our small escape? So, in this in part Yeah. So, so what we we what we are doing is we are not pulling entire elementary

information, right? So, what So, if you look at back Hello. So, so we have we are fetching the in I mean required information from the main log and putting into knowledge graph. So, that's what these properties are. Like what service it is. It's a gateway log. Yeah, it's a gateway. But Hitler headers are getting the respective service. Like say in the payment service. Yeah. So, we need

logs for payment service as well. Yeah, yeah. We need logs for all three. Also, we what we define is we will while calling the API itself will I generate a log. This is the first call and this is calling the connected API and will try to capture that logs as well. And so, we can ingest Yeah. and we normalize it. Correct. Then we represent Yes, we'll do

So, so that we get the propagation path, right? The main idea is to put more context from the logs and also try to pull the relation I mean relation between the services. So, if you if you generally look at, right? So, there are many use cases we we can think about is here is one I said the payment. If you think about you know, e- I mean

especially on e-commerce system. So, it it comes consider the order, right? It has a lot of APIs need to be dependent like promotion, payment, and you know, rewards, all these things. So, to answer your question, yes. So, we need to bring entire elementary information, but when we share it to LLM, we don't need to share entire elementary because there is a security issue. We don't need to

share production error logs or service logs to the LLM, right? Yeah, you got to shoot. You can solve this problem with the tail of the net. Sorry? Open telemetry. But, we can solve this problem with the So, how this approach is better than tail and if it's template tools to make tail order? Uh, open telemetry. Open telemetry. Uh, so, what what is that? I I don't know.

So, so, the ability to fetch again it actually introduce that trace ID automatically in the board. Products conducted and I find my role in that area. You got to shoot that tail with how many tail. So, it will introduce a trace ID in the like take the complete challenge from the 140 and then take it to be the trace ID in the span ID. Later then you

get automatically in the traffic again. So, so, the would be able to integrate multiple platforms. Okay. You got to go file or take it in the front of like a services you are actually going to inspect. You give it a one level of the service like the other it is or So, so, would would would you able to define that? I'm not sure about much details about

that product, but there's a relationship. What about is it It actually introduce the trace ID. That trace ID will be able to post it through through all the board products. Whatever the microservices that are being called out to that trace ID get taken just all the directions come. Okay, then it's it's might be the similar use similar use case that we are what Technically it's trying to

solve the graph problem. So it's a finally When when we try to see what is the relationship and notes then it comes to your graph problem. So which is maybe very similar to that. So yeah. Uh the key takeaways What I'm trying to say is logs are you know very necessary but Uh it's not sufficient to find out the entire I mean find out the root cause

here. And you know So I I did a small project you can scan GitHub and see it's actually a demo that simulate the couple of API and it it generate the ingestion structure also included the final graph result. Sorry. Yeah, this is specific to only just to prove this prototype. Right? So you can see Yeah, yeah you can customize it's it's Yes. This code This code is

a small project you can use it for just to you know understand the prototype. That's it. So it's not a big project here. Just a demo project. >> [music] >> Mhm.