Saving lives with AI and the JVM, Danail Alexiev

In this talk, Daniil Alexiev, Engineering Lead at Infinite, discusses an innovative project aimed at saving lives in war zones using artificial intelligence (AI) on the Java Virtual Machine (JVM) platform. He highlights the collaboration with the David Knott Foundation, which focuses on training medical personnel in hostile environments. The project involves developing a mobile application for field medics that leverages AI to provide real-time surgical advice based on extensive medical knowledge. Alexiev also explains critical concepts such as Retrieval-Augmented Generation, grounding knowledge, and semantic chunking while detailing their practical application in creating a robust AI system. The talk emphasizes the importance of validating and structuring medical data to ensure accurate decision-making in high-stakes situations.

I would mention before introducing Dakta that Dakta was from the other side, or rather from the AI ​​side, right, because I know him, many years ago we ended up in the same company and Dakta was working with Vlado Tsanev at the time. I don't know how many people know him, and Tsanev gave him some task. He was a junior and the paper shows the solution to the

problem, but he wrote it for two days. I sit across from them, I shout, what's going on? It shows what he wrote. And Tsanev just looks and says. whereupon Dakata writes again two days later and shows him a new solution, after two days Vlado looks at it and says it 's better and there's no more feedback Dakata cancels it a third time and shows it to Tsanev.

Tsanev says it 's not great but I'll fix it a little bit, at least that's how I use it now, right? In the sense that I make him do something and I tell him [ __ ], that's not what I want. He tries a second time and I tell him [ __ ] it. I'll fix it. If only I had remembered the token scheme. Well, yes, you

should . So, without wasting any more time, let's talk about how we save lives with AI and the JVM. In a moment you will understand what is wrong with Java. Because it's written in Kotlin. Because I write in Kotlin. So there would be a raffle at the end, after the last lecture. Yes, if anyone came at the last minute, there will be a raffle after the last

lecture, so don't leave. What? Three three licenses, so if you don't use the internet, you can leave. Yes. Oh, hello, nice to see you today. My name is Daniil Alexiev, Engineering lead at a company called Infinite, so that's why I'm here, right? Lambda. Lalama. Ouch. Oh, I have 18 years of experience as a software engineer. 10 of them have switched to Android, the others to Java, Backend

and all sorts of other things like that. In my free time, I'm a self-proclaimed expert at making stupid jokes, so we can exchange experiences in that regard. And today I'll talk to you about how we're trying to save human lives with AI and the What are we going to talk about? A little about parallel realities. And I'll mention a few words about our partners at the David

Knott Foundation, who made this entire project possible. And really, how can we, as people involved in technology, help solve some of the most pressing problems worldwide? And I'm going to present you with the solution to a problem that my team and I created. I'll explain how you can do it entirely on the JVM, whether it's Java or some other language that uses that platform. I'll tell you

how far we've come with this whole project and I hope we have enough time left for a few questions that may be of interest. If not, I'm here for the rest of the event, so I'd be happy to discuss anything you're curious about. So, let's talk a little about parallel realities. Now, unfortunately, I won't be doing my best here, but rather an incarnation of Neil Des Tyson,

let's talk about physics, right, about some interesting things. The whole thing will be much more prosaic, unfortunately . And we all live in a very cool bubble. Whether we admit it or not is not important, because it is simply a fact. As a society, we have never been safer, we have never been healthier, we have never been richer, we have never been so kind. However, at the

same time, on this same planet that we share, there are, according to various sources, between 60 and 130 armed conflicts and at least 20 humanitarian disasters. We don't feel them directly, thank God, but they are here and they cause suffering to millions of affected people every day. And of course, there are organizations that fight this as best they can. One of these great organizations is our partners

at the David Knott Foundation. And this is a foundation based in the United Kingdom, and their main priority is the training of medical personnel, of doctors who work in hostile and war zones. That is, they take people who are willing, trained medics, and train them further, because being a field medic is actually something very different from being a normal medic who works in a nice hospital with

a roof over his head, without bombs falling from the left or the right. Oh, and then he sends these people to places to work in real military conditions, to train the people who are there and to save human lives. They do this absolutely flawlessly. And since 2015, they have trained over 2,500 such people around the world. You can see a breakdown on the map here. They have

basically covered the biggest military conflicts that are currently raging, unfortunately. And the new task they have set themselves is to make this whole thing much more accessible to doctors themselves and at a much better price, in order to reach as many people as possible with the help of new technologies and, above all, artificial intelligence. And here comes a logical question. How can we help? In our case,

we have the David Knott Foundation, which systematizes and organizes medical knowledge and has enormous know-how in training people. And this thing meets the possibilities of modern technology, where using the language we speak every day, we can access huge volumes of knowledge, right? These are those IAI AI chatbots we're all talking about, right? My colleagues before me had learned trading with them. I'm also learning some things. And

the best thing is that these technologies are widely available. Each of us has one of these gadgets in our pocket, whether it's social, with GPT chat, or something completely different, that can use such a natural language interface to learn new things every day. The idea was to harness all of these things together so that we could help all of these brave people who are in war zones

and trying to save lives, to do the right thing and help as many people as possible. So we came up with a prototype of a mobile application that would be used by such field medics, that talks to one another with a backend system, powered, of course, by artificial intelligence, and also using the knowledge of the people at the David Knott Foundation, to ground the thing that artificial

intelligence says, grounding. We'll talk about that later. If you are not familiar with the term, don't worry. I'll try to explain it. And after we came up with this thing, we set about bringing it into some kind of working form. Now this project has drawn a short straw. It fell to me to work on it now, and that's why everything was done in Kotlin, respectively in the

JVM. Ah, but in my opinion it wasn't such a bad decision. so. Let 's see how we can build something like this on the JVM. Step one and this probably carries about 80% of the weight for the success of the entire endeavor. No matter what cancer system you do cancer. This is Retrieval Augmented generation. And generally speaking, the idea is to fill this context that everyone has

been talking about so far, which is fed to the artificial intelligence with useful information, so that it can compile an answer. After all, it 's very important in these systems not to rely entirely on the general knowledge of artificial intelligence. Why? Because they are limited, they are cut off at some point in time. They may be incomplete, they may not be good enough for the specific domain

we are working on. Especially in our case, right, with a field surgery I can't let the chat GPT decide what to do. And the idea is that you have this kind of grounding knowledge, right, this grounding, from which you extract data and put it in context. This is basically the essence of Retrieval Augmented generation. Jordan, who will speak after me, I assume will also explain these things

in detail, because his lecture is focused exactly on that, so I won't go into even more detail. Ah, but this grounding is very important, because grounding is what actually prevents artificial intelligence from making things up, from hallucinating. Such a system, which aims to be factually correct, to provide true, not just good-looking information, cannot do without grounding in some way, because otherwise we risk very serious consequences, the

seriousness of which depends on the domain we are working in, but considering that we were doing something for healthcare, this is the greatest possible impact, right? You can't afford LM to make things up. And the most important thing we can do is to become extremely familiar with the source of this knowledge. What are the things that will teach our LM how to respond? And if we understand

how this knowledge is systematized, how the information in it is transmitted, only then can we have a successful basis for implementing any cancer system. And this includes not-so-sexy work like reading documents, reading text. Often this text is not structured, but a discussion with so -called subject matter experts, right, people with real experience in the topic, so that they can explain certain things, to actually explain to you

what this document gives you. And in our case, we worked with a field surgery textbook written by two of the founders of the which was very, very interestingly structured. I'll try to make an analogy for technical people. Imagine that one part of the textbook gives you a definition of methods and one part of the textbook gives you an algorithm using the definition of those methods. Of course,

because it was written by medical people, it wasn't that well structured. There was a little reading comprehension on our part, but that took about 70% of our time. To understand how the information is systematized, how it is organized, how we should interpret it in order to derive maximum benefit from the system. And I'll probably repeat this for the umpteenth time, but it's absolutely critical when you use

cancer. You need to know what you are starting with in terms of knowledge. From there on, you think about everything else and things are much easier to implement. That's right. The next step is this good thing, after we have read it and we have understood it, to bring it into a form that the machine can also read and understand it. And anyone who has worked with PDF

knows that it's not the easiest job in the world. Oh, my neighbors have enriched their vocabulary with a lot of new interesting words, in general. Ah, but what are the general steps? First, you need to extract data from the relevant PDF. Now, not all PDFs are created equal. They have different internal formats for themselves and overall it's a bit of a nightmare. And what did we choose

as a methodology? When you have a PDF, it 's important to get three things out of it. First, it's the text without any formatting, right? Just with sentences. It takes it out that way. Then it's a good idea to figure out the way this text is arranged, right, what font it uses, any spacing from neighboring texts, and so on. And something that we realized that further strengthens

things is if we take a picture of the page, we capture these three things, we send them to an LLM with the instructions. These are your text, how it's arranged, what the final page of the PDF looks like. Take out my marker. what can you use in the Java world and the JVM world to do this PDF processing? Achi PDF Box, this is probably the best library.

It's quite easy to work with. Now there are some specifics, especially if you are going to extract images from the text itself and so on. But for extracting the text and the way the text is arranged and for screenshots, it works easily, quickly. And it's good for writing. It turned out to be a lot simpler than I imagined Now that we've extracted the data from the PDF,

what I told you happens. We collect them, send them to an LLM and tell him to transform them to Mark. Now why Mark, we are all familiar with this format, and it allows us to semantically express, right, some text, to group it. There are paragraphs, there are headings, all sorts of things that they really like. Why? Because they love structure. Ah, that makes it Machine Readable. This

is what we need. To address LM, there are 1.5 million ways, regardless of which platform we are on. And we used Long Chachain For, simply because it was the easiest, and it saved us a little time. You can talk directly to the LM-a you are using. You can go through all sorts of other such framers. The decision is up to you. To us, this just seemed the

fastest and most convenient thing to do at the moment. And once you have the final markdown, the question is to save it somewhere, to use it for later. Here you can now make any decision depending on the situation. Whether it will be on some S3 bucket, whether it will be some other storage in some other cloud provider, depends on the pipeline, right, that you are building at

the moment. Many of you are probably wondering at this stage why we did it manually, when there are libraries that do it. Well, the answer is that the source, this textbook that we had, we had to treat as confidential data. One hundred of these cannot be released in a public cloud, in a public LM in any way, which automatically ruled out any such solutions, because there are

many of them. There's Lлаama Pars, and Google released something else too, right? You upload a PDF and it takes out Marku. We couldn't use something like that because we would be violating the agreement we had with our partners. And also keep in mind that the companies you work for don't always want everything to be public, right? So also before doing such things with ready-made libraries, inform yourself

about this. Can everyone see it? Is it okay to leak? Is it okay for an outside agent to train on it? And make a decision about these things. My advice is, if something needs to be even slightly confidential, never trust something you don't know where it will lead. I can tell you that it is only local. If you're not going to check it out, don't believe it.

In our case, once we have the markdown in question, what did we want to do? We wanted to present him as a count. Why Count? Ah, I explained to you how the book was structured. It relies heavily on references to previous sections, right, previous paragraphs. It seemed most logical to us to recreate these connections through a data structure that most accurately approximates this thing. And this is

a graph. And there is also such a subcategory, cancer cancer Retrieval augmented generation, which is a cancer graph. And in general, traditional cancer relies on vector databases. Graph theory relies on graph databases and relationships between information units, right, titita. And we wanted to be able to recreate something like that . And how can this be done on the JVM? First step. Once we have this beautiful huge

markdown, we need to break it down into some pieces. This is called chunking. For any cancer system, this is an important step because, you know, we have limited context. We can't cram it into context in 200 pages of PDF. It won't work. Now, chunking is the process of breaking a document into many smaller documents. There are many ways to do it. My advice is established empirically. Don't

rely on the simplest chunking methods, which are 200 tokens, 200 tokens, 200 tokens, and so on. This very rarely works. You should have an extremely simple document. What you're trying to do is what's called semantic chunking, right, semantic separation. This means, for example, if you know that you are describing some algorithm, each step of your algorithm is a piece and you can put meta information to that

piece. You could say for each piece, this is my algorithm for making pancakes, right? Weave, weave, weave. So that these pieces can both be interconnected and have some meaning on their own. This brings us back to understanding my data source. If I don't understand how the data is organized there , it's extremely difficult to get this semantic chunking right. And that's why I'm telling you that a

lot of effort needs to be put into it so that these decisions can be easy later. In our case, some things were obvious, right? They describe, for example, a surgical incision. That's a chunk for you. Great. But for others, we had to be a little more creative, because the algorithm, because it was written by a doctor, is not a first, second, third, fourth step. E first, second,

second A, third C. And for steps two and three, apply six, but maybe eight. Ah, so while we were doing all these chunks, we sweated a little. So yes , that's also important. I'll say it again. Semantic. Separate semantically. Don't rely on length. Don't rely on the most basic, right, token length, word length, sentence length, and so on. This works very rarely. In other words, if you

want to demo to your colleagues I have cancer, great. But the question is whether we have made the steps as nodes in the graph. Yes, there was basically a plan. Every plan has multiple steps, and each step has various actions. That's how we built the thing. Ah, but it's conditional , because some steps were different and so on, but generally yes. And let me just ask, after

this chunking, do you show where it came from? Because you might have made a mistake. Hmm. and you can tell the surgeon what to do, but maybe he wants to see the original steps with look seven and eight to be sure. That is, Mhm. And basically what we did was to make sure we were chunking correctly, we showed the output graph to surgeons. They said okay. The

other thing, besides the graph with the entities and relationships, is that you can keep something called a lexicographic graph, and there you basically keep track of which chunk each chunk came from. And if you put the right information, for example, this is from page so-and-so section six and so on, you actually have a representation of where exactly each piece of the book came from and if you

see that something is wrong, you can go and stop it, say, oh, we did n't make a mistake here, and do this debugging, right, and so on . I don't know if that's what your question was about. Just one more question here. Yes. Assuming that the input material is not a PDF, but rather hypertext, say, scattered across 1000 separate HTML documents, does that in any way change

the generation of this aa of this graph? Well, do the principles remain for clarification or do the principles remain the same? So, in my opinion, the fact that they are smaller is better, because it's easier for you to assess each of them and how to break it down. And the fact that there are 1,000 of them shouldn't bother you. If something works for one, it will work

for 1,000. It will also work for 1 million. The question I would ask myself in this situation is, do they all have to be broken down the same way semantically or is their structure somehow different? If it is different, what basic groups can I divide them into, right? Again, because we are currently working on something similar for another of our clients. We have several thousand documents there.

They are broken down into several categories. Each category is understood differently, right? Not everyone should use the same chunking, it all comes down to knowing the content. It's so disorganized, it can live in eight different systems. Will it be hypertext, will it be PDF? These are already implementation details . I would say that the algorithm steps remain similar. I just want to, I just want to, my

question is related to that of my colleague, which is again about data validation, because the way you described that it is actually validated is largely manual, right? Yes. And how does this actually allow the data itself to evolve in the future, and not just evolution, but how can this process be automated at some point, because with every change, this means going through the entire validation process to

see if the data we have is actually correct. how does the mains of this data continue, or is it clear that it is fixed and will never be changed again in the future? Well, first of all, it depends now, considering that this is a textbook, right, these things are not likely to run so far into the future. Ah, but you are absolutely right in what you say,

that validation is important and maybe the manual method is not always the best. And ways of validation can be devised . What have we done? The surgeons in question give us about 10 cases from their practice. We introduce these cases, give them the output and they start saying point by point, in the sense that it's not a graph, we went through it once and then we don't

sit down after each change, right, to read it together, but we simply give them the output and use it to compare, depending on their experience, whether they agree or disagree with this thing and evaluate it. And this is very similar to what is called in real systems like this. If you see that GPT chat, for example, they were asking you if you liked this answer, you give

bottoms up, bottoms down, right, ttt. All these things come in later as feedback and can lead to retraining models, it can lead to some structured document that someone can read and say: "We have problems here, here and here, right, let's go back, let's fix things. Ah, but if I had to summarize the answer in one sentence, it is mandatory to have a subject matter expt. Whether its

participation is manual or automated in some way like here is the input, this is the output that we expect. And now it is a matter of specifics, right, what the respective project wants to do. There are ways to validate whether agents work correctly in some way. Yes, yes. It was certainly the case with us. Now, if this thing reaches production, right, and so on, whether it will

have to be done constantly, I can't tell you. Bearing in mind that this thing changes maybe once every few years, and we will probably stay with this option. So. Ah, let's get back to the disentanglement, if there are no more questions. Now that we work with Markdown, an easy option to divide into pieces is using its semantic elements, headings, paragraphs, sheets and so on. And in the

JVM we can use the Common Mark library, which can parse and generate Markdown. And in general it is quite powerful. You can divide in all kinds of ways. We used this to form our pieces and the result is good. It is obtained as we need it. So. The next step, after we have the pieces, and in the case where we are making a cancer graph, we need

to extract the things that interest us, the so-called entities and the relationships between them. What do we need to have for this purpose? We need to have a predefined schema. That is, what entities do we want to have and what are the possible relationships between them. The purpose of doing this in advance is not to leave LM to compose some things on its own. For prostitution it

can work. However, in my opinion, it is good in such situations to put it into some rails as much as possible, so that he can't make up things, change your schema, invent new entities, new relationships And it's very important, when you create this schema in your head, that you are able to describe it well, what each entity represents, what each relationship between two entities represents, what are

the possible relationships between this type of entity We chose to do this entity relationship extraction, right, extracting these relationships with the help of LLM for ease and speed. That is, we submit the description of this schema, we strictly prohibit any creative activities on this schema, we submit the piece of text, and we receive Jon, for example, structured information. And you describe the ontology that you used. Actually,

I assume that you used one that was given to you or that you built yourself. This must be from some experts who are, that is, that Yes. That is, that we didn't use an ontology. Ah, ontology, that is the other way to do this thing is with natural language processing, without introducing LM into the whole thing. And then you work with ontologies, just like you said, which

are exactly from experts. What we did based on reading the we invented some graph, that is, you became experts and made the Well, to some extent, yes, of course, that went Okay, fine. I understood, they said this is and then otherwise yes, that's what I was going to tell you, if you don't want to do this with LLM, that is, to introduce some deterministic moment into the

whole thing, it can be done with natural language processing. And there is an Open Source library at MIT for Natural Language Processing, and you generate, that is, you make an ontology for your domain. the ontology. These are generally exactly concepts, important concepts, the relationships between them and you can get such a deterministic output from the whole thing. There are also tools that do it. Some people are

called leonata. Oh, they deal with this thing. I don't know if I missed it at the beginning. Can you briefly repeat what the ultimate goal is? These surgeons who are on the battlefield need to be able to quickly understand this problem, what the solution is and implement it. Is that what they are trying to do? Yes. In general, they provide information about the victim, their condition, and

various things there called hemodynamic indicators, right, pulse, respiration, saturation, and so on, and they receive a surgical plan of action, right. An operation is required, you have to make such an incision. When you make such an incision, you have to do this procedure, the next procedure, sew it up this way, and so on. And this happens in a chatbot-type interface. I would ask if this works locally

on the device, because after all, when you are in some kind of military action, you probably don't have connectivity or anything like that. Great question. We asked that too. We said that they told us that this is actually a much smaller problem Starlink works. Internet connectivity is good. In a sense, you can count on such something. I was just being honest . And they don't have alcohol,

but they have the internet, but yes, these are the realities. Oh, so, if you want to leave the questions for the end, because I see there are a lot and it may not be enough to answer some of them. That's why I said Okay. And when we create the graph, we have to persist it, save it somewhere. There are a variety of graph databases . We chose

new for J. And it's a breeze. It has a very nice interface for locally running queries and so on. It has a Java driver. And nothing that says New For, it's not actually an exclusively Java library. Oh, but yes, it works well and did a wonderful job for us. So, once we have the graph, we can now think about the agents themselves or the agent, depending on

the case that will work with So, the first thing in our case was to come up with the algorithm of action for this agent. What does it do? And it was to understand what the surgeon wants to do, whether to ask a general question or to treat people. If he treats people, what are their hemodynamic indicators? After these hemodynamic indicators, which group of patients do they fall

into. When you find the group of patients, where are the injuries, knowing what the injuries are, and how to approach them surgically and so Um, it was in no way an option for us to do this orchestration by LLM. Why? Because it is deterministic. And very often, when you have such algorithms for actions, they actually have very clearly defined steps, right? You shouldn't - bring any creativity

into the whole thing. So my advice is, when you have something like that, always question it, write it down in code. Don't rely on LLM orchestration. The simplest way is to use some kind of state machine or as the English say, a state machine. And I used a Tinder library for this purpose as I said, some people use Tinder not to be silly. I use it to

make a state machine. Ah, but yes, different phases of people's lives. Ah, but yes, this is very important at least in our case. In In some cases, the creativity that comes from L- might be okay. Not with us. That's why everything was from a state machine. All these things that I mentioned to you were clearly described. What follows what, what is allowed after what. And actually, the

management of which phase, to which phase of the conversation it goes is pure code. Nothing LLM, nothing fuzzy, nothing non-perministic. Then, of course, you have to define some tools that LM-a can use. We needed a framework so that it could manipulate the database where we had systematized all this knowledge. That's why we used Neil for Jr and Longchain to make the definition of these tools. And finally,

we created the very definitions of the agents, which are system prompts, and a little bit about how to pass context to each other when needed, which tools they can use. And how this is done depends a lot on the approach you use, on the framework for working with LM that you use. In our case, it was Longchain for J. These things are easy to do for Spring

fans, annotations, it works. Very cool that it even works. Then we have to make this whole thing accessible to the wider world, and to have some kind of entry point, some kind of and that You see, the whole thing is in Kotlin. Everything from the backend runs on the JVM with tools only for the JVM. And now the clients there don't have the JVM on the client

yet. That's a separate question. Now you're probably wondering if this whole thing works and what its status is. And the status is that we compared its current state against chat GPT, Gemini and such general models in general and according to the people who gave us the assignment, it does it better, which gives us confidence to say that the prototype is successful and currently representatives of the foundation

are communicating with other of their partners, possibly to secure funding for the implementation of this thing to such a state in which we can safely let people try it out, right, it certainly won't be to trust it 100%, but at least to validate what they really do on the battlefield with what the tool in question advises them to do. And here now I have a note on

how the whole thing works. And it's called Word Doctр, which is the way field surgeons address to ourselves. And here we go. This is called I forgot what it's called. It was some very twisted term. In general, this is how they say what the victim is. Information is given, age, what time the incident occurred, what actually happened. Now this thing is with a lot of throttling and

a lot of debugging to make it look okay on the record. This is so slow. In reality, it moves much and you see, and thanks to the hemodynamic indicators submitted, this thing using the knowledge from the textbook, determines what the category of this patient is. Some things that need to be done. You see, it provides an opportunity to choose what to do by the doctor depending on

some indicators, asks questions that specify what the next steps of the intervention are and Yes. The question was whether this thing is being considered to have image recognition. Absolutely. But that's just for a two-week prototype, that's all it's called. And so on, this is certainly something that will be even more useful than writing a text. And the people from the foundation have literally thousands of hours of

real recordings of interventions, carried out on the battlefield. And in the plans we are to include this whole thing with image models that can work and possibly based on this to strengthen the work. But we need time and resources to put it into action. we are what we started and this, however, if we say I am a lot and I don't know three of the arima I

don't know three steps what I do then can I say I don't know how to do this I will show it I don't know how to do it he will explain I don't have time to read 10 pages how to do this procedure so the question is is there enough time to react on the battlefield when it comes to some situation, whether to chat, if you don't

know how to do something ask the bot and so on. And if I have to be honest, I asked the same question . Wow these people have seconds to react now to take out their phones. It turns out In fact, in all the chaos, right, on a battlefield there is an awful lot of organization. And before people take on a patient, they usually have about 10-15 minutes

in which they know what his condition is. They have time to prepare. I did n't know that either. And the idea is to go through this process exactly in these 10-15 minutes . Okay, my man is like that, because a medic came who took him from the battlefield and brought him to the field hospital. I can write to him about his condition, to see, to consult. A

man even explained to us that it is not uncommon for them to talk on the phone. Look, in 15 minutes I will go into the operating room, the man is so-and-so. Can you explain to me what to do? Well, except in many crisis situations, they have time to prepare, which is actually the time that we want to use. And specifically, can the little guy give you a

description of something, some procedure, if you don't know it, you can, but still relies on these people having at least a basic level of surgical skills to be able to do it in the field . Still, and for these 15 minutes it's clearly not for others, but still maybe there should be text in voice and everything else. Yes, yes, yes, yes, yes. Certainly, because I don't even

know from the previous lecture Stefan what they're writing so much. I'm telling her what to do. And, just to ask, is there any manual work from the process of scanning the PDF textbook to tokenization and how is it validated that it was tokenized, that is, the textbook was read correctly, that there are no misunderstandings? Well, manual work is only when we already have the pieces and the

graph and a surgeon has to come by to say if we're not confused. Otherwise, you literally give the path to the file where the PDF is right now and you wait for it in the graph database after a coffee time. But But otherwise, by the way, I would say that videos and things like that are said to be available. I was very impressed. I think that chat

GPT can do it. When he tells me some video to tell him: "Well, don't give me the video, no one will watch it. Send me a screenshot of the things I'm asking." Yes. And it actually to get the important things out, where I can see them directly on the screenshot, which some other questions we have another 4 minutes, by the way, so the Dhaka is Marto wants

to ask something. wait, wait, wait. You mentioned cancer, but the topic of MCP is still very relevant. Is there a need to integrate it with some tools that additionally, for example, provide context for some procedures, additional things, some completely different tools? Not for the moment. the truth is that very often these people work in very spartan conditions in the field and they don't have a lot of

equipment, right, and they can't count on there being a connection. On our part, we haven't identified anything, right, technical, that could work for us. There might be something. The point is that we didn't have much time to delve into it. We had a very strict deadline, right, to bring in some kind of demo. And this thing is the only tools that can be relied on to be

able to to poke around in the database, right. That's it . And if we find something, of course we take advantage. The question is to give is the appropriate one. No, maybe rather, if you don't have a procedure for some of the things, would you use some party MCP server, to donate the data that you have, because for example someone said one discount the steps I can

I don't know how it is, because you may have completed this, but you may have just collected how exactly this is done and you do n't have it written down, because it's um well here the question is very political also, because our partners at David Nott are absolutely sure that at the current moment they don't want, right, the content that this thing is trained on, to go

outside the framework of rather it is to insert external content and to call the content that they have, with some additional things that are not their content. Well, in my opinion, if they give you yes, right, there is no problem at all. Oh, one more question. Did you have to have any PII data from this kind, which data should remain only on the phone and how did

you deal with it if you had to? Something that you didn't want to go on servers. Ha, something that we don't want to go on a server. Well, the easiest thing is to just never want it as input data. The question is, if someone decides to give it to you in some way, right, to filter it. The question is that this depends a lot on the data

itself, right? You have to be able to recognize that something is your PII in order to remove it from there. Ah, but in general, I start from the fact that I want as few things as possible as input. Ah, and otherwise, if you ask for patient PII data and so on, if you see, they work absolutely anonymously. The only thing that is shared is what your hemodynamic

indicators are, what your age is. And this is in no way enough to uniquely identify each person. And in general, even with them, even if they have to keep their documentation manually, they don't use names, right? This is the patient, he was identified by his hemodynamic data and what happened to him And I don't know if there are any other questions, but I would like to ask

how it works, because you are actually making a cancer graph. But I know that for example when you write code, let's say, I'll say Intel Jidya, there is an AI assistant, which is a complete mess, because it actually uses cancer, does a search and doesn't find anything, but some people pay for $ credit. Anyway, I'll come to that now. So how does this work in comparison to,

for example, context engines or how does this work in comparison to how the LM notebook works, where you fill in your data, it still creates an internal graph, but actually uses it as a real node and then it really doesn't miss any connections, because with you, even though you are making a cancer graph, you can still miss some important connection for the specific case. That is, how

and in fact, so this is that we are actually, if we have to be absolutely honest, making a hybrid cancer. Ah, because in addition to having this graph, we are embedded and each chunk. That is, in addition to the graph, we also have a vector representation of this thing. And when a request comes in from the user, we do the following. And, we embed it, we do

a vector search, we take the top few things, we put them in context, we take the user's request, we make a small graph out of it, using our schema, and we use this graph to impose it on our graph and we catch absolutely all the connected nodes, we pass them into the context. Separately, we also do a full-text search. the content of these chunks, if something bothers

you like a word, we take them too. Yes, context is blurred, but this way we reduce the possibility of missing some important node or some important some important aspect of what you need to return as information. That is, we don't rely only on okay, I'll look for the graph. We do three things to fill in the context, before the user starts answering. Here's another Yes. And I'll

validate my understanding. The idea that we use a graph comes after you enter in the domain itself, the book itself, you understand that the information itself is semantically in the form of a graph and that already gives you the idea. Okay, here I need a graph. If there were, as they said a moment ago, 1000 HTML documents and each one is for itself, then there would be

no need for a graph. You wouldn't think about a graph at all It depends, right, the answer to all the swinier people. And, the truth is that there are some situations in which a graph is better than a vector. A vector is very good, if you have Here's a question, give me an answer. And a graph is better, when you have to do more complicated workflows, right,

to follow the logical path of something, then a graph is better. And with us, one thing is that you have a logical sequence of steps, secondly, all the knowledge relied on some kind of interrelationships between individual components and the two things overlapped and there was simply no way we couldn't go to a graph. And thank you quickly. I also ask what kind of things do you put

in the metadata of the chunks you're extracting? In this case, I put which section of the book they're from, which page they're on, and in general, I keep all the sections nested within them to give me additional context when searching, and so on. But I can't give you a definitive answer. This is for our case. What will work for yours might be something totally different. Okay. And,

let's thank Dakata for passing the time. We have a short break and the last lecture. Thanks. Thank you.