A Practical Introduction to LangChain4j - Venkat Subramaniam

This talk provides a practical introduction to LangChain4j, a Java implementation of the LangChain library designed for interacting with various language models programmatically. The speaker discusses the significance of using language models, particularly for writing code and enhancing product functionality through AI. He emphasizes the need for flexible APIs to communicate with different language models, explaining concepts such as the dependency inversion principle and the adapter pattern. Examples are given for querying models like OpenAI's GPT, incorporating context, and using retrieval augmented generation (RAG) to process local data. The session highlights the advantages of LangChain4j in streamlining AI capabilities within Java applications and touches on utilizing agents for enhanced query handling.

I'm going to talk about uh a practical introduction to LangChain4j. So, let's get going. What I want to do here today is to talk a little bit about uh what is what is this uh particular uh tool do? Uh why should we use something like this? Uh and then we'll take a look at some examples of it as well along the way and uh see how we

can put that to use. So, we may we're living in a very exciting time as as you know uh where we can see uh LLMs playing a really important role. So, the question is, how can we access those LLMs? Uh I want to to look at it from two different uh use cases, if you will. So, the very first use case that we as programmers often tend

to get really excited is about using LLMs to write code, uh to deal with various aspects of uh software development. That's still useful. Um but, the timer here is wrong. If you can fix the timer. Uh so, uh that that's still useful, right? That's really great. But, the other use case we want to really think about is, how can we use uh AI to enable our products

so that the products can provide value to our end users. So, the question here is, we have a user uh using our product. They perform certain queries or make a certain request. How can we bring information into it and make AI respond to uh their request? So, we're going to look at some examples to achieve that today. So, in in short, we want to be able to

access AI programmatically within our application. So, great. I want to be able to use AI, but as we know, there are multiple different models we could be using. And when it comes to using a particular model, how could you communicate with that model from within your code? I think the answer is obvious to everyone in this room, right? You talk to stuff that you depend on using

the APIs they provide. Okay, so we're going to be writing some code to talk to the API. That's great. But we also want to be able to either switch between different models, and sometimes you want to also bring uh results from multiple models. Let's talk about those two use cases for a minute. Why do I want to switch between models? Uh there could be a number of

reasons. Uh a particular product may be using one particular model. As we know, a new model gets released like every other day. So, if you want to use a different model, you have to switch to the different model. Does it mean we're going to modify a lot of code? If we have to modify a lot of code, that's expensive, that's too time-consuming, that's not a good thing.

Secondly, as we know, AI is rather very non-deterministic. Well, when AI is non-deterministic, one of the ways you can deal with non-determinism is to really use multiple models, and maybe see if there's a consistency among the models, so we can lean towards a particular recommendation or solution, or at least gain a bit more confidence. So, if one model is hallucinating really bad, you can say, "Gosh, that's

so out of what is expected. Maybe I'll form a consensus with these other responses." Okay, so the question then is, how do we talk to these different models? Now, obviously if you want to talk to a model, you have to use the API the model provides. But, what kind of API are we going to talk to? And I want to step back and talk about this a

little bit so we can think about you know, some of the problems we want to really think about how we can solve it. So, programmatically what do you normally do? So, your code in general can talk to a dependency and the way we do this is by using an interface. So, typically your code will talk to something with an interface and then you can communicate, make a

query. Let's take something we are very familiar with. Let's say you want to talk to a database. Well, how do you typically talk to a database? You could, for example, using Java, you could be using things like JDBC. Well, JDBC is a is a standard API and you can talk to JDBC, but in effect you can talk to different databases that provide access to the database using

the JDBC API. So, for this, you will have to have uh different, you know, vendors, if you will, uh implement those so you can have implementation one, depending on a particular model that you may be talking to. Uh you may have another implementation, implementation two, and and three and so on. Well, that's great. But this this really boils down to you need to have one interface and

everybody needs to be able to implement it for you to be able to use it. But there is one philosophy that I like to really follow and and that is that standardization, right, uh is is extremely important. We we want standards because if we don't have standards, it's really hard to uh communicate with things. So, standardization is very important. However, I also firmly believe uh before innovation

uh is a bad idea. So, the reason I emphasize this is I I like standardization, but there is trouble if we try to standardize too early. We have seen this in the past where things have not evolved far enough, but we rushed to standardize as well. Well, in the space we are in, there's a lot of things that are unknown. We are rapidly in the development phase.

So, I would say any effort to standardize right now will really be not be successful. So, the question then is if you don't want to force towards standardization yet, how can you really have a standard API to talk to and yet not require them to standardize and and deal with it? Now, let's think about things we already know. If you look at this particular diagram here, so

to say, you probably know the solid principle that's behind it, right? So, the solid principle behind it is the dependency inversion, uh principle, right? So, this dependency inversion principle says that your code can depend on an interface and the implementations of that interface can bring in the execution under the hood, but your code can decouple. So, tight coupling makes hardware software really hard to maintain, makes it

hard to extend. So, we normally go towards dependency inversion principle. But, dependency inversion principle requires that these implementations actually follow the interface. Well, but what if we cannot? We know that a particular pattern can help us in this case. So, what pattern can help us if there is not a good standard to follow, but yet we want to conform to a standard, right? So, this of course

you probably know is the adapter pattern. So, the adapter pattern allows you to adapt an interface to different implementations. So, potentially you can have an adapter where the adapter could be implementing that particular interface, and this could be routing this call to yet another implementation. In this case, maybe let's call this an implementation four. So, a combination of these can help us to solve problems. Well, so

literally this goes back to some design principles and design patterns we are really used to. So, think of LangChain for J as really an implementation of the dependency inversion principle and the pattern adapter pattern to get us there, so we can remove this tight coupling. So, the question then is you have a client which is going to talk to an interface, and the interface can be implemented

by an adapter which can talk to an implementation. Well, let's step back for a minute. So, programmatically a lot of these things progress a little earlier on the Python side. So, there's a library in Python which is the LangChain, and the LangChain basically is a library that allows you to talk to multiple language models. So, the word Lang uh should make good sense right now. Lang is

basically is referring to talking to different language models. Well, what does the word chain really mean? Well, the chain really means you can talk to multiple models, so you can have a chain of those models to talk to. So, that's basically the idea behind the LangChain, which is a Python library. Well, the Java folks excited about using this in Java decided to implement a Java implementation of

LangChain. So, the for J added to it is a Java implementation of the LangChain, so you have LangChain for J, which is a Java library that you can use. So, this provides a set of interfaces and adapters to talk to multiple different implementations. So, in a way, this is why I would compare this to as an example to how we use JDBC. So, if you're ever written

code using JDBC, your code talks to JDBC, but you need a driver to be able to adapt that interface to a specific database. So, if I just only give you JDBC and say good luck, you're not going to be able to do anything. Your code won't run. Your code will compile, but it cannot execute. Well, to execute, need the driver that takes these calls and then runs

it across the database. In a similar way, you need the interface, but you also need the implementation. So, let's take a look at that part first before we go any further. So, you can see in here, we have the LangChain for J, which is the interface itself, which means your code will compile with that interface, but during execution, you need the implementation that carries the calls to

the specific language models. So, you have, for example, OpenAI, and that'll help you to route the calls to OpenAI. So, you say, "Gosh, but I want to use Gemini. What do I do?" You'll bring one more dependency, which is the implementation to Gemini, and so on. So, depending on the number of models you want to talk to, you're going to have the base implementation, the interface, but

also these drivers, so to use the word, that routes the call to those implementations as well. So, this is why the comparison to JDBC makes a lot of sense in my mind, because it's a very similar approach, though an entirely different purpose and the intent for what you're trying to do. So, let's talk about how we can use this to communicate and and and just look at

an example. So, we want to do this programmatically, be able to talk to things. There are two use cases you want to think about. The first use case is your users may be interested in a very generic information that's available, uh, you know, to the world. For example, let's say you're creating an application that helps people with, uh, travel. So, they can book hotels, they can book,

uh, you know, flight, whatever it is, right? And I travel almost every week around the world. I'm in different parts of the world. And maybe I want to use this particular application that helps with my travel. But one of the things I may be interested in is to ask the site, uh, "Hey, I'm in Bangalore this week. What are the interesting things to do in Bangalore?" Or

maybe next week I'm going to be in, uh, you know, Iasi, uh, Romania. What am I going to do in Iasi? I want to know about things I can do over there. So, I want to query about things to do in a particular city. Now, let's take this a little bit further. If I say what's interesting in a city, that's a very broad question. Uh, what do

you How do you define what's interesting? So, for example, uh, I'm a very boring geek. When I say, "Tell me what's interesting in a city," usually I want to know what user groups are meeting that week. So, I can go to a Java user group or a certain, you know, programming user group. That is a definition of interesting to me. When my wife travels with me, the

word interesting takes a very different meaning because she has life unlike me. So, in this case, the system needs to provide a different set of things interesting to do than if I were to travel alone and ask for interesting things to do. So, we are talking about a context in this particular call. But remember, this is nothing specific to any particular application because what are interesting to

do in a city is a general information. So, interesting things could be, you know, cultural, could be programming related, could be you know other events related, but all that can be fetched. So that's the first part we want to look at, then we'll nail down to much more interesting later on. So I want to be able to access in this case I'm going to use open AI

and obviously you need a key to be able to access these services. You're probably going to be paying for this. They also want to be able to limit the access so that somebody doesn't abuse their systems and overload it. So there is an API we're going to be able to use. So as a very first step as you can see I have this right there is the

open AI key which I'm going to be getting from the system environment variable so that I don't have to expose that in here, but that's been set on my system level. So when I execute the get environment it's going to be returning that for that particular key and that key is going to become available for us to use. Now, what is what is the next thing we're

going to do? We're going to be doing very general here that is not specific to any particular application. So what can we request for? So I was thinking what kind of request I can make. You know you can ask for things interesting in the city, but then I realized one of the things I like the is telling dad jokes. Dad jokes are awesome because those of you

who have children know this. Children hate dad jokes. And as a parent I realized very quickly my mission in my life these days is to embarrass my children. And and the more opportunity get to embarrass them in front of their friends it's even better. So I've never seen my children roll their eyes so much as when I start telling dad jokes. So I said let me just

start with a dad joke here, right? So I won't be fed with more dad jokes. So why not ask AI for dad jokes. That'll be really nice, isn't it? So, let's use AI to get some dad jokes, if you will. So, I'm going to first of all in the main say, you know, output we'll go ahead and call it as a chat. And what am I really

asking for? I'll say dads over here as a way to customize the joke I want. You could ask for other types of jokes, right? You could say give me a joke related to politics. You can say give me a joke related to weather. But in this case, I just want a dad joke. That's what I'm going to be asking. So, in this case, I'm going to get

a string as a result, and we could say type or or we could say nature, you know, of of joke, right? I want to ask for a nature of joke and return it. So, what is this going to return in this particular case when I make a call? So, I'm going to say in this case a chat model is equal to and I'm going to use open

AI. So, we'll say open AI. Okay, so before we go further, programmers in different languages do different things to program. Uh and and for some reason over the years, Java programmers and whether you like it or not, gravitated towards a builder pattern. And the irony is it's actually not the builder pattern, but it got the name, so everybody runs behind it. Uh personally, I'm not a big

fan of Uh I I kind of cringe when I see it. This is just my personal preference. I was working with a client recently, and when you work with the client, you always want to be very polite. You don't want to say bad things to them. But but I was looking at their code. I was just kind of coding away. One of the, you know, key developers

came to me and said, "So, what do you think of this code?" I said, "It's interesting." That's a very polite way of saying noncommittally, right? I said, "It's interesting." And then he said, "Yeah, yeah, but what do you think of the builder pattern?" And I said, "Well, it's kind of interesting, too." And then he's like, "Come on, tell me how you feel about it." I said, "Can

I be frank about it?" He's like, "Yeah." I said, "It's a bunch of noise with no real, you know, uh purpose and benefit." He's like, "I'm glad finally said that, right?" That's how I feel about it. It's kind of overblown, in my opinion, but that's what you're going to be seeing quite a bit when it comes to this. They kind of use the builder pattern. So, you

call the builder on it, and now you're telling it what to do. In this case, I'm going to provide the API key, and this is going to be keys.openai. You're specifying the key to use. Then, you say the model you want to use. In this particular case, I'm going to say GPT-5, you know, five maybe {underscore} one, right? You can choose whatever model you would like to

use, and you can see up in the top where it's coming from as the model to use. And then, you are saying, in this particular case, you can provide what's called the temperature. A temperature is giving the freedom to be creative, or please stay closer to the facts. So, a lower the value, the more closer it's required to be to the fact, the higher the value closer

to two, it can be very creative. So, it depends on what you're trying to do. In this particular case, where you can vary the temperature, and then finally, you can do a build on it to get the chat uh model, if you will. You can wrap this into a separate function, that way the function doesn't have to depend on even the OpenAI chat, and the rest of

the code can be independent of the model that you're, you know, using. So, then I can say chat model in this particular case, uh dot, and you can ask a chat on it. And then, of course, I'm going to return the result of that. In this case, I'm going to say, "Tell me uh a person's S joke." In this case, dad's joke, right? And then, of course,

formatted, and we'll simply say nature of the joke and call it. So, this simply says, you know, "Tell me a dad joke." And we are asking for uh that kind of a joke in this case for it to return, and we are simply executing it, right? That's all we are doing. So, this is making a request, and then it says, uh "Why did the scarecrow win an

award?" It says, because he he was outstanding in his field. Okay, so that's kind of a boring joke, but that's basically a dad joke, and you can specify that. So, that's basically an example of one of the jokes it said. But, of course, you don't want to just say a dad joke and walk away, right? Because we are not just dads or moms, or we're not just

parents, we are special. The The worst of the kind is you're a parent and a programmer. That's when the double whammy comes together, right? So, you want to not only just say a joke, but you want to say a programming-related joke as well. So, talking about dad jokes, this is not AI-generated. This is Venkat-generated. I'm a very As you can see, I'm very proud of this joke,

right? So, here is a joke for you dad joke for you. Let's see if you're able to answer this. What is the most favorite animal for functional programmers? You want to take a shot? What's the most favorite animal for Python comes close, but even better. What was that? It's lambda. Yeah, there you go. So, I tortured you with one. But, we're going to ask AI for this.

So, remember what we talked about the previous example we talked about, right? So, one of the things I would like to do is to customize it. So, when I say, "Give me something interesting to do in the city," the answer is, "Interesting for whom?" Well, for a programmer, obviously. So, we're going to take this a little further, if you will. So, let's go back to this and

say, "Aha!" But, I'm not going to just chat with it without the context. So, I'm going to say system message is equal to we create a new system message. So, what is the system message I want to create? This is when I can provide a bit of a context to this, if you will. So, I'm going to say with the system message I'm something like you are

a programmer, right? Let's say you are a programmer. In this particular case, or we can say programming nerd, right? And then I can say and you would rather you could say, right? Talk about in in a sense. So, talk about rather than so then let's say code than anything else if given choice. So, you're saying you really want to focus on programming. So, keep your response, you

could say, right? Response we could say as much as possible related to programming. So, I've kind of tilted it towards programming. Then I can say user message in this case is equal to new user message. And the user message message is no different from what we asked earlier, right? I just basically send exactly the same message. So, the difference in this case is we just did not

send the message alone asking for a dad joke. We set a context to say you're a programmer, now tell me a dad joke, right? So, so that's basically the context. This is what I was giving as an example earlier. Tell me something interesting to do. Oh, by the way, you are a person interested in music. Or you're a person interested in, you know, other kind of cultural

events. Or you're a foodie, whatever it is. And you can specify that. So, in this case I'm going to say a list of this is going to be system message and user message. You want to pass that in and that returns to you an AI message and I'm going to get the text out of it. So, that's all the change we made. We tailored it to a

particular context and we are executing it and see what the response is going to be. So, this time it says, "Why do programmers prefer dark mode? Because light attracts bugs." I think that's pretty good, right? So, so this is a proof that AI is better than me. Right? Compared to the joke I said, I think this is actually a lot better joke than than lambda. So, so

that's of a query that tailored this to something relevant to what we are asking. Okay, so far so good. So, we we looked at an example of, you know, asking for a joke. We called this upon the model. We got the response back. We then saw we can use a system message. We can get a specific message, you know, tailored towards that particular thing. Well, that's all

great. But, uh typically though, we want to go a little further than this in general. So, for if you're developing an application, you probably want to not just use only information that is globally available, but you want to tailor this to your specific, you know, information. For example, let's say you are working for an insurance company. Well, if you're working for an insurance company, you might want

your application to make recommendations to your users based on certain insurance policies and rules. Uh for example, somebody may come to your application and say, "I've got this kind of an incidence. Can I apply for, you know, this particular insurance claim?" And you may want to make some recommendations. Or, lessons I've learned over time. Before the pandemic, for example, Uh, of in the US will charge you

for a a change fee. So, within the US if I want to change a travel, I got to fork out $100 per change. Uh, across, for example, uh, the world, you probably have to pay as much as $200 change fee, which I really hated. So, for years, I would keep telling airline that, "Hey, I literally live on your airplanes. You Why are you charging me this excessive

amount? It would be really nice if you would make me uh, able to change things without costing so much." They they would never listen, right? When companies say, "We want your feedback," which means we'll just file it, we won't read it. Well, pandemic changed everything. And post-pandemic, the airline said, "Wow, you can you can change your travel anytime you want to." Well, this is where rules kind

of kick in, as I found it the hard way. Within the US, if I change my travel, there's no cost to change the travel, except you have to pay for any, you know, fees to pay a higher or a lower value for the flight you are taking, there's no change fee. Well, as it turns out, very tricky, that only is true if your travel starts within the

US, and not if your travel starts outside the US. So, I had to book a flight traveling from Europe back to the US and back to Europe. Well, guess what? I had to change my flight, and when I went to the site and said I want to make the change, their immediate response is, "We're going to slap you with a $200 fee." I'm like, "Whoa, wait a

minute. I thought there's no change fee." And the system is like, "Yeah, but that's only true if your you know, travel starts within the US, not if it starts outside." But, these are a lot of different rules that you have to go through and process it. So, the question is, how do you write the application that needs to work with all these rules and processing? So, what

did we do in the past? What was our experience in the past 5 years ago? Uh, you could be sitting and writing custom code for a lot of these things. And when you write custom code, as you know, it's a lot of effort. And when the rules change, it also becomes a lot more difficult to make these changes along the way as well. Uh then we started

doing a few other interesting things. For example, we started using libraries and products like Drools, and these were rules engine we could use within our application. And the rules engine gave us the ability to exercise these rules, and we started using DSLs, domain-specific languages, so we can easily modify the rules, but we don't have to write code when the rules were changed. And the system could then

reprocess these rules and apply them. Well, that reduced the coding effort for us, but you were constrained by what was possible by way of implementation within these rules engine. Well, if you really think about that kind of use case or scenario, AI is taking you a lot further in that area. Because AI can uh process any natural language and text, you can use AI to infer from

all these documents and make recommendations. With one caveat, that AI is rather non-deterministic, or more important and more bluntly, it's rather inconsistent. So, you have to be a little bit careful. You want to probably put guardrails around it to make sure that it's not saying something that's completely wrong by way of hallucination, it's making up stuff. But, ignoring that, right? This is the irony, right? You have

something extremely flexible, but completely inconsistent. Whereas, there's something that's very consistent, deterministic, but not so flexible, not so easy to use. And that's a trade-off we are dealing with. It's a It's an enormous trade-off, right? You can deal with something that is absolutely deterministic, but it's a lot more difficult to work with, or something that's easier to work with, but completely non-deterministic. Go figure, you got to

choose between them. So, how do you really, you know, get to that particular point? But, ignoring that non-determinism for a minute, you want to be able to use your own information. So, this is where RAG really comes in. So, what does RAG really do? So, essentially, the idea of RAG is a retrieval augmented generation. I would actually argue it should have been augmented retrieval generation, because you

first augment it, only then do you retrieve it. So, the order is kind of mixed up here. So, the first thing you do is before you perform a query, before you retrieve data, you augment your query with your custom information. So, for example, in the earlier example, we simply said, "Give me some dad jokes." Well, that's kind of global. You didn't have to say anything specific to

your environment, your organization, your application. But, on the other hand, in the example we're talking about now, you may have a PDF document. You may have uh various forms of document. You could have uh images on your side. You might have a vast amount of data from research, whatever that could be. So, you're taking the wealth of your data, your documents, your uh you know, whatever is

internal to your organization, uh or specific to your business, and you are augmenting those into the query. So, you're telling the LLM first of all, "When you execute a query, just don't use the global information that is known to the world. I want your query to be executed based on the context and information of the document that's pertinent to my organization, my particular application, uh and and

process it." So, once you augment your documents, then you can say I want to execute my you know a query and retrieve the result. So when you retrieve the result, your results are going to include not only information from any global information that's available, but specifically your local information as well that it can pull in and make use of it. So so in this case we're going

to do an ingestion and then we're going to use a retrieval. So I wanted to create a little example for it to see how we can create it and I thought I would use something very specific to local data. Well, I just literally finished the conference last week. We were running that in Austin, Texas and and and the conference was focused on AI. So as I was

you know kind of preparing for that conference, one of the benefits I have from that conference or like other conferences, since I'm running it, all the information is at my fingertips. I've got all the access behind the scenes to get information. So I basically ran some queries on my system and I gathered the information I would like to use and that's what I'm going to use here

for my queries to to see how this is going to respond to it. So let's take a look at what I'm going to use here just to give you an example of it. So the first thing I have here is is the title of the conference and you know this could be any event you are running and I'm got I've got a list of speakers as you

can see and the talks that the speakers have. So I have a speaker a name of a speaker and then the you know talk title and you can see that I have a bunch of these talks. Brian will be here this week as well so you can say hi to Brian. He's giving a talk in this conference as well. But as you can see a Neil is

here as well so there's a lot of speakers you know that you can see in both events. And so all I have in this particular document is the list of speakers and the titles of their talk. That's all I have. as you can see, that's a quite a lengthy set of things. That is all the proposals and the talks we have at the conference. Then you have

this one here. I have a talk title and an abstract. That's all it contains, as you can see. No reference to the speaker themselves. So, we have a title and an abstract title and abstract and of all of that listed here. And you can see that's a pretty lengthy document as well. So, those are the two pieces of information we have. So, what is the list of

speakers and their talks, and the other is talks and their abstracts. That's basically the information we have. So, what do I want to do with it? I want to be able to perform some queries based on that information. So, how do we approach using that information in here? So, let's take give it a try. So, what I want to do here is to be able to run

some queries, but as you can imagine, this could be a lot of information that could be internal to your your organization. One of the challenges often you run into is when you perform a query, if it's a well-known information, AI can bring that information and give it to you. Again, ignoring hallucination for a minute, right? But if it's a proprietary information, what do you do about it?

Well, this is where you can feed that information to AI and say, "Now, give me response based on what you learned from this information that's non not known to the outside world, but but internal to this particular you know, application or organization, if you will." So, let's see how we can proceed with this in in this particular context. So, I want to first of all start with

an a little example here. Oh, before we go further, you can see I also in addition have an embedding uh mini LLM as well. So, you can use a mini LLM if you would like to. So, in this case, I'm going to say that I want to create a embedding model in this case is equal to new. And this is going to be a embedding model that

I'm going to bring in as an object. So, you're going to use a global LLM or or or a local LLM, the doesn't matter, but you're going to use a powerful LLM, but this is also going to be a mini LLM that's going to be used for really processing the local data that you're going to ingest. So, in this case, I'm going to say embedding store is

equal to and new. And I'm going to simply bring in an in-memory store in this particular case. I'm going to bring a text segment in this case and use it. Before we go further, let's step back for a minute. This is a very simple example. I'm going to use an in-memory embedding store, but let's be a bit more realistic. There are two kinds of information you may

be dealing with. One is you might have, you know, for example, travel policies, right? So, an airline may have a travel policy. That policy can change over time, but that policy is the same for anyone who is going to be booking. It doesn't matter whether you make a booking or I make a booking, the travel policy is the same. So, you don't have to re-ingest this over

and over and over. That makes no sense. So, you can ingest it once and until it has not changed, you can just use the ingested information repeatedly. So, for that reason, you can keep them completely separate from each other. You could do the ingestion separately, put that into a vector database, a store, and you can then use the ingested information in when a query comes in. So,

even though in this example, I'm going to show you both, don't assume you have to do both all the time. You can do the ingestion separately and the retrieval separately. But, there are times when your ingestion may be very specific to a particular user as well. So, this could be, for general travel policy, but it could be possible, but the policy may vary depending on the status

of the traveler. So, a traveler might be a person who is infrequent traveler, or the traveler may be somebody who has a status with the airline, and and the rules may be different for some of these. So, part of your ingestion may be specific data related to the request of the user who is currently logged in and using your application as well. In that case, you might

have to really ingest some information that's specific to the user along with the information that's been ingested, which is common to everybody. But, this is where you can have multiple pieces of information that can come together, and you can make use of them as well. So, there are a few different variations that you can benefit from in this particular case. So, I'm going to say in this

case an embedding, we'll call it as embedding, you know, embedding. In this case, we'll call it as a store, we'll call it as the ingestor, and the ingestor is going to help us to really ingest the data from whatever documents we're going to be bringing in. So, I'm going to say ingestor, and you probably know where what's going to come next, the builder in this case, right?

So, you're going to call in the builder, and you're going to tell the document I want to bring in the document splitter, and in this case, I'm going to say document splitters. dot recursive and I'm going to specify a certain size for the document that I want to really be able to read in this particular case. So what's going on in this in this particular example? So

this is going to be a text document. You saw the example of that earlier, but when you have a text document, you need to be able to ingest the document before you can make use of it. So document splitter allows you to take those document into segments and then store them in the vector store. Now this is unfortunately more of art than a science. So how you

split this can make have an impact on the results you are getting. So you may want to play with the splitting a little bit to see how this is behaving. So in this case I've said split them as 6,000 characters, but have an overlap of 100 between the splits. So the segments can be stored and there's a little bit of a commonality you can pull in. And

again, I would say play with this to see how the quantity of the request changes when you when you work with this. So then I say embedding model and this is going to be the embedding model we created up there and I'm going to say embedding store and that's going to be the embedding store we created up there as well. And then once you get those two

in place, all you are going to do is to finally do the build to get the embedding store ingested. So then of course, this is this ingester, but we've not ingested any data into this. We need to do that next, isn't it? So this is going to be speakers is equal to and I'm going to bring in a load document in And this is a function we're

going to be bringing in and this load document is going to take in that's a function with a static import as you can see we brought in and this is going to say a path of this is going to be coming from the source resources and remember this is the arc of AI speakers, right? That's the name of the file that you saw here speakers.txt that we're

going to bring in. So, I'm going to say speakers.txt. That's a file we are actually asking it to read in this particular case as you can see. So, we're just creating a you know, pulling in the document in here. Similarly, I'm going to say abstracts is equal to this is a load document path of this is going to be a source resources in this case and this

is going to become the arc of AI underscore let's say this is going to be the abstracts, right? .txt and we are pulling that in. So, basically this is pulling in two documents. Like I said earlier, this could be coming from various locations on your system. This could be database you can pull it from PDF documents, images, whatever that may be relevant. Based on that you need

to be varying your you know, way that you're processing and you're going to be storing this into a store that's going to be a changing if you will. So, we got the two documents. What's the next thing we're going to do? So, embedding store ingestor dot and we're going to ingest at this point but we're going to ingest the speakers and the abstract as well. So, we're

pulling in multiple documents into this. This is where I was mentioning earlier you could be just ingesting only a data that's common to everybody. In a later ingestion, you could be ingesting data that's specific to a particular user and then you can put them together and use them. So, there are a few different combinations in which you can use this as well. So, so now that we

did that the next step here as you can see we we have finished with the ingestion point. We are ready to do the retrieval. So, we can say in this case, uh retriever, uh we can call in the retriever and is and this is going to be embedding store contain content retriever. Uh I once again the builder as you can see and we are saying embedding store

and this is created up there at the top and by by this time we have already added the data into that store so we can simply uh use that store in here. And this is going to be the embedding model and that's going to be the embedding model that we specified up there as well. And then of course, you can say how many results do you want

to receive for this query uh that you want to pass along and you're calling the build on it. So, this gives us a retriever we can use as well. So, finally a chat model just like what we did earlier is a message window chat memory uh that we're going to bring in and I'll talk about what this means in just a minute and with max messages and

you can specify a particular value for it. Let's say 10. So, what does this actually do? So, what a a memory chat memory basically says when I'm having a conversation, how much of the conversation I'm going to remember? So, this is a window of 10. It's a sliding window. So, as you collect more of these, it'll drop the earlier ones and there's of course consequences to how

much it can remember, the amount of memory it's going to use and so on. So, you can control that along the way here as well. So, then I can finally say a model is equal to I'm going to bring in the open AI model right there. We went through that before. You're going to do the builder and API key keys. Very similar to what we did before.

You're going to specify the Pardon me, model name. And this is going to be GPT-5 _1. You can bring that in Uh, here as well. And then finally, uh, once you bring that in, you can say a dot in this case, you can specify the temperature like we did before, and finally a build uh, to create the, uh, you know, a request for the model itself. So,

now that we got the model, it's time for us to perform the retrieval uh, all together in this particular case. So, the question is, how do we go about retrieving the data in the very end of this? So, that's the very last point here. So, we can say chain is equal to and in this particular case, we can say this is going to be a conversational uh,

retrieval chain that you are bringing in. And again, a builder towards that. And you can say a chat uh, in this case memory. And we already created the uh, in this case the memory itself, the chat memory that we created. So, we can bring in the memory. So, this is going to be the pardon me, this is a memory not a model. So, let's change So, we're

going to bring in the chat memory. So, we'll come to that in a minute. So, we'll say chat model. And and the model I want to use for the chat is going to be the model we created. And the chat memory, if you will, is the chat memory we created up there. Uh, so we can specify those two in the chain. And then the content retriever is

going to be the retriever we created up there as well. And then we'll do the build on it. So, this gives us the uh, chain of using the uh, you know, larger LLM. This of course is going to bring the smaller LLM. And that's where the chain really comes in as well. So, we're done with all of that. It's time for us to exercise what you want

to run. So, what are we going to do in here? So, let's say here a loaded up, let's say in this case, the speakers and the abstracts. Right? So, we we've done it. So, this is basically uh, to your specific application. You bring in the data that's relevant to your application, whatever way is you're doing, you're interested in using that, and you can specify what you're done,

and you can perform the query for the users, right? So, we just put a little message to let us know that we have brought in those those pieces of data. Okay, so we brought in that. What's the next thing we're going to provide? We'll be very interactive with this, right? So, we can ask the user, "What would you like to know, right, you know, from we could

say the event, right? From the, you know, talks, you know, at the event." So, I'm going to ask it, "What do you want to know, user?" We can perform the query, and you can build those information in here. So, let's go ahead and provide a little, so to say, a prompt for a user to enter the information and make the request. Okay, so now it's time for

us to bring in the query, right? So, query is equal to we will say io.readline, and we can take the user's query right now. So, let's go ahead and say while and query is not equal to, in this particular case, a dot equals, and we'll say exit. So, we can tell the user, right? So, what would you like to know to talk about the event, type exit

to quit, right? So, we could give them the prompt. So, if it is not equal to exit, then continue with your processing. So, we can say, "If it's not equal to exit, do some work." So, what are we going to do in here? So, we can say, "You You are You asked, let's say, a percent S, and and specify what they asked." So, this is the query

we'll bring in, and we're going to execute this query, right? So, response is equal to and we created this particular request in the chain that we can execute it, right? So, chain. and we can then ask it to execute the query that we have at our hands. So, that's going to give us a response. We can output the response. And now that we did, we can then

output again a print and we can ask for the next one from the user and we can say query is equal to io.readline and ask for the next request from the from the user. So, let's look look through what we what we did, right? This is a bit more involved than what we saw earlier, but but a lot of things fit in together really nicely, right? So,

we first have a a mini you know, embedded model and we're going to pass this embedded model for the vectorization and for retrieval as well. We'll see that in just a minute. You can use different stores. So, if you go to the LangChain for J integration page, that's a good page to look at. So, if you you know, Google for or search for LangChain for J integration,

you're going to see a page which list a various different models that are available. So, those models can be used from within LangChain for J. And on the same page, you can also look for stores and you can look at variety of stores they have available as well and depending on your needs, you can use any one of those stores. Stores. In this case, I took a

easy approach by using in-memory embedding store. Like I mentioned earlier, if you are running through an application where you need to process a data that is across multiple requests from multiple users, in-memory store is not going to help you. You really need a persistent store and that's when you would use one of the other facilities they have and you can process to to types of stores that

are available and you can pick and choose what may make sense for your application. So then we are creating a you know, embedding store ingestor and this one is referring to how it's going to split the document. It's telling you what model it's going to use while splitting and the store where it's going to store the the vectorized you know, information into the into the system. Then

we are telling what documents to load as well. In this case we are interested in the speaker you know, speakers.text. We're also interested in the abstract.txt. We are loading those two documents. And once you do, you are ingesting those documents into your your ingestor. Like I mentioned earlier, if this information is common for anyone who's making the request, that part doesn't have to be repeated. So typically

you can do that separately and save it away and you can just reuse it in in in the later context. So you can save that effort of ingesting and again like I said, in a general case you wouldn't use an in-memory, you'd use some kind of other kind of a persistent store to keep that around. The next phase here is the R of the rag, right? We

finished the A of the rag. We are augmenting the system with the data. Here is the retrieval information. So the retriever says, I'm going to use the embedding store content retriever, but you're specifying the same store that you used up here and you're using the same model that you used up here as well and and so you're saying, I want to pull the data from that store

and here's the model to use and and you can specify how many results you want it to carry. So what's going to happen in this particular case? So when you make a request, the local the embedded LLM is going to take the data from the vector store and package it and send that information relevant for the LLM to process so that the query can come through that

and give you the response for you. So, we create a chat memory and then finally we said here's the LLM we want to use. You are specifying the model to use here as well. You can vary the models the way you want to and finally here's the chain that connects the model that we're going to use that we created up here, the chat memory we created up

here, and the retriever which is going to be the one that is going to do the augmented data retrieval and bring that in and finally we are ready to execute the query on that chain and get the response. Assuming I wrote that all properly, we can run this and see what it's going to do. So, as you can see it's firing up the request, reading the data

and it says load up the speakers and the abstracts, what would from the talks at the event and you can perform some queries, right? So, for you know, how many you know, were there at the you know, at the Arc of AI conference, right? So, I'm just querying that. How many speakers were there at the Arc of AI conference and you ask that it says there were

30 speakers, it says. And you can say in this case, you know, was you know, Simon there and you can make a query and say, you know, was Simon there? So, it says a little bit more information, right? So, it says yes, Simon was Simon Maple was there to speak giving multiple talks including context is king king and and a few different things, right? I'm like, yeah,

that's kind of interesting. There's a you know, these talks about various things related to AI and I can say who else spoke about uh context engineering. So, I want to pause right here. Notice uh the context memory we created, right? So, who else spoke about context engineering? Uh in a normal conversation with humans, we carry the context from one to the other, right? So, if I ask

you, uh you know, uh "When did you arrive?" and then I ask you a follow-up question, it's based on your travel. We are smart enough to carry that conversation. Well, in this case, I'm I'm not giving any more details. I'm saying, "Who else spoke about context engineering?" So, it needs to understand the question is about the conference. The question is about that particular event, and we are

carrying through that conversation, right? So, who else spoke And it says, uh "You asked who else spoke about context engineering. Besides Simon Maple, uh Brent Lancaster also spoke, you know, especially about context in this talk." And then it gives you a list. Uh it it actually a lot more, uh you know, talks were on context engineering as well. Uh you know, uh I'm going to say, "Was

there others who spoke uh about it as well?" Again, notice about it as well. I didn't have to repeat what I am talking about. But not being satisfied with the response, I can probe a little bit more and ask it. And it says, "Oh, yes, in addition, uh Mike also spoke about context engineering themes." Uh this is one of the challenges, right? You you're not going to

get a uh deterministic response from it. The last few times I queried, it gave a slew of responses. This time, it's like, you know, uh on Tuesdays, I'm a little lazy. I'm not going to do that for you. So, it can Maybe there's a cloud cover out there, so it doesn't want to really respond. It's very non-deterministic. You cannot really predict what it's going to do. But,

knowing what you do, you might know that it's not enough and you can query further, or you can iterate through a few things in your own application, and you can get some more information as well. So, as you can see, in this case, it says a stock making API, you know, AI reality, explicitly the context, etc., etc., etc. And then, it provides some of those information as

well. You can continue you, right? You know, what else what else did he talk about? Can you list the list the topics? So, so this is where it becomes very conversational in this particular context, and you can keep building on it as you can see, and it knows, right? When you say he, it knows we're talking about Mike, because that's the last exchange we had with it.

And so, Mike spoke on these topics, and gives you the list of topics here as well, and it's able to list some of those talks and and bring that into the forefront. So, so that is basically example of, you know, rag ingestion and retrieval as well, and you can use your own data. That's basically what you're doing here is to bring that in. Like I said, the

data can come from multiple different formats and media, right? So, it could be, for example, it could be based on a PDF, it could be based on images, it could be query from the databases you pull in, it could be a bunch of JSON data, could be text data, whatever it is. There are different kinds of ways to consume that information. And then, once it's vectorized, you

can perform the query on it, and you can get the data. What else is available besides what we just talked about? So, so at this point, you can see we are able to run the rag queries using the LangChain for J. So, as uh as I mentioned, right? There are various stores, various types of memories you could be using to bring this data in here as well.

Um so, we were able to perform general queries with it. We are able to perform augmented queries with it. Uh LangChain 4j is evolving as well. You could also use LangChain 4j to uh you know, query and interact with uh agents as well. So, agents can be a couple of different types of agents. Uh so, this is the funny part in our industry, right? We can do

the same thing we do, but we can give a new name for it and get excited about it. So, all your web services and all your services in general, we don't call them services anymore. We call them agents because that's so cool. Uh and then we call it non-AI agents versus AI agents. So, your application could be a mixture of both AI agents and non-AI agents. And

in this case, you can ask LangChain 4j to perform the query and flow through a bunch of agents. And the agents, like I said, could be AI or non-AI. It doesn't really matter, but it gives you this common API to send queries to these different agents. And if you want to implement an agent, there's an interface that you can implement, and you can say, "Hey, I'm an

agent." And you can perform a query. Here are the you know, calls I will receive. You can expose the functions that are available, and so you can implement it. Then you can define the chain to follow, and it can go through all those calls to the agents and get the response for you, and then you'll get the result. So, in in summary, what uh LangChain 4j basically

does is it gives you this ability to uh programmatically interact with uh you know, n number of models behind the scenes. It allows you to deal with your own local data as well or our custom information and also allows you to deal various scenarios or flow of execution. So, if you're programming in Java and you want to be able to make full use of AI within your

application, then LangChain4j is one option to do it. Uh you can also use other solutions. This is not the only solution available in Java. So, if you are a you know shop where you are implementing different solutions and you want to be able to use AI within your products, you can use LangChain4j. Uh if you are using Spring, then you can either use or you can use

Spring AI also. Uh I'm sure there are other talks this week on Spring AI. I also have a talk later on in the week, I think on Friday. So, you if you're more specifically using Spring, uh look for Spring AI talks as well and you can see how you can make use of that in Java. So, from Java you have multiple different ways to access it. Uh

so, what it really boils down to is your language is not programming language is no longer a barrier uh to use AI within your applications. So, it doesn't matter what programming language you use, it's a question of figuring out what uh libraries are available and uh you know as I said, LangChain was predominantly implemented in Python, but you have that available here in Java as a way

to implement the LangChain4j and you can benefit from that for your applications as well. Hope that was useful. That's all I have. Thank you. >> [music] >> Oh.