Natural Language Data Access in Java with Hibernate, Quarkus, & LangChain4j | Marco Belladelli (EN)

This talk, presented by Marco Belladelli, discusses the integration of natural language processing with data management using Hibernate ORM and LangChain for J. The speaker explains the challenges of mapping relational database data to Java objects and how Hibernate serves as a bridge between these two paradigms through Hibernate Query Language (HQL). He introduces LangChain for J as a tool that simplifies interactions with AI service providers, allowing Java applications to process complex queries using natural language. The presentation includes a demonstration of a Quarkus application, showcasing how AI models can generate and execute HQL queries based on user input while maintaining security and validation protocols. Overall, the speaker emphasizes the significance of HQL's expressiveness and how it enhances AI's ability to comprehend and manipulate structured data.

[music] >> Thank you for coming. Um my name is Marco. Uh as you can see, Marco Belladelli. I'm from Italy. And today I will talk I will teach you, I hope, a little bit about how to talk with your data uh through natural language using some of the nice frameworks that that are listed there, okay? So, first of all, as you have already seen from other speakers,

you can use the this QR code. You will get a link to the Slido website where you can leave questions for me for later, but you can also, you know, raise your hand at the end of the presentation and I'll I'll will answer, okay? So, I was saying as I was saying, my name is Marco. I'm a software developer at IBM, and I've been maintaining the Hibernate

ORM project for almost 4 years now. And I also contribute to other uh open source projects uh like Quarkus and LangChain for J. You can find my uh my GitHub profile and my email address. If you have any doubts after this talk, feel free to reach out. I'll be happy to reply, okay? So, quick questions. Uh who knows here what's uh what an ORM is? Raise your

hand. Okay. Uh who knows what the Hibernate is? Okay, almost the same, guys. I'm I'm glad. So, just quick recap. I don't need to spend a lot of time, but uh we use ORMs to deal with uh relational databases. Um and the problem with relational databases you is they represent data and they store data in a tabular format. Uh you know, the the example that we will

we will we always make is the spreadsheet, right? Tabular data might look like this, uh which is kind of easy to understand what what it is about, right? It's company and some date and some country code, maybe. Uh but it also can sort of like this, like a bunch of millions and millions of rows of unstructured data that you need some some middleware, some software to to

do a first first processing of before humans. And the software we write is in Java. Okay, you're here. It's a It's a Java conference. And what's the problem with Java applications and tabular data? They don't really know what that is. Like Java likes to represent objects in a in a class structure. Like we have our properties, our types, and we have associations, we have collections, and that's

not something we can represent in a tabular format like in a database, That's where Hibernate ORM comes in. So, it tries to bridge the gap with it between these different paradigms, you know, tabular data and and objects and Java objects. And for if anyone doesn't know, one way you can interact with your database through Hibernate is HQL queries, which is Hibernate Query Language, or you may also

have heard of JPQL, which is Jakarta Persistence Query Language. The the spec that we that Hibernate implements is called the Jakarta Persistence spec, and it has their own version of of the language. Hibernate translates those queries to SQL queries. We extract the tabular data and we map it back to your Java objects so that your programs might easily consume them. So, I've briefly talked about what Hibernate

is today. I'll be using a couple more frameworks in my presentation for today. One is Quarkus. A lot of you have heard heard of it. A lot of you use it, I know. For anyone that doesn't, it's kind of like Spring, but a bit better, I would say. And then there's LangChain for J, which is a newer kind of framework that helps us when we have to

deal with AI services. And you know, I know one of the maintainers of the LangChain for J project is called Mario Fusco. He's a fellow Italian. I've met him a few times and he always goes to conferences. And when he does his presentations about LangChain for J, he he always likes to present it as the Hibernate of AI. Uh which yeah, I guess it's a kind of

a good very very easy definition. So what Hibernate does other than map our tabular data to to object objects is it allows you to not care about which vendor of database you use. You don't care which JDBC driver you're using, which specific functions or or or versions or or or semantic or syntax that you need to use for your database. Hibernate lets you uh interact, you know,

regardless with its own APIs and you don't you don't need to care about it, right? That's kind of the same thing that LangChain for J does. Um But with AI providers, AI service providers. So if you're using OpenAI's ChatGPT, if you're using Google's Gemini, if you're using Claude or Granite or whatever kind of model, um LangChain for J probably supports it and you can interact with it

easily in your Java app in your Java apps. But so AI models, um do you think AI models could understand tabular data easily? With this example, yeah, maybe. I mean, models are are getting bigger and bigger. They can understand probably what what this data might represent. But if you remember the first slide, you know, the unstructured data that the massive amounts like billions billions and billions of

rows, they might not not make sense of them, right? Um can they understand Java objects? Well, maybe you could try and serialize Java object in some format and and pass it to your AI model, but then uh you know, dealing with Hibernate objects as we have in maintainers know is not that easy. Um there's concerns of uh you know, uh for example, uh circular circular associations, lazy

associations, all kinds of problems that you might run into. So, it's not actually that easy. And also, AI models don't know what the hell all this data represents. Uh you know, at the end of the day, uh they don't really know what a company is, what why it has these properties, and what they represent. Right? So, AI models, they were trained with text. They work with text.

Uh they produce and consume massive amounts of And that's what they know, okay? So, if we if we talk about context, uh we talk about context engineering, there's been lots of interesting talks at this conference about it. It's all text at the end of the day, right? And uh do you know what has a lot a lot a lot of text? So, this is the Hibernate user

guide, and um as of uh I don't know, 1 month ago, and uh this is me scrolling at a at a at a certain speed. Like, you you couldn't read the text on the screen this fast. And uh yeah, there's a lot there. And also, Hibernate has been around for like 20 years now, and it's been open source for all the this time. There's been people blogging

about it. There's been uh you know, questions on Stack Overflow, forum posts, uh issues. And all this stuff is public, right? It's on the internet, and models were trained on them on this stuff. And so, they they know about it already. That's where we thought, what could we do to bridge the the gap? To to help AI enter the world of our uh relational database data and

of uh structured data. Well, um two things. Uh so, Hibernate already knows the shape of your data. They We know about the classes that you use the JPA uh annotated classes that you use to map your uh, relational database data in your application. And we know what their properties, the the types of the type of the the the the attributes, you know, we know everything. And we

can let LLMs know about your uh, And the second thing, which as I said before, is not trivial, is is uh, serializing query results. You know, letting the the model know what you extracted, the query you executed, what it returned. And uh, yeah, you'll see briefly what that helps achieve. So, I think we can switch to my IDE so I can show you something. So, what you

see here is um, a Quarkus application, all right? If you're not familiar with with Quarkus, this is uh, this is it's application properties, just a few configuration properties for our database connection, and uh, this is configuration for LangChain4j, right? So, as you can see, we are connecting to uh, a locally running old Llama. If anybody has heard uh, is hasn't heard of it, it's like a small

program that lets you run uh, containers with LLM models running on your local laptop. So, everything you'll see today is running on my laptop, and we're using an open source open weights model, and yeah, some other configuration for logging and debug. We have a few entities, uh, like the company one I was showing you before, right? Which has uh, as a simple name, it has an address,

which is actually a a component a composite property, so it has three sub attributes, and it has also an association with the employee entity, which also has its own uh, stuff, right? And so, what can we do with this? Well, little spoiler, but uh, I built this uh, so this UI, I totally built it myself, not using a coding assistant, in a couple hours. And I can

ask it, "Extract all employees." Oh. And it thinks a bit because yeah, the model is running on my laptop and would you know it, we get a table with the list of employees, right? Um so we have there's me, there's some of my colleagues, you know, this is just some demo data. Uh I can explore these. So, I mean, the UI is is not great. I I

lied. And I made this, I didn't I'm not a front-end engineer. But um as you can see, we can navigate the properties. Uh we can navigate associations, we can navigate uh embedded properties, right? And just so you believe me and you believe I didn't lie to you, uh this is the insert.sql script that I used to initialize some of these data. So, as you can see, it

was pretty easy, I just wrote an English request, so natural language, and I got some results back. How did we do it? Well, simple enough. So, this is the log of the HTTP request going to my model, okay, to my Ollama model. And I remind Let me remind you this can be your OpenAI, your um Gemini, your Granite, your whatever model you're you're using. >> [snorts] >>

And yeah, what the new module called Hibernate Assistant did, uh it produced this JSON here, right? Uh if I format in a pretty way, you can see I already had it actually. Yeah, you can see it contains information about your classes. So, there's the address class, there's the embeddable, there's the employee class, uh and you see it has these its properties, their types, and uh yeah, all

the other all the other good stuff that you mapped with Hibernate. So, as you see, it's all text. And then we asked it extract all employees. This is my message. The model itself produced an HQL query, so Hibernate Query Language or Jakarta Persistence Query Language query, which quite simple, right? And then Hibernate translates it to um to SQL. We then execute the query and there you go,

we have a result. How did we take advantage of LangChain for J's APIs to do it? Well, it's quite simple, but just to cut short, we have a a meta model prompt template. So, we are we are telling the AI, "Hey, you are an expert in writing Hibernate Query Language queries. Please, uh create some selection queries for us." And here where you see this placeholder, we actually

we actually injected the meta model serializer um JSON that I just showed you. And this is where the Hibernate Assistant module comes in. It lets you initialize an AI's context with relevant information about the shape of your data, so it can create relevant queries when asked to. Right? >> Okay, let's move on. So, the here I had the same example, but a little bit very, very slightly

more complex with uh yeah, extract all companies that with a name starting with A, and so it it put, you know, a where clause there. Why all these idea could not be that great? Uh I don't know if you've heard about these a couple of two or three weeks ago, actually. There was this news around the internet, it was everywhere really, that something, someone using a certain

very powerful coding assistant had access to production database of 2.5 million or billion rows, I forget. And yeah, they had no backups, and the coding assistant dropped their entire database. Right? So, databases and AIs, you you know, concerns start to appear when you talk about this stuff. Uh and you you'd be right to be concerned. So, why why should you use Hibernate when interacting with databases uh

maybe through LLMs? Well, we There are some advantages. So, LLMs can only access the entities that you map with your Java classes, right? Your JPA annotated classes. They cannot access the entire database. So, don't even need to care about restricting at the user level. Um HQL queries are validated early. So, they fail before having to reach the database and doing a full round trip. Um the error

messages we produce are much nicer and much closer much more understandable to uh to the user and also to an LLM. Trust me, I know. We support more than 13 or 14 databases and uh yeah, we see the the error messages that some of them throw like error minus 422. Go search on Google what that means. Yeah. Uh Hibernate works across all database vendors, so you don't

need to worry about where to you're deploying your applications. And HQL is a further abstraction level up from SQL. And sorry, you can express much more complex queries very very easily uh through normal you know, association through normal Java syntax kind of um And so, it's it's a lot easier for the LLM, which knows natural language very well, to create you know, interesting queries. Right. We get

to rag. Who knows what rag could mean in this context? You, sir. Yeah, of course I do. Someone wants to answer? No one? You. Yes. It's not my grandma's cloth that she used to to clean the the floors, okay? Uh rag is retrieval augmented generation. So, I know you've probably you might have heard of it already or just a bit shy or just tired because it's after

lunch. I'm sorry. Um, but so rag is is a popular method to enrich the LLM and LLM's responses by basing them on actual data on on real data that you that you might have in you might already have in your domain, right? Um, so does anybody know what we might use for rag? The new Hibernate assistant module. So, let me show you another thing I did. I'll

refresh the page so that get a clean slate and I'll ask um how many USA companies do we have on record? And this time I'll use the chat icon. And the model thinks a bit. It might run a SQL and HQL query. And there you go. We have five USA companies on record. Anybody want to guess how we got here? Like how this how did happen? Did

this Sorry? Yeah, so we ran a a select count query. Let's see if that's that's the case. Uh, let's go back to the logs. So, this is the query from before. This is our most recent request, right? And you see the LLM generated this HQL query. Select count C from company C where the country is USA. So, you see this for example is is a good example

of of uh the Hibernate query language over SQL. You can just navigate through um composite attributes or or or even associations really using the dot notations and notation as if you they were Java objects, right? So, easier, more more more intuitive than than SQL. Then, well, Hibernate translates it into SQL. In this case, it's very simple, the SQL itself, but it's not always the case. And then,

we ran the query and yeah, we said, "Okay, LLM, please answer the original question um, on the following data." Result of executing the HQL query we just saw, it's five. What did it say? How did it know it? Because we had them on our database. We had them mapped through Hibernate or through JPA, um, in our because we already used it maybe in our application. So, we

have a Java class that represents a company and and we just let the LLM know uh, what the our database contains. So, here I have a slightly different example. Once more, how many USA companies with at least two employees? And yeah, you see it it also generated uh, the correct uh, query where you see uh, another syntax which you can do with with HQL, which is the

size operator on a one-to-many association. And and you know, be careful because that c.employees is is not a property. It's not it's not a simple thing in a database. It's a it's a foreign key, right? Maybe you have a joint table. Um, so, this is very simple for an LLM to understand. A big query with three or four joins or some weird operator to uh, to extract

the results for you might not be as easy. So, um, here is a quick uh, mention of a couple things we you can already play with um, if you use the LangChain for J Hibernate or the experimental version. Uh, so, these are two contributions, me and my colleague here, Christian, made uh to the LangChain for J library. And one is the embedding store integration, which if you

know about vector databases, um they can contain our embeddings and then you can use them to do traditional rag uh against a vector store. Uh you can use our own Hibernate extension to interact with any relational database which also supports vector stores uh like PG Vector, MySQL, DB2, Oracle. I mean, most of them, the the the most recent version they support, or you could use the experimental

feature uh which is called Hibernate content retriever. And the content retriever API is exactly an example of how the rag uh pattern I just showed you works. Okay? So, it it it's basically a built-in implementation that you can you can pass your Hibernate metamodel to and then it will allow you to do exactly what you saw in the query just just before. So, if you want, you

can have fun with Uh let me show you another thing you can do. So, let's say um Let's actually ask, yeah. Extract all employees of the Red Hat company. So, yeah, I made these slides uh when I was still at Red Hat, so yeah. Uh I was still still working there. And we got our result, right? There's me and my colleague Andrea that we were working there,

okay? This is just demo data. And then I can ask, uh you know, what projects are these employees working on? And this time I used the chat icon again. It thinks a bit. The employees of the Red Hat company are working on the following projects, Hibernate and Quarkus. Seems Seems legit. are there any other employees working on these projects? So, you see, I'm just interacting with a

chatbot. This This might be your uh you might be accustomed to using ChatGPT, to using, you know, whatever uh whatever chatbot you can find on the web, but the answers I'm getting, they're based on the actual data contained on my Uh you know, this is uh this was relatively easy, as you can see, and um uh this is all based on on on reality and on my

Java application, which is in control of the data. And uh and I can decide which which LLM to route this data through, uh which, as I'm showing you, this is all running on uh on a laptop. These These These LLM models that I'm running are very small. They're very fast. They don't take much to reply, right? And still, they're producing uh Hibernate queries um correctly, and they

are also extrapolating data uh with ease, right? let me show you a bit more around the code, how I managed to do this. So, um all these uh nice requests you were seeing uh we were making through through the um through the chatbot interface was were going through this assistant resource, which is an annotated uh Quarkus endpoint uh uh REST endpoint, Uh uh as you can see,

there's a query to JSON method and endpoint, which uh invokes the assistant.createAIQuery method with your uh user message. And if you look at how this is was implemented, pretty simple. Here we have um you know, we serialized our our meta model through through the meta model serializer as I've shown you before and we with that we created a system message. System messages in the LangChain for J

world are context initialization prompts that always stay at the top of your context and serve the purpose of letting the LLM know how it should behave, right? So they are important messages that are always kept in memory. we we simply add your user message to the memory, right? We say, "You're an expert in writing queries. You have access to this kind of data." And then the user

asks, "Extract all employees." Great. We send these these messages to the LLM. The LLM responds with some text, right? The text that we get back is then used to create an an Hibernate query, a selection query. This will be important later. we execute the query to JSON and how does that work? Marco and yeah, we use the results JSON serializer implementation built into the new Hibernate system

module to natively transform your results to JSON. JSON is LLMs understand text. That's why this all is so easy. Um and then yeah, with that text we might choose to just return it um to our to our endpoint to a rest endpoint, right? And just display it in a table format as I've shown you. Or we might do the next step, which is take a natural language

request, execute a query, and then pass the results back again in JSON format to the LLM once again and tell it, you know, produce a natural language response back so that the final user doesn't have to look at a complex table and doesn't have to understand the raw data. Uh you know, you have a rag pipeline that that produces natural language responses. And what what I wanted

you to to note looking at all this code is this is very simple stuff, right? It's not many lines of code. Okay? So, to to create the query itself, the the core logic is I don't know, 20 lines. And this is the to create to make an LLM create an averaging query. Right? So, this is not much. You can add on top of this whatever you want

you'd like, right? You can customize all this logic and that's the good thing. We give you the building blocks to do this sort of interactions with LLMs. So, you can get the queries and execute them. And then you your program, your Java application, your web server, your whatever you're using can can inject whatever they want during this whole process in any of these steps and and and,

you know, process this data, validate it, restrict it to to some to some degree, right? Yeah, this is just the example I've shown you. Let's move on. So, MCP, model context protocol. This is also another very powerful tool set of tools and trend that's going on in the AI world. Uh everybody knows what MCP is? Let's just show some hands. Yes, so you listened to a lot

of the sessions in this in this conference and I'm glad you did. So, just to to keep it short, MCPs allow, you know, AI providers to to connect with external tools that might, you know, be deterministic or or not, but they they act as an API endpoint for for our to integrate with external components. So, as uh as a set of tools, you know, for for our

builders or uh for ourselves, MCP acts as a set of tools that we can make uh at the at an LLM's disposal. And uh yeah, well, Quarkus uh Quarkus makes very makes it very very easy to create MCP tools. How, you might ask? Well, you can just annotate uh some methods in your Java code with the @tool annotation, right? And then you give it a name, and

you give it uh you know, some description so that the LLM knows what the tool is about, why you might call it and and and when. So, for example, as you see, I have this simple tool Hibernate get metamodel that uh the the the only thing it does, it it returns the metamodel JSON representation that I I used in the in the in the code I've shown

you before to just initialize the LLM's context, right? This This is a tool, and and we read it as a tool the LLM itself can decide whenever it needs to access your Hibernate data and look and and understand the shape of the data, right? So, another tool might be to right? And get the JSON back and and do some sort of automatic rag pipeline, or or even

just generate the natural language response directly to be more informed uh about your data, right? So, Quarkus gives you all the uh you need to expose these functionalities uh through and and the MCP protocol, and and I bring the uh lets you very very simply implement them. So, I have the application running, and I can show you I like to use this this very neat tool, which

is by the the MCP creators themselves, which is called uh metamodel meta model context protocol inspector. So, I just started real quick just to show you. And it opens my browser and it's this very nice interface where you can connect to an MCP endpoint. In this case, it's my localhost Quarkus application that was already running from before, right? And I can ask a list of the available

tools. This is just the hybrid ones. Of course, here you would have all the tools that your application or your MCP endpoint exposes. And then I can, for example, run a tool. And I ran the get metamodel to get the JSON representation of of my of my data. And here you see the result, right? So, it's running. It's exactly the same thing I've shown you in the

Quarkus But this this the the the good thing about this is the LLMs themselves can can decide when to invoke these tools. Okay, so this functionality comes pretty much for free. So, what did we achieve? Thanks to the power of LLMs and and modern AI that can easily understand, you know, text and natural language, a new module which lets you convert user prompts which again are natural

language, are plain English. Or whatever language actually. They translate them to HQL queries. So, any person, no even if they are non-technical, they don't need to know about your how your tables look like, right? Hibernate already took care of that. They don't need to know SQL. But now, they don't need don't need even to be Java developers. They can be any level of person. It can be

a I don't know, an HR representative or or or a business analyst looking at some statistics about the orders for your company. In the examples I showed. And they can just interact with your existing data, your actual real data on your database through natural language. And you can present such data yourself. So, you can uh choose to uh get the data, uh get a SQL query executed,

get the data back, and and do some post-processing, or use it in in your own business logic. Or, you can choose to implement a sort of rack pipeline like the the ones the ones I showed you uh to give natural language replies we've learned today that uh Hibernate what your domain looks like. We already if you already have a Java application that uses Hibernate or Spring Data

JPA, or just any JPA uh user application that uses JPA we know what the shape of your data, uh the attributes, the types. HQL is much simpler than SQL. It's more powerful, it's more expressive, and most importantly, in this context, it's closer to natural language. So, results of LLMs producing natural such natural language are uh much more correct most of the time. Queries, complex queries are very

much more simple for the same reason I've I've just uh told you about. You can augment and create rack pipelines to augment uh LLM replies with data based in on reality, based on your actual database contents. MCP makes it extensible, and it's very easy to implement such MCP tools. Uh it's really at your disposal if you're using Hibernate. Uh and you you look at the new module,

and this is very very easy code. Right? Th- it's all it is. I don't have any more like hidden logic somewhere. Uh the the only new thing you need to add to your application would be the the new Hibernate Assistant module. So, how can I can you try this uh this new stuff? The new Hibernate assistant module is available since the ORM version 7.3, Hibernate ORM 7.3,

uh which is a relatively recent version, but it's out there. The code is out. The the the Maven artifacts are out. You can you can use them. Quarkus natively integrates with both Hibernate and LangChain for J. You need to go to add the Quarkus LangChain for J extension if you want to use the cool features like the chat model and the chat memory that I've shown you.

LangChain for J has the benefit other benefit that it supports any LLM provider. You don't need to um switch to to a custom one. You can also uh choose where to deploy your application uh at a later point and you will be able to use any any provider, right? And you can also add some MCP servers or the Chappy extension that I will tell you about if

time. So, and the last thing I wanted to leave you is is the code that you've seen, so you can play around with it and use it uh run the application yourself, uh try to run some queries and ask the uh you know, the LLMs if they can retrieve your data. Um it's all open source. It's under my uh username and I also left a couple references

to the Hibernate and Quarkus websites where you can learn more about great projects. And with that, we have a bit of time left. So, I'll look if there are any questions or if anybody has a question, they can ask. And if we have time at the end, I will show you another cool thing. Sorry, access control? >> Right. Yes, so you're asking how to how to restrict

access to your data and and keep it safe, not give it back to the Right. So, yeah, it's it's one of the the great advantages, I believe, of of Hibernate over SQL. It's Well, for one, it all already maps to a subset of your entire database data, right? Uh you all you have only mapped the company and the employee table, like the ones I've shown you. If

the LLM tries to run a select query on I don't know, the orders table, which you don't have mapped as a Java class with the JPA annotations, the Hibernate Query Language processor will tell you "Error. We don't know about this entity. You cannot select from it." So, we are not running uh native SQL queries, we are running Hibernate or JPA Query Language queries, and they are validated

against your Java classes. And that's the main that that's the existing benefit. The added benefit for this is you can create dedicated metamodels, so a dedicated set of annotated classes, uh which is more restrictive than maybe your normal existing applications model, uh to give into the LLM, because you know that, for example, those tables don't contain sensitive data. They don't contain personal information or uh GDPR protected

information, right? And you can give those to an LLM, even though it might not run in your local domain. And finally, Hibernate, yes, has some built-in uh restriction functionality, like the filter annotation, the SQL restrictions annotation. You can set custom filters, uh even SQL expression to manually filter your uh your uh tables uh whenever they are selected from. So, yeah, there's a bunch of tools at your

disposal in the Hibernate world for that. Okay? Yeah. Thank you for your question. Any more questions from the room? You, sir? Mhm. That's a great question. So, he asked Sorry, and I forgot to repeat his question, but I guess you all guessed what it was about. But, his question is about any any other ways we can restrict uh results, but this time in a in a performance

or in a cost-related uh manner, right? So, yes, uh Hibernate has functionality like pagination. Uh you can restrict endpoints to select, you know, only 10 records at a time. Uh or maybe you you create a query, you you first ask There's an a Hibernate API called uh get result count. So, you create a query, and then first you extract the result count, and you see it's Oh,

it's 100 million rows. And you tell the user, "Hey, please restrict your search criteria uh so that the LLM produces a more restrictive query that extracts less data that it can then interpret and and then you can show to your user, or you can feed back to the LLM without burning your tokens for the next 2 years." Okay? Yeah. Thank you for the question. Was there any

others? Yes. So, yeah, to the latter question, absolutely, yes. Uh so, LLMs are not perfect, and they absolutely will produce uh wrong stuff. Uh they might, you know, confuse HQL with SQL. They might just not understand the the the the query that you're that you're that you're asking on, or you might just ask to extract a different attribute that doesn't exist. But, that's when the failure validation

and the HQL validation that Hibernate does and the clear error messages come uh come in, right? Um and yeah, I the the question about how how good they are at producing producing SQL, as I said, Hibernate has been around for a long time. There's been a lot of content written about Hibernate on the web. And so, they are actually quite good. Uh from my initial experimentation, I

was blown away but how by how not the bigger frontier models like the very very big uh service uh AI services that that are exposed by vendors are are good at that because of course, as the the the better your model, the bigger your model, the better results you can get. But even the smaller models like the the ones I've shown you today, uh it's uh it's

called Qwen 3. It's just a 30 billion uh parameters with active 3 billion, so it takes like very very little RAM, like 4 and 1/2 gigs. That's very That's very small and that's very fast, right? It's just running on my laptop. And uh yeah, even just with that, which is a coding uh model, I always got very good results, even for very complex queries. Um and SQL

makes complex query queries easier anyway, so Yeah, you sure have a great idea. So, he asked, uh can we inject more information into the context to help the LLM understand not only the technical side of our classes, of our mapped models, but also, you know, the business side, the business reason why a date might have uh you know, might not have milliseconds or why a country code

might be three letters long. Uh not right now. As I said, this stuff is quite new. It's the new model has just been released. Uh but we are we are working on that. We are planning on possibly integrating a bunch of information. For example, if you have information on the on the column size of some, you know, of some property, we might add that so that the

LLM can expect which kind of information is there. We are We are thinking about adding bean validation information, which is another Jakarta specification, which is used to again validate your data in in automatic fashion. You can have a description there, and so we might add that description to the property description so that the LLM might know what to expect in the property or uh yeah, we are

thinking of adding more context, more information to the context of ways that you can add custom information there, but it's in development. And it's also open source, But that's a great idea, yes. We are actually working on that. Thank you. At the end? You, sir? Yes. So, I can show you His question is, "How did you How can you extract a structured HQL query?" So, something that's

that must expect a certain format uh from a natural language response that the LLM may produce. They The LLMs just produce texts, right? So, that's a great LangChain for J lets you do that in in a few ways. The best way I found, which is the by far the most reliable, is using these response format API. This is one of the LangChain for J's APIs, which basically

creates a JSON schema. Uh if I'm If anybody's familiar with what JSON schema are, which is basically a description, right, of what a JSON JSON reply from the LM should look like, and we can say we can build a JSON schema that has a single parameter called HQL, and and then we instruct the the model with our system prompt, so the initial prompt, to just create um

the HQL statement without any comments, without any additional description, anything. So, that's This is the part that's non-deterministic. Uh, so what you said is is great is correct, of course. So, LangChain for J has a few tools to let you play work around that. LLMs that support JSON schema are becoming more and more. For those that don't support it, you'll need to do some sort of context

engineering and pattern matching, possibly, yes. Thank you for the question. Do we have any Slido questions? Sorry. Let me I lost Don't know why it doesn't show me the link. Do you have it open? Yeah? Sure. Oh, we have a lot. Sorry, guys. So, I'll start from the top. How safe is just exposing an endpoint that can create a query? Well, That's That's what I was talking

to you about earlier. So, there are ways uh, to to limit this as as much as you can. You can also do some custom filtering on your side if you wanted to. We are also thinking of adding more and safe functionality safe mode functionality to to the hybrid model way of working exactly for that reason. Okay? But, you can find me around the conference, and I can

tell you some strategies around that. Safety mechanism for auto-generated queries. Yeah, it was a similar question Um before. So, I think I've already answered it. Um can this tool perform data analysis as in data anomalies, record completeness, duplicate Absolutely, yes. So, what I've shown you is just very very simple question and answer stuff, right? This is just a chatbot interface. Nothing is preventing you from plugging your

own custom logic in any step of the process. And what another thing I wanted to teach you, which you might not have realized looking at all these AI-related talks, is AI just work with text. It's not magic. It's text. It's information. It's context engineering. So, you can do whatever you want with your context and with the data that Hibernate helps you extract from your database, okay? Uh

very quickly, multi-tenancy. Does this also work with multi-tenancy? Yes. So, multi-tenancy is a way to restrict data based on on a some sort of magic value, right? Or a or a connection or a specific database user. Absolutely. So, you can restrict your model that way as well. Are there solutions for integrating mixed data sources like databases and then API? Yeah, LangChain for J has those solutions like

the vector stores integration and the rag patterns. You can have more than one in the same Quarkus application. So, you can have more than one content retriever, the the rag API in LangChain for J is called. Uh your colleagues will not like the answer. Why Qwen Qwen 3 and not Granite 4? Uh Qwen 3 coder is a coding-specific model. So, it's been fine-tuned to do well very

well at coding and it can produce much better HQL queries than a generalist model. Okay? And also, Qwen 3 30 billion active 3 billion for the AI model LLM models nerds, which might not be any one of you. It's like the perfect fit for my MacBook. So, it just runs great and gives you the best results. Um so it's just a coincidence. In your demo for rag,

you didn't use any vector search. It It's Hibernate ORM is doing this process under the hood. No, so vector search was was not involved in any of this. If you want vector search though, Hibernate vector is the extension the module that lets Hibernate interact with that Christian wrote. Mostly yeah, it's the module that lets us interact with vector stores. And the Hibernate embedding store LangChain for J

integration lets you do that seamlessly, you know, automatically. And lastly, isn't JSON too verbose? It will It really dump a lot of context. Are there alternatives? And CVS or I guess CSV comes to mind. Yes, you're already talking my language. You're already thinking about how to optimize the context or the your data representation to fit it into more context and fit more data into it. So yes,

I I I can't wait to hear what the community says about that. There are strategies. I can suggest some of them, but also the industry is trying to find those strategies to make context bigger, but at the same time optimize them more and to fit more stuff into them. So I I'm run out of time. I'm already over time. I wanted to quickly show you the Quarkus

dev UI and how all this stuff I've shown you already is available in the Quarkus dev UI. If anybody doesn't know, the Quarkus dev UI is a nice web interface that gets started automatically with your Quarkus application. I'm not even German, so I don't know how that happened. Uh Christian, it's your fault. Um so the Quarkus dev UI is started automatically whenever you start your application in

dev mode. You as a developer can interact with the chatbot very similar to this. Play around with your HQL queries. Create uh ask the LLMs to create HQL queries for you and then reuse them in your own application. Okay, so check that out and if you want more to me to show you more, just ask me. Okay? Thank you so much. >> [applause]