NODES AI 2026 - Graph-Powered Storyworlds: Using Neo4j To Keep 1M+ Word LitRPG Epics Coherent w/ AI

This talk discusses the creation of an AI-assisted authoring platform for LitRPG novels, a genre where characters operate in game-like worlds with complex stats and narratives. The speaker, Prem from ThoughtWorks, addresses the challenge of maintaining consistency across expansive texts utilizing large language models (LLMs). He explains the limitations of LLMs in tracking essential narrative details and presents a solution: implementing a knowledge graph that functions as a memory system for the narrative. This architecture enables automatic extraction of elements from the text and flagging of inconsistencies. Additionally, it utilizes Neo4j to manage the schema and relationships essential for narrative coherence, allowing authors to maintain control over their stories while leveraging AI for assistance. The talk highlights the complexities involved in the extraction process and the need for a human-in-the-loop system to ensure accuracy.

Okay. Well, we are talking about graph powered story worlds. That's the fancy title that I came up with. Basically, my name is Prem. I work for a consultancy called ThoughtWorks. I play the role of tech director, but it's basically a glorified uh a title for developer, I guess. So, I've been helping build this AI assisted authoring platform for lit RPG novels. Now, if you don't know what

lit RPG is, it's short for literary role-playing games. So, so lit RPG is this this genre where characters live inside game-like worlds. They have stats, skills, inventories, quest logs, level ups, all sorts of fancy stuff, all embedded right in the prose. I think Ready Player One meets Dungeons and Dragons. The author writes the story, but they use AI to keep it consistent across a large body of

text. And that's the challenge. So, let me show you what we did there. So, you ask the LLM to provide you with some idea, to maybe finish a chapter. You're stuck with something. It happens to authors a lot of times. And it writes something perfectly reasonable. Until you remember what happened 20 chapters ago. Marcus lost his right arm. The LLM forgot. This isn't This isn't hypothetical, okay?

It's from a test manuscript that we built when we were testing out the system. So, this happens to LLMs all the time. They hallucinate, they make up stuff, and uh uh when they they don't know, but uh but that's not great if you're actually trying to do validations using an LLM. So, here's why it matters, right? Um LitRPG is a genre built on precision, okay? The readers

are absolutely fanatical. They track stats, levels, inventories across millions of words. They'll catch what your LLM won't. your readers literally have a better knowledge graph of your world than your LLM does, right? So, the question is how do you how do you cope with that? How do you fix that? The obvious answer of is that you throw all of it in the LLM's context window. Uh but

the research says otherwise, and also our practical uh experiences say otherwise. Now, coherence, even with uh an extremely large LLM, so some of these um ones from Google and now also Anthropic have million uh token uh context windows, and Gemini, I think, has 2 million context window. Coherence drops like uh you know, at a cliff to you know, with just approximately five interactions, you know, because that's

there's the that's the amount of text that you're dealing with. Only two of nine LLMs, even with this, right? Only two of nine LLMs actually had a uh had a uh consistent worldview even for those amount of uh limited interactions. And remember, we are uh looking at authors working with this for period of time. It's not just going to be five interactions. It's just an unusable system

if we just limited it to that. So, the point is that attention is obviously not memory. So, the other obvious answer is what about rag? So, here is another scenario. Uh Elena has blue eyes. Elena has green Now, blue and green eyes land in the same neighborhood in embedding space. Cosine similarity is 0.97. Semantically similar, but contradictory. Right? Now, vector search finds this as relevant context. It

cannot find contradictions. And on structured queries like stat tracking, vector rag literally scores zero. So, similarity is not consistency. So, we need something fundamentally different. So, here's the insight that we had. What if the LLM that knows what's true, when it becomes true, and what rules apply? Not a context window, not a vector index, a knowledge graph. An immune system for your narrative. That's what we bet

our uh our our architecture on. Let's see how we did. So, here's what makes um LitRPG the perfect domain for this. The game mechanics are right there in the prose. Level ups, stat blocks, quest updates. Uh regex pulls all of this out at 100% confidence. No LLM is needed at this point in time. Free facts. Okay, because uh you know, you can do that with the uh

with LitRPG books because they're written in a certain way uh that uh readers expect. So, we can exploit that. So, basically every LitRPG novel is secretly a database change log wrapped in a story. But now, let me tell you the full architecture. Of course, if we were just able to use regex, we would not really be talking about this. So, um yeah. So, here is what the

architecture looks like. The pipeline is pretty simple, if you ask me. Manuscript goes in, knowledge graph comes out, agents use the knowledge graph, contradictions get caught. Each phase has patterns that we need to solve because obviously the architecture looks simple, but uh not that simple in reality. So, let me walk you through some of these. So, so we had Neo4j, we had a manus- manuscript. We just

needed to put the stuff in the graph. How hard could that be? Turns out the first question is not how to extract, it's what to extract. So, what nodes and relationships uh does the graph need? Or as the graph folks say, what's your ontology? Um we work with domain experts to first define a genre-level uh schema, a base schema. The stuff that's true for most RPG books,

characters, locations, facts, events, and so on. One important decision here is that uh facts are nodes, not properties. So, we'll see why in a minute. We feed that plus a few sample chapters into Neo4j's uh schema from text extractor. It comes back with a bunch of entities and and relationships. But, that's not uh that's not what uh you know, the author might be satisfied Why? Because it's

still uh using an LLM, and it might not be exact uh because that's the name of the game, right? But, what our experience tells us is that more or less in the ballpark. So, what we have is a human in the loop, the author, because they knew know their book the best. So, they take a look at what the uh the um you know, the LLM has

come up with, and and now they prune what doesn't make sense, and they basically create what is called the books anthology. So this is looking at at what makes sense in the context of the book itself. Okay? So basically what what what you're saying is the machine proposes, the author disposes. So now the question is what exactly how how do we get these facts into these notes.

So now we've got the the the skeleton. Now we need to actually get facts into these So here's the thing, right? Here is something that you might see in a in a in a literary in a litRPG book. She grabbed her staff. Great, but who is she? That's the pronoun problem. You can't just throw the entire chapter at at an LLM and pray. So we solved it

in in four passes, each one feeding the next. So the first pass, like I told you, was to do a quick regex search because it's cheap. You don't have to use LLMs, which means you don't have to spend tokens. And more importantly, the confidence of this process is pretty much 100%. The next thing that we do is we give to this gives basically the next step, which

is we give this to the standard Neo4j graph rack pipeline. It's think of it as a head start. Okay, so you've got some entity names, you've got some such a data even before the LLM touches anything. So that's what we call entity discovery, right? This is roughly what this thing this utility called LLM graph transformer, which is part of the graph rack package, gives you. You find

all the names or you find most of them. And then pass two is the coreference resolution. Coreference resolution is just fancy words for trying to grab pronouns and those kind of things. So she becomes Lyra, the healer becomes Civ, and him becomes Kale, and that kind of thing. Now, the LLM knows who who everyone is. So, the next step is attribute extraction. Um but now with the

resolved names, each extracted fact becomes a confidence score. The LLM reports how sure it is, and it does that pretty much everywhere, right? uh and then uh cheap passes uh get fed. Um feed in uh feed expensive passes, okay. And and the facts are now basically in the graph. Again, I simplify this process. Uh it was a lot more complex, but I've got uh only this amount

of time and uh real estate on my screen, so more or less it was what what you're seeing here. So, now we've got the schema, we've got the data in the graph as well. So, the question is what we can do with it. So, the facts are in the graph, like I said. Now, we need to check um what what's there. So, so what we do is

we let two kinds of rules, right? So, now um one technical users can start writing Cypher graphs uh Cypher queries. Um and they can do that uh and we put these as as static rules because the administrator has to make sure So, this is the platform uh administrator has to uh that, you know, there are still certain rules that you need to follow as part of writing

LitRPG books. So, these are non-negotiable rules, and we don't want uh to go through this process of having an LLM write some query. Although, we allow that as an experimentation thing when you're trying to experiment, but basically we save them as Cypher queries directly. So, these are rules which technical users use. Non-technical users, uh you basically describe a rule that they want in plain English and then

that gets converted to to to cipher and finally, you know, we use that to flag whether a uh, you know, a character's eye color changes inadvertently from blue to green or what have you. And then, uh, either way, like I said, these rules get saved and then they run against the graph. So, now when you're doing that, you know, so you basically are able to flag a

bunch of incoherences, if you will, in the book, right? But these these are all static rules, by the way, right? You really the the fun of the of the of the thing is when you're able to move past that. But, you know, some authors are finicky about what kinds of things they want and they don't want and they want somebody to look over and and and do

that. So, this allows both the administrator and both technical and non-technical users, to express some of those, uh, you know, constitutional rules, if you want them want to think of it that way. So, another related idea that this allows you to do is is is a a different set of cipher queries for what you call causal chains. Okay, so, um, so what you just saw was was

that these are these these facts are are static, right? Now, but you might want something that is that is dynamic, like I said. Um, and these are these Oops, are you able to see it? Sorry. What do you Yeah, you're able to see it, right? Uh, yeah, so so events are dynamic, right? So, and and and it's usually a chain. So, you kill a wolf, you gain

some XP, you level up, your stats increase, and you you basically see if a reward shows up without a quest completion, the chain is broken, another plot hole, you know, caught by the same rule engine, But authors keep rewriting stuff. Uh, you know, they they change their minds. So, the the idea is that we need to keep all of this current. So, how do you do that?

the graph has to keep up, right? We mark facts from the editor chapter as stale, we re-extract, uh, we then clean up, but the re-extraction step is a is a bit of a trade-off. You've got two approaches to do that. One is incremental where you just re-extract the chapter that changed. That's fast, cheap, you do it on pretty much every save, but you might miss some cross-chapter

impacts, right? You might have said that this character has a has blue eyes in chapter one and now you might be contradicting yourself by saying, uh, you know, um, it's the they have green green eyes or blue eyes or whatever, right? So, in that case, you might need to do full re-extracts. That is a bit expensive because, you know, it's you know, usually we're chunking and doing

all of that, uh, even even with the with the, you know, the, um, utilities that that Neo4j provides, uh, it's still three, four LLM calls per chunk. And more if your chunk sizes are bigger But it's guaranteed consistency. You do this at You do this at at at at milestones like you've completed a chapter or you've completed a section or you've done an entire book and that

kind So, the graph is really a living document. And and you keep it current. Now, the question is how do AI agents actually use it? So, here is the thing, right? When an author saves, a validation agent kicks off. It does two things. One, the static rules that we just saw, uh, the same Cypher queries, they run automatically against the graph. That's easy. Uh, alert I, you

know, eye color drift, dead characters, missing limbs, that kind of thing. These are known problems. Second, and this is where it really gets interesting, the dynamic query part. Right, the LLM reads what the author just wrote, looks at the graph schema, and then generates its own Cypher queries via text to Cypher. It checks things that you don't think to write rules for, the unknown unknowns. Both layers

hit Neo4j. Context selections come back as inline comments. the graph turns 500K words of novel into uh about 2KB of targeted context for the And and and look, you know, so so these patterns are not necessarily limited to just fiction. We built this basically for uh RPG novels, right? But anywhere content is long-lived, interconnected, and evolving, you basically hit the same problem. So some examples might be

legal briefs uh reference prior rulings, medical records that span decades, uh product docs um that 50 engineers are are editing simultaneously. These days with uh with spectrum development that is uh an unfortunate reality sometimes. It's the same problem and the same So if your system needs um to remember, check, or stay consistent in general, you most likely need a graph. So I might make it feel that

um all of this is is uh is child's play. Uh but here are some things that that were that were hard or that things that we still found hard. Right? One, the main thing was entity entity resolution. And we had all of this we had multiple passes at this and so on and so forth, but it was still still still pretty hard. Okay? you know, so the

healer the healer Lyra she her all of those things if you can't resolve those a below about let's say 85% or or so accuracy the graph is basically um full of orphan facts and everything downstream basically is wrong. So so this graph extraction process is is definitely one of the most critical uh that we had to solve for. And that's why the whole idea of introducing the

human in the loop, Especially the author. And turns out that uh the author also uh relies on on this this thing where they call they call them friends and family like you know, these are like fanatics who are willing to to do reviews of books even before they're published. So uh the platform owners might engage with them and uh the authors might invite a few uh select

folks to do that. So we do that in conjunction with humans being in this loop. But what you get from uh from uh from the AI is you get a good reasonable starting point. And you also get a confidence score where the AI itself will tell you that uh you know, it's uh uh it's only 60% confident because uh you know, the the narrative is fairly complex.

Now um we we we did we did have to iterate over this quite a few times, right? Because when those rules ran whether they were static rules or dynamic rules uh it was really predicated on this graph. But once you got the graph in a reasonable place, right? Then you're really, really talking. And it was usually tough in the initial phases. Uh, especially if you've got a

manuscript that is more or less ready, right? Now you've got a very large chunk of text. Or if you're writing, let's say the fourth book in a series, uh, you know, that kind of thing, right? Where you've already got three books. Uh, and it's also a time-taking time consuming thing. So, uh, so extraction is expensive. So, if you're working on problems like this, you know, that's I

guess the uh, the the experts here will will agree that that's that's one of the critical pieces. Um, obviously extraction cost as well, you know, so the the bigger the book, the larger the cost of of you having to having to having to basically cough up LLM tokens to do that. Uh, maybe right now with LLM tokens being subsidized pretty pretty heavily, maybe we don't really care

about that as much. This didn't really come up, but uh, but we do keep track of how much we're spending. Because if it does become expensive later, then um, uh, then yeah, obviously, you know, you you need to keep that in in account. So, we had a lot of observability going to see how many tokens um, uh, you know, are getting spent. And also, what we did

was as as a as a um, you know, as a commercial offering, we were giving these authors uh, uh, you know, a a budget, right? Okay, you've got you've got the silver plan or you've got the bronze plan or you've got the gold plan, and you get a bunch of tokens for free. And after that, you can basically pay for your own tokens if you want to

do that. Uh, because we really didn't want authors or the platform owners didn't want authors to just have the AI write the entire book for them. They really wanted to use it as an assistance mechanism, not a full writing alternative. Another thing is that false positives are are are annoying. I mean, so LLM does make mistakes, although although with with the newer LLMs like Anthropic's uh you

know, 4.6 Opus and all of that. I mean, it's it's pretty pretty close, right? I mean, you know, it does a fantastic job. Uh, but but obviously you still have the human in the loop. And and there are some very very very very critical what you call um uh corner cases like uh you know, you might have a flashback in the book. So, just because something just

appeared early on sequentially in the book does not necessarily mean So, for example, you've you've you've got this thing where you've you've chopped off your right hand, right? And now you're saying in in a in a in a later chapter that you're wielding an axe with your with your right hand. You know, if you just just look at it in a plain way, then then yeah, I

mean, that's a that's a contradiction right there. But if you say that it's a flashback, then uh you know, it's it's it's not. Right? So, those kinds of corner cases still are a problem. Uh and and those need to be need to be taken into account. Uh but but it's more or less more or less there. The other thing is is is is thinking during rapid edits.

I don't think this is necessarily a graph problem. Um this was more a design problem that we had. But but again, I I if you're trying to do this in real time, it was something that we had to contend with and we had to we had to make we had to make architecture um you know, we have to account for it in the architecture. So, things like

only uh on a on a save, only do it manually sometimes, certain kind of checks only do manually because they're pretty expensive, and so on. Uh and and do that only at specific milestones and so on. So, that kind of thing. So, here's the thing, right? The the really does not does not replace uh human judgment. It it amplifies it. And that's that's basically enough for what

we were doing. Neo4j played a huge part in in in the solution, and that's why this talk. I'm happy to take any questions you might have. Um let me know uh if there are any, and um um Okay, I'm trying to read what's uh there's question or Q&A tab. Okay. Uh what do you use for entity extraction and entity um Oh, yeah. Uh so, yeah, there isn't

a whole lot. We had to uh some custom uh stuff for that. you know, the the main thing that we did use was uh was the schema to I forget the name, schema to text uh text transformer or some uh you know, something like that. I forget the exact name. Uh but but that that is something, and then and then there was there was one more thing

as well. Um as far as uh what do you call as far as um entity extraction there is an there is a new project utility. I forget the name right now, but Give me a second. It'll come back. Yeah, yeah. So, there is this thing called LLM which might help. But uh but there is nothing for so that does help a little bit in terms of discovering

entities. Uh but there isn't anything that we could find for coreference attribute is extraction and so on and so forth that that was something that was directly usable. Uh so we had to write a bunch of those, uh we were using LangGraph as as our AI um AI um you know, framework. So, so that was Any other questions? Marcus, looks like there might not be anything else.

If if there isn't then I guess we're done. Do I leave stage if uh is that what I need to do? Sorry, I'm not not familiar with this with this platform. Perfect. Thanks a lot, folks. It was it was great being here and hopefully you picked up a couple of things and see you around uh uh next time.