NODES AI 2026 - Infinigraph: Vector Data at Scale

This talk covers InfiniGraph and how Neo4j can effectively manage large vector data through sharding. The speaker, Hannah Zaida Venham, a senior software engineer at Neo4j, discusses the necessity of sharding databases when they grow to terabyte sizes or more. She explains the architecture of InfiniGraph, which allows for property data to be stored across multiple machines while maintaining the connective nature of graph databases. The presentation includes a demonstration using the LDBC social network benchmark to illustrate the performance differences between sharded and non-sharded databases, especially when handling vector embeddings. Venham highlights how the sharded architecture can achieve greater efficiency and scalability, facilitating the use of AI applications that rely on large property datasets.

[music] >> Uh thanks, Marcus. I hope everyone's having a great conference. Um and welcome to my talk on InfiniGraph and how to use Neo4j to handle vector data at scale. Uh to introduce myself, my name's Hannah Zaida Venham, and I've been a senior software engineer here at Neo4j for 2 and 1/2 years. I work in the sharding team, where we tackle the problem of what to do

if your database is too large to fit on a single machine. Some of my colleagues should be in the chat of this talk. So, if you do have any questions as I go, please feel free to ask them, and they'll do their best to reply. Uh so, in this talk, I will first cover why you might need to shard your database. Next, I'll explain the architecture that

underpins Neo4j's new InfiniGraph offering, explaining the motivation and how it works. Finally, with the help of an example data set, I'll show the impact of sharded property databases on Neo4j database dump sizes and query performance. And then, there should be plenty of time for Q&A afterwards. First of all, sharding. What might be some symptoms of you needing to shard your database? Uh you might have a database

which is approaching multiple terabytes of data. In this case, running it in a single instance becomes expensive because you need to provision larger and larger cloud instances, and there might be a limit to what you can request within your organization, or if you're running it on your own, what you can request of yourself. Alternatively, you might have an initially small database, but you know that it's likely

to rapidly grow as you add data so that it runs into this previous case. When we talk about scaling a database, there are multiple ways to go about this. Neo4j has features to support high read-write availability by adding primaries and secondaries via clustering as seen on the left. This involves adding extra machines that can serve requests and provide redundancy so that the traffic doesn't all have to

filter through a single server. We also support graph federation which allows you to use one query to collate data stored in multiple databases. This is especially useful if you had, for example, sales data which is split by country and most of the time you want to just update your databases per country, but sometimes you want to combine it into one report. You can find out more about

these in our documentation, but I'm not going to be discussing them any further in this Instead, I'm going to be talking about sharded property databases. This is the architecture that powers the InfiniGraph feature that was announced last September as an early access program and as of February is now generally available for anyone to use. In this architecture, we split the data for a single graph across multiple

machines. Unlike graph federation, it means that you can use the same Cypher queries as for a standard Neo4j database, but it allows graphs 10 times as large to be run using the same types of server. Now, when we think about how data is stored in a graph database, we have our nodes and we have our relationships. Each of these can have labels that characterize them. So, here

we have an actor label, movie label, animated label, and the acted in relationship type. >> [snorts] >> Uh and we also they properties which holds data that distinguish all of these things from each other. So, uh the Toy Story node has the properties title and released. >> What makes graph databases so powerful is being able to quickly traverse the graph to find new relationships. In a relational

database, if it grows beyond the bounds of a single machine, you can partition it by splitting tables and storing them on different machines. However, in a graph database, we can't do this in the same way because we want to maintain the connective nature of the graph. If we were to split the graph, uh we would have to decide where we wanted to do this, and it might

mean that we would lose some of the links between our entities, or it would add network latency when we're trying to traverse uh relationships that go between So, as customers came to us wanting to use Neo4j at the scale of multiple terabytes of data, we needed to work out how we could get around the limitation of the size of hardware that can be provisioned, and we didn't

want to fully split the graph. We wanted to maintain this graph structure in the center. When we looked carefully at the sort of data that Neo4j is used for, we found that about 90% of the store size comes from properties stored on the nodes and relationships. And in fact, only 10% was the innate graph structure, which is to say how the relationships connect different nodes to each

other. So, if we could move where the properties are stored, that would allow on average a 10 times scaling of the core graph. sharded properties databases, uh this is our new architecture, which is on the right. We go from having a single database which contains the entire graph to a sharded structure which has four physical databases. Now, each of these physical databases can be run on a

different server so that no single instance now has to be large enough to hold all of the data for your database. Uh we also have a virtual database at the top which the user can interact with as though the database was not sharded. They can just send normal Cypher queries there. It will work. to go more into the architecture, we call this middle layer the graph shard.

It contains the graph structure which is to say how the nodes and relationships are connected. But, it now no longer contains any property values other than node and relationship IDs and all of the labels and relationship types associated with the entities, but no properties. Instead, the properties are stored on this bottom layer which we call uh the property shards. So, each node in the graph shard has

a copy on one of the property shards and all of the properties for that node live on the property shard. Uh relationships are also represented on exactly one property shard. Uh these are connected to either their start node or their end node depending on the relationship density of each. There's a sharding function that knows uh which property shard contains the data for each specific node or relationship

ID. And this means that when read queries come into the virtual database at the top level, we can do graph traversals at the graph shards level and we only call out to the property shards when we need to retrieve values. So, for example, if I had a graph containing person nodes and friendship relationships, if I wanted to find all of my personal friend of friend of friend

relations like people, I could first match on a node which had the name Hannah, which would involve accessing the property shard to check the name property and find the one which is Hannah, uh but then finding the friend of friend of friend nodes can be done at the graph shard level, and I would only need to access the property shards again when returning the names of my

connections. >> Uh we can also apply all of the standard Neo4j clustering techniques to Infinigraph. So, to increase the read-write performance, we can add more primaries and secondaries for the graph shard and what we call replicas for the property Um we can also collocate several property shards or graph shards on larger servers to allow for future data expansion. So, say I want to provision a reasonably large

machine cuz I know my database is going to grow in size, then I can start with the a replica of property shard zero and a replica of property shard one on one server, and then as they grow in size, we can relocate them onto larger individual servers as necessary. now you've had a whistle-stop tour of how this new architecture works. I'm sure you're curious about how it

actually performs. So, internally, we use the LDBC social network benchmark as a toy data set for testing purposes. This is an industry standard data set, which represents a social network which has users and posts and comments, etc. Uh we have some standard queries that we use on this database, such as uh you could find uh person nodes which are like six news relationships away from Kevin Bacon

to get Kevin Bacon number, or you could find out how many posts a certain user has created in a certain country. We can generate uh mock data for this at various sizes, which means we can use the same Cypher queries to see how the database performs at different scales and with different configurations. So, I started by generating input data for this at scale factor a thousand and

placed it into uh a standard Neo4j This creates a dump which is 300 GB big. Um however, using the same data set and generating the sharded database with eight property shards, uh you can see that the graph structure is only a fraction of the size of the data set. Now, the mathematicians among you might say the graph shard here is actually only a third of the size

of the database instead of the advertised tenth of the size. Uh this is because the social media model we're using contains mostly toy data that is more to do with the connections between things so that we can check the graph traversal stuff rather than property data. Where the sharded property database architecture really shows its potential is when we have more property-heavy Um and this is in fact

perfect for the AI use case where we want to generate and store vector embeddings. Now, quick side note, what do I mean by vectors? Given we're at an AI conference, I assume that most people here are aware of vector embeddings, but I'll give a bit of a brief overview for anyone who hasn't encountered them before or wants a like recap. Um when we're trying to find novel

links between data, we need to represent it in a way that a computer can understand. To do this, we use models to generate a vector embedding that can uh turn an image or a block of text or any other type of data into an array of numbers. And then the machine learning algorithms can use these numbers, which they understand, to compare semantic similarity, which can find new

implicit relationships between data. So, in this example, there's a vector embedding I've done of the opening of Pride and Prejudice. If we had embeddings for a range of other text snippets as well, we might be able to detect not just further mentions of the Bennet family, but also Austen quotes from different books because they'd be written in a similar style. Uh but you can also see that

this vector embedding is a lot of data that I would be adding to my database. Um since version 2025.10, Neo4j has actually had a native vector Uh so, this can be specified in a header file that is used in the admin import, or it can be added in Cypher, or there are many other ways you can generate them, and that is documented in the Cypher manual. Uh

this means that instead of storing vectors as a variable length list of the database can enforce constraints that guarantee shape and d-type for your data, so it's safer to use. This also allows us to optimize storage and add some vector specific features. So, how did I add vectors to my data set? In the LDBC data set, there is a node type called post, which may or may

not contain a text property, which is called content. So, I wrote this simple Python script using the sentence transformer package to generate an embedding for each property and output this into a new file that I could use as replacement input to the importer. I used the free Google embedding demo model for this, use 256 dimensions per vector, and this generated about 220 million vectors. You could set

it higher if you wanted more comparison points. When you have over about 10,000 vectors, we recommend that Neo4j adding an index to the database, as this allows Cypher to query the data much more efficiently. It can plan for it. Building the index does increase the size of the database. However, around the 10,000 mark, the performance improvements are definitely worth the trade-off. But, I had generated myself 220

million vectors, so I definitely wanted to add an index. Here you can see the index that I added to the schema file during import. It specifies that I want to make a vector index for nodes which have the post label and the content emb property. You can also specify what dimensionality your data has and which similarity function you want to use when comparing vectors. So, default is

cosine, Euclidean also supported. Now, if we look at the size of the dumps with this added vector property and index, we can see that this doubles the size of the non-sharded database from 338 to 774 GB. which is bigger. But, on the other hand, when we look at the sharded dumps, although the property shards have tripled in size from 23 to 78 GB each because they have

like both a property and an index that are stored on the uh the graph shard has stayed at a stable size. Uh this is because we haven't changed anything about the graph structure of the database. We've just added property data. So, that just gets added to the property shards. This means that we'd be able to run the database using much smaller and cheaper hardware than the non-sharded

vector version. And all of these dumps are still much smaller than even the non-sharded non-vector dump. Now, because we've introduced networking in this distributed structure, we can't directly compare the performance between the sharded and non-sharded databases. Um different queries will perform better or worse on sharded property databases depending on how many times the query needs to access the property shards. So, something that quickly scans across lots

of friendship relationships would perform like much closer to the non-sharded version than something that is filtering on every hop for, say, when you met the person. Uh the Cypher teams have been working hard to optimize how queries are planned. So, for example, we batch reads to the property shards to grab property data for as many nodes or relationships at a time as possible. Uh and we expect

to continue improving this as we continue working on the the product. Uh however, for vector use cases in particular, because we have we now have multiple indexes, we have a vector index on each shard, we can search them in parallel, and we get some surprising results. uh here I have written a query that uses the vector index that I added to the social network database. Um if

you go if you're reading ahead, please feel free to drop in the chat what you think my top result for this query is going to be. Uh and I will explain the query to everyone else. Um so, I wanted to discover whether given 10 starting posts that are about John Lennon and no one else, um I can use vector similarity to find 100 posts for each of

these that are written in a similar way. And if so, what they've been tagged with to see if I can find some new artists or whatever that I might be interested in. People are posting about John Lennon. I like John Lennon. What else might I like? So, in the bottom right, you can see a subset of the graph with post tags and the post has tag relationship

that are the nodes and relationships I am using in this query. Without using the vector search, it would be easy to find tags that co-occur with the John Lennon tag, but with this query in particular, we'll be able to find posts that are implicitly similar from their content rather than just from the graph structure. Uh so, I'll give two more seconds for anyone who wants to guess

who is similar to John Lennon. And the answer is Sigmund Freud, which was a surprising result when I ran Uh, in this dataset, it turns out that posts about Sigmund Freud are most similar to John Lennon, and you have to get to position 29 to reach Yoko Ono, who didn't in fact appear before any of the other Beatles, um, which was interesting. Uh, so this was just

fun looking at the data, but how does this perform speed-wise? Um, to check this, I ran my query 100 times against the sharded and non-sharded versions of the database. Uh, I wanted to make sure that I used machines which are large enough to hold the entire graph in memory, so that we don't see any impact of missing the page cache. Uh, so I chose AWS instance types

with approximately double the amount of memory as the size of the database dumps. Um, this is slight overkill, and the database would start with smaller instance types than this, but it will be less performant the less of the graph it's able to hold in memory, and you would require, like, attaching a really big EBS volume or NVMe to start the regardless, in the sharded architecture, you can

see that we're able to run the database using much cheaper AWS instances than the single large machine, which, like, I gave it a terabyte and a half of memory. Uh, to read this graph on the left, on the x-axis, we have the time taken each time the query was run, and box plots with the 25th to 75th percentiles marked. Uh further to left means it ran faster.

And so, surprisingly, we can see here that despite the added networking of having to like network and go to the property shards for the initial and final tag name matches, uh because we can run the vector search in parallel across our eight property shards, the timing of these results is actually very comparable. And in fact, in this case, the sharded version of the database performs better than

the single large machine. And we see similar results with other queries that are using vector indexes and higher numbers of um of shards. So, in conclusion, what have I shown you today? First, I introduced the concept of sharded property databases, which allow a scale-up of about 10 times larger databases without changing your Cypher queries. Uh secondly, I showed how databases with large property stores will especially benefit

from this, as the graph shard stays the same size as we add more data Finally, I showed that by querying a vector index in parallel, we in fact see equivalent or better performance compared to a non-distributed version of our test data set, even despite adding network latency to the system. Now, as vector embeddings are used more and more in the AI space, we foresee Infinigraph and sharded

property databases allowing companies to take their property-heavy databases to even greater scales. And we have lots of further improvements to the stack that we're currently working on. Uh thank you for your time. If you have any questions, uh, feel free to leave them in the chat or the Q&A, and I will stick around and answer them.