NODES AI 2026 - Ghost-busting with Neo4j Graph Analytics in Snowflake

This talk introduces the integration of Neo4j graph analytics with Snowflake, showcasing how to analyze and process data efficiently using these technologies. The speakers, Tim and Stu, demonstrate the collaborative capabilities of working across different locations while capturing and sharing ghost data. They explain the use of the Cortex Code to build applications for data input and processing, highlighting the process of creating and managing agents in Snowflake for specific data analysis tasks. The session dives into algorithms like betweenness for analyzing relationships within ghost networks, and the fast RP algorithm for clustering ghost data into meaningful categories. Overall, the speakers emphasize the potential of graph analytics for various use cases beyond just ghost data, illuminating how organizations can harness the power of data collaboration.

[music] >> Thanks uh Hannah. Uh so I'm really excited to be here at our first OSAI. Uh last year in May we launched Neo4j graph analytics for Snowflake uh and I'm really excited we've got a Neo4j agent. It's been in preview uh for a few months now uh and I'm hoping that we'll have that uh by GA for this time next month. Um so uh Tim and

I are going to take you on a a ghost-busting tour. Uh he's out in New York and the New Jersey area as our senior ghost solutions engineer collecting all the data in the dangerous line of fire like fighting ghosts. I'm here in London uh our data HQ. Um I'm the lead data product manager. I'll be doing the ghost analytics for you today. Um so I'm just going

to hand over to Tim now who's going to kind of set the scene explain everything he's doing in in Snowflake. Over to you Tim. >> Thank you. Yes. I got a couple extra slides there just for when people get the slides later. But what I want to really show you is how easy it is to work with any data. It's not scary anymore no matter what it

is. You may not understand your data but with the combination of the right tools and the right uh people and technology working together, you could solve any problems and with uh Snowflake and Neo4j and Cortex Code and AI and MCP, we can get down to everything we need to do. First thing I want to show you real quick is how we could collaborate across continents and time

zones and you know astral realms here. So, first thing we do is take advantage of the close partnership between Neo4j and Snowflake and I am sharing my ghostly data that I'm capturing here in New York and Princeton and I have a private share from Virginia going all the way over to Neo4j so that we can work on the same data. So, Stu can do the analytics on

the data as soon as I capture it. So, over here in the United States, I've used Cortex code to build up a couple of quick applications so that I can get the data in. Some of it we acquire through uh public channels, some of it's is us in the field whether I'm here in North Jersey or over in Manhattan where there's lots of different things going on.

Some of that reported by the public, some in social media, how have have you there. I've got a couple different ways of looking at this data and capturing it because with the power of Cortex code, I could just build these up almost instantly. In fact, for this one here, I saw Josh Long say good luck on our session and I'm like, "Hey, we should have a spring

interface to add new data." So, yesterday I took 20 minutes and had Cortex code build a Spring Boot app just for you, Josh, so that we could do this and bring in data when it comes in really quickly, process it with Cortex to get the basics in there, use agents to make sure our data looks good before I hand it over to uh Stu, so we only

have the you know, the latest and greatest data. So, Stu, going over to you in London. >> Cheers, mate. Um if you uh could uh stop sharing uh then I'll uh Cool. Be great. >> You got it. >> Excellent. So, uh as Tim explained, he uh sent us a a data product. Uh so, uh I've requested that. He's um approved it. I received that uh request by

email. Um so, uh jumped in uh into my dashboard. Um and uh what I'm going to do is just take you through um how um I've got it set up here inside of Snowflake. So, Tim's got the ghost data that he shared with me. Um I've got kind of ghost information, uh the ghost sightings. Um I can see a preview here. Um now, because in my analysis,

I'm going to need to write data, um I've had to create a a view of uh Tim's data. So, if we scroll here, I've got my ghost detection, uh and I've got the very same tables uh shared as views here. Um so, I can see the same information. Um so, what am I am I going to use to perform the analysis? Well, uh as I mentioned at

the beginning, uh Nearform G um built a native application for Snowflake. We've been in the marketplace since uh last May. Um it comes with a 30-day trial, so please, you know, after uh nodes, jump in, have a go. Um and um Um there's lots of great content provided in in the app listing for you to find out and get going. Um so to do this analysis, I'm

going to create an agent. We've got this psychokinetic energy meter PKE uh agent that I'm going to create. So to do that, I just need to specify the name of the agent. Um the schema that provides the data, so that's ghostdetection.data. And then um I can only read from that schema. To write the results, I've got a separate schema, so you keep your data that Tim's provided

separate from the data done in the analysis. that's just going to create the agent. If we come over inside of Snowsight, you can see that there's an agents menu. It gives you a list of all the agents. You can basically have an agent per data set, so an agent specializes on the the So I've got one that analyzes open flight data. That's a another great demo. So

if I hit refresh now, I should see my PKE meter agent, so we're locked in and ready to go. I've got a a prompt. It's a bit more elaborate than you would probably need. I just want to make sure that the Snowflake intelligence and the agent do the right thing in the shortest amount of time. Uh and at the moment the agent would try and use threat

level and verified, which are varchar, but we need to process only numeric data. Um so I'm going to submit my prompt. And the great thing about the agent is it provides a tool that explores your data, figures out what are the nodes and what are the relationships. So, you don't need to do that. It makes it super simple to do some exploration. So, it's inspecting I'll just

expand here. You can see um it's identified what's in the the database. It's going to propose some views that it can create. So, the views of nodes and string properties so you can make sense of where I can make sense of the information at the end and then the relationships view. Okay. So, um we're now ready to do some analysis. Now, Tim and I have been talking

about the problems he's facing in the field. He's confronted with all these ghosts and he needs to kind of really focus. So, we suspect that some of the ghosts are acting as gatekeepers, right? Find the top I can't type. Top five. Um which basically means that he can now get going and focus on specific I'm just going to wait. It needs to update the semantic view. So,

all the tables and views that it creates it records in the semantic view so it knows how to combine results with other queries which becomes really helpful in later analysis. Um so, we're all ready to go. It gives you a nice summary of the graph structure. This again, if you're new to graph, is super helpful. You don't really have to figure stuff out. Obviously, if you know

graph and you know really well, then you know, knock yourself out. You can specify your own. Um, so let's uh see what it's going to do. Now, I haven't specified the algorithm that I want to use. Um, you know, if you don't know, uh you just describe the business problem, right? What's the business problem? Or, in this case, it's not really a business problem. Uh we're being

overrun by some ghosts, right? Uh and it's picked up from gatekeepers that really it needs to use betweenness uh as an algorithm. So, uh what's betweenness about? So, if we think about uh ghosts, uh basically betweenness uh is looking for influence. Like, which ghosts are creating the most influence over the whole network? Uh they essentially act as a bridge uh and communication point between all of them.

So, you measure the shortest path between pairs of ghosts, and then the ghost with the most shortest paths that pass between it uh is has the highest betweenness uh figure. So, uh this is super useful for analyzing communication networks, uh helpful for transport networks, spotting disruption that can occur when ports are out of action, maybe a warehouse fire and how that can impact um if you're in

the um pharmaceutical industry, then how proteins um work and influence on uh the uh body system. Uh if you're looking at communication networks, uh then maybe like your routers and stuff, which ones are prone to bringing down uh entire parts of of of a network. So, a range of kind of analysis pure problems beyond just ghosts. Um, so let's cut back, and hopefully um we've uh run

the analysis. It's um figured out the uh configuration. Let me just expand what it's thought about. Uh it can get the algorithm config, uh generate that configuration. But I think one of the things that's kind of helpful is it knows what it it's doing here. and uh it's figured out the configuration. So, if you're new to our uh graph analytics application, you haven't learned the syntax, uh

it's predominantly the kind of like a JSON-based format for for uh parsing in to the the SQL uh procedure uh that describes the nodes, uh the relationships, uh and the algorithm that you want to run. Um but you don't have to learn that, get going um uh super fast uh analysis. Um so, hopefully uh it will uh finish in the the next few minutes. and uh please

uh post uh any questions. Um hopefully uh Tim and I will have time at the end uh to be able to answer those questions, uh especially while some uh thinking is going on. Um so, uh please uh knock And I am hopeful that that it's going to get there pretty soon. Um what while it's doing that, uh let me just talk about what we're going to do

next. Right? So, the next the next problem, right? You know, there's loads of ghosts all over the place, but really it'd be kind of helpful if we could categorize them. If we've categorized them, we kind of know what Tim and uh his team are going to be facing in the field. Um so, uh that's going to be the the next prompt. Again, I haven't specified any algorithm.

I'm just telling it like what the problem is that we we want to solve. So, uh bingo. Excellent. It's come back. It's told me how many ghosts there are, their locations. and um it will just come back. Uh so, to give me something that I can understand instead of just like a bunch of numbers, um it's going to combine the results uh with uh one of the

other views it created, which had all the string properties, uh which then means that it can give me some really helpful Hey, the ghost name Clauneck Spectra apparition one, its threat level, uh how often it appears, and its confidence score. Uh and it's got this super high between the score. You generally only see something this high in ghost data. You would probably wouldn't see it in many

other types of data sets. So, um uh we're all good there. So, I just uh forgot the prompt. Oh, yeah. Uh that's it. Uh so, lots of information on ghosts encode it for me. So, we can categorize into five types. I think that seems uh pretty reasonable. So, uh Survey Contingence is off analyzing my request. Uh and it's figured out that there's two embedding algorithms. So, when

I said encode, um fast RP and node to vec. Now, fast RP or fast random production uh is a super fast way of analyzing really high num- large graphs, like up to a billion kind of nodes. So, we're talking like a lot of ghosts. Um it'll it'll probably pick fast RP. I did say kind of quickly. And then it's figured out that K-means um is a [clears

throat] good way of clustering. Um so, whilst it's off um doing that, um I'm just going to talk a bit about um, like how um, fast RP works. So, as I said, it's a way of encoding information in our ghostly network. Um, it's super fast. Every ghost starts with kind of like an aura. Uh, it's a random aura. Um, and then um, because we said K equals

five, we would typically have five centers. In this graphic, it's I've simplified it to just three. Um, so it picks three ghosts that act as the center of these clusters. Then it kind of loops around all the ghosts um, comparing the um, the auras, uh, assigning them the ones that are closest, um, it joins that uh, ghost. Uh, and then it kind of iterates round uh, until

such point that the network is stabilized and you've got um, a a vector that represents the ghost in the ghostly network uh, in it uh, in in space. Now we've got the vector, what do we do with it? Well, uh, machines and machine learning love numbers. Uh, that's really the only thing they understand. So, these vectors are a a great way of basically um, comparing the vectors

using cosine similarity to figure out um, uh, the closeness. Um, so you start off all the ghosts have got uh, their I uh, vector. Uh, you pick three as a a center. Uh, and then you kind of iterate round um, joining uh, those clusters. Uh, a ghost can only be in a in a single cluster. Um, and uh, you basically repeat um, whilst ghosts figure out which

cluster they belong to based on those vectors. Um, and then once things are stabilized, uh, you've got your categorization. Um, basically this This a form of unsupervised, um, machine learning um, our ghosts didn't have, uh, a category, uh, and we hadn't come up with any beforehand. Um, so it's kind of figured out how, uh, these ghosts kind of all relate to each other. So, fingers crossed, hopefully

we can now cut back, uh, and, uh, check on how Snowflake Intelligence is running. Uh, it's still running, uh, fast RP. Uh, it had to figure out for 32 seconds, what it needed to do, uh, in terms of the configuration. Uh, it did hit an issue, um, in terms of, uh, the data. it's basically, uh, done that, and, um, we've now got our ghost embeddings. It created,

uh, 64-dimensional vectors capturing the whole network structure. um, and now it's going to now cluster those, uh, into five clusters using K-means. Uh, and we are doing pretty well for time, so, um, I have my fingers crossed that, uh, in the next few minutes, uh, we will have, um, got the the results. Um, whilst it's just running K-means and figuring out, um, one of the, uh, things

that Snowflake has got in preview is these artifacts. Um, so, earlier today, like when I was testing this all out, um, it generated, some, uh, uh, nice, uh, graphics, uh, dashboards summarizing the results. Um, so you can kind of, uh, share them, save them, share them. I could save them over and share them to Um, let's, uh, just go back. Oh, not new tab. I want to

go back to, um, how it's getting on. It's still running. Um, one of the other things that's helpful as a prompt if you're new to the product. what problems can I solve with graph Oh, I haven't typed that. I'm sure I'll figure out what I meant there. one of the tools obviously our tools know what algorithms we've got in the product and it kind of comes back

and it can kind of summarize the different types. So, we've got community detection and clustering, path finding and routing, you know, finding shortest paths for logistics and delivery routes. So, again, if you're new to graph and you want some inspiration about the kind of problems that you can solve then you know, that's a great prompt for you to to use. Uh, so let's just cut back and

see how it's now I thought for a second case somebody's created creating the views. It's still executing the tool. We got any questions that we could maybe take while it's just finishing off its last little bit of thinking. >> Yeah, you got a couple of good ones from Vinod down there in the chat for the session. >> Uh, oh, so yes, any resource on algorithms are available.

Yeah, besides the documentation. I think that prompt that I just showed is is the best way. It'll tell you exactly what algorithms are there, what business problems they can solve. Um, so I think that. Can I call this into my agent as an MCP instead of finding it on the marketplace. what is that? It's my agent as MCP instead of finding it. Uh so, the app itself

um is only available on the the marketplace. So, you would have to install that. You don't have to use our agent. Um your own agent um if it has access to the Snowflake data that it's created, you can query that um yourself. So, yes. Uh does this container allow us to build graph visualization based on the graph model instead of the Uh yes, so we have uh

the Neo4j Python visualization library. Um it's great. Um it's basically the same library that's used by Bloom and Browser. Um so, you just do pip install um and you can create um some nice styled graph visualizations. Um you just need to be aware that when you're visualizing the kind of very large graphs created here, you can only um show like about 4 to 10,000 nodes realistically. Um

you don't want to try and visualize a a graph of uh a billion nodes. So, uh you you need to kind of create views that of the data that um the visualization can handle, but you can build Streamlit applications um using that. Um how do we edit the graph given the Snowflake is editable? Where does this get it stored in Snowflake? Yeah, so um the way uh

the product works is uh you pass in um some configuration. It's a configuration that specifies the view that provides the nodes and the view that provides uh the relationships. It takes that, projects it into the Snowflake container into its memory, runs the algorithm, and then writes the result back to Snowflake tables. That means that you can control the information that you pass into the graph by just

specifying a view. Changing the view, you will change the amount of data that's in there and presented to the to build the graph. da da Um Keep the questions coming. Yes, please do. I use Cortex CLI already in my agent to pull the schemas, but looking to integrate Neo4j into my agent to build a Cortex layer. Yeah, great. Um It'd be cool to see Snowflake Streamlit and

use this very cool love the ghosts. Thanks. Yes, um I didn't have time to pull together a nice Streamlit app. Um yeah, you might say that's actually a bit lazy considering how quick uh Tim was saying it is to build one. Um but uh yeah, I I've been been making sure that we've got a nice reliable run through um with our agent while it's in preview. Um

so yeah, we've got a few minutes left. Um yeah, as I say Um it's looks like it has correctly uh categorized the ghosts into five types, um identifying kind of the largest group and the smallest group. Uh and as I said, it updates the semantic view with with that information. And then it's giving me a nice summary. Um and those summary tiles, you know, I can save

them. They'll appear as artifacts as I showed there. Um the other really cool and handy thing, like, you know, you're probably not going to use this in a kind of production pipeline, right? This is really good way of exploring Um or if you want to democratize data within your organization. But, you know, if you want to build a pipeline and run things regularly, then all you can

do is just copy the configuration and that we provided there, paste it into your notebook and away you go. So, here you can see kind of we specify the instance size. We've got a estimate procedure that if you pass in the tables for the nodes and it will kind of try and give you a good estimate for the size of the instance you need. we specify the

table prefix that goes goes up results for where I'm writing The node tables, the relationship tables. It's going to be it for K means as well. that is the end of what I'm glad to say is a successful If we got time probably for a couple more questions. Yeah, Tim, big thanks to Tim for the inspiration. Hopefully I'll learn a badge, proper Ghostbusters badge to to to

go with my boiler suit that I've got. But yeah, his his data set and when I saw it came across it I thought that's so cool. I was is that going to be in the post? Excellent. Cheers to myself. Excellent. Yeah, best costume winners. Aha, yes, absolutely. If if you know anything about Neo4j, I do love getting dressed up. So, uh Thanks for all the great questions.

Tim, any kind of closing comments from you? >> Yeah, thanks for showing off the artifacts. Those are pretty cool new feature in Snowflake Intelligence. You could save charts and graphs. It's a pretty nice thing. We could actually share them, too. Maybe next year we'll have shareable artifacts and a Cortex code skill that builds up Neo4j agents and NCP between them and then, you know, actually be able

to send messages right to Ghost to have them do agentic AI for us. We'll see. We're working on it. >> Awesome. Yeah, just one last thing. So, Olga, she was pretty instrumental in creating the the agent. Um And I just published a blog. It's Neo4j graph machine learning. So, how to detect network intrusion in IoT network all your data in Snowflake. We use KNN. And the really

cool thing is because we've got a Python runtime, we're able to run graph stage on a GPU. And we get fantastic results for predicting network intrusion. So, please check out the blog. And as I say, you've all got the opportunity to use a 30-day trial whether you install it in your company business account if you're able to do that. Snowflake provide trial accounts where you get a

certain amount of credits. You can install our native app in there and use your credits for our native app as well. So, Doc sells out. Hope you enjoy the rest of NodesAI. I can now sit back and enjoy the remaining sessions. So, thanks again for all your great questions. Yeah, Vincent, we definitely need a collab on Ghost Neo4j t-shirts. Yes. >> [music]