About this talk
This talk focuses on distributed SQL databases in the context of artificial intelligence. The speaker, a senior vice president at Oracle, emphasizes the importance of high-quality data and consistency for effective AI applications. He defines distributed databases as those that store data across multiple physical locations while presenting a unified interface to applications. The speaker discusses several commercial distributed SQL databases, including Google Cloud Spanner and Cockroach DB, while highlighting Oracle's advancements in this area. By leveraging features such as strong consistency, data sharding, and replication methods, Oracle's global distributed AI database can meet the increasing data demands of AI technologies. Additionally, the talk covers the use of vector searches and model fine-tuning as crucial components for optimizing performance in AI workflows.
Full transcript
can all right if uh hello you can hear me yes great thank you yeah it's it's as I said in my previous session I can't hear myself so it's uh so so my name is way uh I'm the senior vice president president president president president of development at Oracle corporation right uh I work in the database team and oh sorry so today I'm going to talk about
you know distributed SQL databases in the age of AI Right. And so uh I want to start by explaining that good AI depends on good data and good databases. Right? Because um what we see is that agentic AI which is a hot thing now are real time and generate heavy load in databases. AI relies on strong consistency, right? This is sort of interesting, right? Because inconsistent data
or relaxed consistency leads to u systematic errors, right? Because the model is confused, right? Because the data is wrong is actually wrong data and gets the wrong results and stale data actually leads to answers that are wrong for today's reality. Right? The data the model is outdated, right? [snorts] And the challenge really is how do you build databases? How do you design database applications then keep up
with AI's demand for real concurrent current data? So to start with I want to explain what is a distributed database right um it's a database that stores data across multiple physical locations instead of one location right and each location stores a subset of the data right so for example the picture shows the application looks like a single logical database where the pieces are in different locations right
and so the fact that the data is physically distributed is hidden from the applications the application see one database right and you manage all the components as a single database And these are important properties. Now there are several commercial distributed SQL databases, right? I would say that historically the most the the first one I would say is probably the Google Cloud Spanner, right? It was first described
in 2012. It was first commercialized 2017. It's interest. It's important for the following reason because prior to Cloud Spanner, all distributed databases were NoSQL, right? And people assumed that to be distributed, you need to be NoSQL, right? And what Google realized was that it's very hard to build applications without strong schemas, crossroad transactions, consistent application and powerful query language. And that's why they introduced uh cloud spanner,
right? Which is great. Then a company called cockroach DB basically trying to copy the the concepts of Spanner without dependency on the specialized hardware hardware, right? And they built it on top of distributed key value store. Then Yugabi came along later and Oracle came by later, right? And I'll say Oracle's claim to fame is that we are really the most fullfeatured distributed database. And I'll explain why
that is the case. Now in this talk, I'm going to use the Oracle global distributed AI database as an example, right? And I think it's interesting because uh this picture is is actually who who recognize what this picture is? All right. It's a it's it's a it's an oracle at Deli in Greece. Okay. So this is the original oracle where if you go there to to consult
the priest to look at the future, right? Um and so the Oracle AI database actually the Oracle is named after this Oracle, right? But then the Oracle AI database can be used as an Oracle look in the future databases you know because Oracle database has most functionality and features has features that are not yet available in other databases right and it's a good way to see where
the other database in entire industry is headed right so you get to look get a look at the future by looking what it is today all right um and so I'll I need to set some ground rule explain some of the concepts behind what the Oracle global shoot database is Right. So again it's a single logical database physically descripted across different locations. We call the different piece
in different locations shards. Right? And then they replicated survivability because you need to have more than one copy of every shard so that if you lose a shard lose a data center and so forth your data doesn't get lost, right? And of course everything's active active. So you every piece of data every server can be can process requests. Uh it's strongly consistent and that's important. You make
an update in one place, every other place will see the update, right? And lastly, the data can be redistributed across shards, data centers and regions where the database is running, right? Because again, if you want to add capacity or move data around, you don't want to shut the database down, right? And so these are all important properties of a, you know, of the Oracle globally distributed database.
And Oracle distributed database became a distributed database in 2017 when we first support native database sharding. And these are some of the use cases right um so for instance for always on databases that cannot stop right things for online banking stock trading credit card processing so forth uh data residency which is that for regular industries where data as required to be stored in each country right so
for instance in actually interesting India data involve Indian cities cannot leave India right must be stored there right and so you're operating global company you handle that user proximity if you have a global business you want to make sure your customers you know in each country sees data close to them right and it's also good for massive scale for hypers scale analytics AI as well as OLTP
okay uh so I'm going to give you some from very concrete examples so this is one of our earlier earliest examples which is a this is a hypers scale workload on a multi- pabyte database right and it has supports it runs on 104 servers in the cloud right so it's distributed across has 104 servers and so aggregate has 5,000 CPU cores and 77 terabytes of memory. Okay,
so this is actually machine given how how given how much memory costs have increase these machines actually worth more today than they were when they were bought. But anyway, so this is shows what a massive hypers scale system looks like. This one is a never down architecture for for a payment system, right? So what they do is an active active deployment deployed across three different regions in
the world and within each region they deploy across multiple AS multiple data centers right and then it's protect against regional as well as a failures ability zone failures and what they achieved is automatic 3 second application failover with zero data loss right and and this is was a heavy transaction workload so this is really great because 3 seconds is like it's like nothing right and this supports
thousands of transactions per This one is interesting. This is a data sovereigny use case. So I talked about India before. So this is a US bank that initially operated a global database supporting applications throughout the world. All right? Because India requires data to be stored in India. What they did was they took the India data out split into a separate shard for India and it's replicated within
India. Right? Then the United States they have the rest of the world except for India. Right? Because this is a distributed database, the whole thing looks like a single logical database to the application. And so the applications can stay in the United States where they were before and not be affected. Okay. And this is great because now what happens is the India data stores in India. The
rest of the world data stores stays in the United States as before. The application doesn't have to change, right? And so this allows them to meet the India regulations in a cost-effective manner. is cost effective because the alternative would have been to move all the application servers also in India and in every other country where they need to do this and that would have been very very
expensive because the application tiers can be very complicated right so this is one easy way of doing meeting data sovereignty requirements you know cost effectively okay um in terms of agentic AI again we talk about AI you know semantic searches with vector processing because vector searches AI vector search is really the hot thing with AI And so this allow you to scale out for very very large
a workloads. You have searches structured and unstructured data agentic workflows model fine-tuning and so forth. Okay. Now the key thing about distributed database is really how would it perform right because they have they have data all over the world right and then the speed of light says that accessing remote data will be very expensive. And so how do you cheat the speed of light? And that's where
the most interesting things come. So distributed because the data is distributed has some inherent costs right first cross node operations and more more expensive intra node operations because you have to fetch data from somewhere else it's going to be more expensive because you have to go over the network right and again if you have to go over network the longer the distance is the longer it's going
to take just because of the speed of light right and likewise increasing network hops increases the cost further right so to reduce this tax if you will we need to do first send as little data over the network as possible. Ideally, you want to structure the data such that all your access is actually local to your to your database to your to your server, right? To your
data center. And likewise, you want to reduce the network the number of network round trips. Right? Now, if you don't do any of these optimizations, your application will still work, but it may not have acceptable performance, right? Because it may be very it would be very low latencies that mean your your customers, your users will be waiting, right? So, so to achieve this goal of minimizing cross
region communications, you know, Oracle developed a whole bunch of data distribution methods, right? In fact, we have the most distribution methods. Now, what are distribution methods? Think so here's an example, right? Whereby I have three shards in three different locations, right? And I have some tables. So, distribution in this case says that I basically map the tables, rows in each table and then spreads them across different
shards. Okay? So you look at the customers, you know, some rows go to shard two, the the red ones. Some rows go to the go to the third shards, the green ones, and some go to gray shard so forth. And so that's how I spread the data across the different shards. Okay. So how do I take the data and spreads them across now turns out that how
you spread the data around the distribution methods matters a lot, right? And that's why Oracle supports a whole bunch of these things. You know, the first one, the simplest ones are valuebased and system managed. value based is that I distribute the data using a range of values. So for instance, I might look at your columns that say country or phone numbers and so I would distribute data
based on what the country of the record goes to. So US data goes to this shard, India data goes this chart, Japan data goes to this shard and so forth, right? Or likewise using phone numbers and so forth. Or I can use hashing. I take some field and use I hash the thing and just distribute using hashing. And so this is good for things like customer ids,
device ids and so forth that get an even distribution. I talked about the importance of geo distribution which is country specific data. And so what we do is we allow this concept of composite sharding which is two levels of distribution which is at the top level you first divide the data by country right and then the next level you may want to hash it within each country.
Okay now I draw all these things. These things are done automatically for you. You basically specify when you create a database when you want to shard how do you want to distribute the data and then when you insert the data it gets automatically distributed for you. Now this is a nonobvious important use case of a userdefined data distribution. It turns out that most people you try to
start with using hashing to distribute it evenly across everything. It does not work in real life because in real life people are not all the same. Okay. And the example I give is musicians, right? You know, Taylor Swift is far more popular than most other singers, right? So you try to distribute Taylor Swift like everybody else, your system will be overloaded, will get a very poor response
and so forth, right? And so what user defined data distribution does, it allows you to special case distribution for anything you want, right? So you can say that well Taylor Swift I know she gets lots of hits and so what distribute Taylor Swift to a large number of dedicated servers with solid state discs and so forth. And so that's how you can handle things and you may
say well everybody else is not as popular as Swift. I group them all together into a single shared server. Okay. So this is one way of using userdeed uh to handle this kind of hotspots. Uh so that's number one which is you know Oracle tries to try to basically cheat the speed of light by giving you the most data issues as possible and the goal of course
is to prevent a customer from Japan having to access data and going to the United States to access data. You want to make sure that customers that data that Japanese customers access are are located in Japan, right? That's why having all these distributed messages makes it much more likely that we can meet the needs of the application because every single application distri wants to distribute data differently.
Okay? All right. So the next one is networking, right? The unfortunate part is that real world networks are very flaky. Okay. And that's why Oracle supports multiple replication methods. Um so so the one thing that's actually very very very important is this called the raft replication for extreme survivability and this is something that we introduced u fairly recently right and it's basically a quorum replication protocol and
what it does is it takes an odd number of uh of shards or nodes right and the way it works is that you spread your data across different shards different nodes in this case across three different data centers right and if one shard fails the other two shards will basically vote to elect a new leader to take over from the other shard and this takeover process you
know completes in less than three seconds end to end and for the database time only it's actually less than one second okay so one second is is like a heartbeat so within a single heartbeat we take over and you can't tell you can't tell that failover has happened right it's an active active architecture such that all the shards can actively process reads and writes right and when
you do the failover you get zero data loss you don't lose data right and you a strong consistency which says that if you make an update anywhere everybody else will see the update right so you don't get you know stale data you don't get cons you know um eventual consistency so forth so this is really cool this is gets you very fast failover active active zero data
loss and and strong consistency okay so this is what the first example I gave you was the extreme HA is is using um next is that to handle the the flaky key network cases if you will we support what additional replication methods that use asynchronous replication and so forth right then the next thing that's interesting is so I talk about distribution methods to achieve the speed of
light I talk about replication methods to achieve a high availability and deal with flaky networks then there's a thing is deployment architectures which is how do you actually deploy these distributed so what's interesting thing is I talked about how you can break the piece into what I call shards. You can actually pick a different way to deploy the shard on a per shard basis. You can deploy
each shard on premises in your own data center in the cloud and also across multiple clouds. So for example, I may choose to have a single logical database where pieces of the data are in the Oracle cloud in the Google cloud in AWS in Azure as well as onrem. Okay, that's an extreme case, right? And why is this interesting? Well, because for instance, you may operate a
global company and in in one country maybe there's only one single cloud provider, right? The other one is interesting is surviving cloud outages, right? Because we've seen within the last year that clouds have failed, right? And when clouds fail, it's not that the servers stop working. The servers actually running fine, right? But what happens is DNS or some networking elements are misconfigured or misbehaving. You can't get
to the servers and they cannot service requests. So effectively even so the servers are running they're down right but unfortunately we also have seen cases like in the Middle East where actual physical data centers have been [laughter] taken out of service okay so so again so you need to be able to survive cloud right and so so I guess for ultimate high availability you really want to
distribute data across multiple data centers multiple regions multiple cloud providers right and lastly interesting enough it also avoids cloud locking because if you're operating an environment like this we're running multiple clouds. If one cloud provider tries to raise his prices on you, you could just shift your data online to a different cloud provider, right? And so this actually is a very good way of of protecting yourself
against arbitrary price hikes. Uh ah so I talked about I talk about we're talking about SQL processing, right? And so you know so how to do efficient SQL processing? Um if you if you remember the examples I gave in the beginning talking about Google Cloud, Spanner, Cockroach DB, Yugabi and so forth, they have one common element which is that they all build upon distributed NoSQL databases, right?
They they took a distri distributed NoSQL database, right? Then they add a SQL layer on top of it, right? And and I think that's good for for getting things done quickly, but it's not very good for the long term because what happens is if you think about NoSQL databases, the name itself tells you it's not designed to support SQL. So trying to graph SQL on top of
NoSQL doesn't work too well, right? And so what Oracle did was Oracle has a distributed has a scale out SQL engine already that we use for our real application clusters, right? with a cluster of multiple servers seeing a single data center and we basically extended that to work across multiple data centers right so that's why it it just works better for SQL processing okay um now this
kind of something interesting which is that we do a lot of work by providing a single logical database and providing strong consistency we simplified the work of developers using a distributed database right because you don't have manage multiple copies. You don't have to manage, you know, data conflicts, wrong results, and so forth, right? However, the fact of the matter is when you're managing lots of servers across
lots of data centers, it's hard, right? It's harder to manage five servers across five data centers than than to manage a single serving, a single data center, right? And fortunately, this is where cloud comes in, right? Because cloud basically automates all this for you, right? So for for those of you who are able to go to who are in the cloud, this actually is a great solution
because the cloud actually makes it easy, right? So you get the power to just ease of use. All right. So I'm going I'm going to go go back to go go back to the AI angle again, right? So vector search and aentic AI. So the big thing with databases really is AI vector processing. We when you do approximate searches over you know try to find the meanings
of data right and that's important because today's large language models are all trained on open source data they are not trained on your company's private data right because your private data is private it's not released on the open internet and therefore to make them useful you really have to make them work against your data and that's where vector searches come in uh what the or distributed database
can do for you is basically split your data across multiple servers. So you can scale out to maybe a thousand different servers and have lots and lots of of vectors, right? And so here's an interesting example of of this which is the highest speed vector indexes. So let me let me step back. Vector searches work because of vector indexes because otherwise you have to search lots of
lots of data just too slow, right? So vector indexes allow you to make vector searches very fast. the the highest performance vector indexes, things like HNSW are inmemory, right? Which means they're limited by the size of your So what happens is then that limits how much work you can do, right? The distributed database allow you to basically aggregate the memory across multiple servers. So you're putting a
thousand servers, I can aggregate the memory of a thousand servers together, have very very large in indexes. And so so that allows you to run very large workloads. And so here's an example of we ran this h the sphere benchmark right and then you notice how we as we add more and more shards right we can have bigger and bigger um amount of vector indexes right going
from 250 million up to 900 million roughly right now so so that doesn't that that's that's good you know but what's interesting is actually the blue line notice how even as I add more and more shards and have larger larger vector indexes the response time stays the same okay and that's good because the response time is is how long it takes to do a do a vector
AI vector search, right? So, this tells you that with this technology, you can actually scale massively the amount of immemory indexes, you can do very fast vector searches at the same speed as you add more. Okay? And that's great because with AI, particularly with today's large language models, the more data you have, the smarter your AI is. That's it, right? And so so this is really a
good way to build what I call hyper intelligence which is do really really large large scale AI work on your data. Okay. So the next one is model fine tuning using distributed databases. Now I have to confess that when this idea came up came first came out I thought it was crazy right [snorts] and uh and I was wrong. Okay. And and so here's here's here's what's
going on. Everybody's trying to make AI large language models work with your private data, right? And there there are a few ways to do it. The first one is just prompt engineering. You providing you the user prompt the large language model all the necessary information for the large language model to generate useful responses. So you run vector searches internally. you you get some results and just ask
a large language model to convert to the language natural language or you can use retrieval argument generation which means you use a vector database to get relevant information then feed the relevant large language model to generate the final result. The last one is model fine-tuning which you take a large language model then you fine-tune the model you train it with your private data and of course you
must do it in your own environment right because again you don't want to give your private data to open AI or something right so so we're talking about fine-tuning now the way fine tuning works you collect the data you curate the training data you you do weights and so forth right so this is the workflow right turns out that you can run this workflow in parallel in
a distributed database in all these you know so you have very large data sets you can just run in parallel and you get very good speed ups and the reason the other reason for running across multiple shards is for data sovereignty reasons and so forth. what's what's interesting is that you know what we found is that okay let me step back. So you cannot fine-tune something like
a chat GPT right because the model is too big and uh and they they won't give it to you right but what we learned is that you can actually take an open-source openweight uh large language model like for instance mr right and which is not as powerful as chpt but if I if I fine-tune it I can actually get better results than chat GPT okay and the
reason the the reason is following a very powerful model that doesn't know is actually not as effective as a less powerful data powerful as a as a smaller model that knows your data. Okay, that's the key observation. And so we've been doing a lot of experiments and we realized that gee if I just take a say small large smaller language model but fine-tune it I can achieve
better results than a large language model right and so that's that's the key insight and that's something that that I did not believe until I saw the data right and so what this is about which is that you can actually use true databases do model fine-tuning of basically open weight databases with your private data and get better results than a publicly trained information. Okay. And the reason
is of course to to do this is uh you can do compliance so so you can to meet with your data sovereignty requirements. Uh you can do distributed processing to make it fast and you can improve accuracy, right? Because by fine-tuning it you you actually get better results. All right. So I'm going to go four minutes. Oh good. I'm on time. Key takeaways. Um so today I
sort of introduced what a distributed database is. I explained some of the some of the key factors that make a powerful useful distributed database things like and so when you're evaluating you know what distributed SQL databases to use you should look at a number of factors number one is look at the number of data distribution methods it offers right because you want to make sure that it
offers the methods that you need to handle your use case you you need to look at how many replication methods does it support you would run effectively in your networks right I would also ask is it built on a SQL engine core Right? So, such as optimize processing SQL. If you're not using SQL and it doesn't matter, but you're using SQL, this matters a lot, right? And
then if you're looking at SQL, does it support the full SQL language capabilities with all the rich data types, right? Does it support deployments in the cloud on premises, multi clouds, and so forth, you know, instead of just being available in a single cloud? And also, is it available as a managed cloud service? Okay, I think I will skip over this. uh actually I think actually I
think I have time. So, so this this is a this is a this is for slides given by PayPal, right? So, they're using this technology and they're getting, you know, the the sort of like 3 seconds recovery times with zero data loss. And the reasons I like it are these are some of the reasons I like it. So, you can see they sort of echo some of
the some of the key points that I talked about. Okay. So, I want to stay on this slide here, which is that uh this has three QR codes. One is just in general information about the stream database itself. uh the and the one to the right hand side is some sort of called live labs. Live labs are series of examples that you can access on the live
on the web and then you just run through the examples yourself and build it. There's a free version of this right then on the bottom I have two email addresses. One is my email address. So you have any feedback or questions you're feel free to email me and the other one is Richard Dval. Richard is somewhere in this audience here. Oh, Richard's right there. So Richard actually
is based in Amsterdam. Okay. So if you're interested in learning about this technology, using it or doing PC such, contact Richard. He is right here. He's in your time zone. So he can work with you much more effectively than I can from from wherever. Okay. So I think that's that's all I have. Okay. So thank you.
