Alejandro Duarte: The MariaDB Evolution: Just a fork of MySQL?

In this talk, Alejandro Duarte discusses the evolution of MariaDB and its relationship to MySQL, emphasizing that MariaDB is not just a fork but a robust database system with unique features. He provides historical context, tracing the development of relational databases from the 1960s, highlighting key milestones such as the invention of SQL and the rise of open-source database technologies. Duarte delves into the advanced storage engines of MariaDB, such as INNODB and columnstore, and how they facilitate various workloads, from transactional to analytical. He also covers the capabilities of MariaDB Enterprise, including MaxScale, which enables intelligent database routing and automated failover. Lastly, he addresses best practices for migration to MariaDB and shares examples of notable users that illustrate its widespread adoption across various industries.

[Music] ladies and Gentlemen please welcome our next speaker Alejandro dwarte presenting the topic the marry at Evolution just a fork of nql one more reminder you can ask questions after the uh presentation if we have enough time or you can also go to the slid use this QR code and ask the questions in the hall to this is the to so I will read out the questions

if they arise thank you all right right uh thank you for that uh introduction um yeah it's a default database almost we're going to see that later so that was a good uh way to start this presentation which I have titled the Maria DB evolution is it just a fork of my squl spoiler alert it's not you probably guessed it it's highly compatible though uh so I'm

a software engineer I work in Devil with Maria DB I'm the author of these three books on software development with Java Technologies mostly and I'm writing a new one called Mar before developers a little bit of operations as well if you're interested uh just take a photo of this one if you like this presentation you probably like these uh book as well once I get it ready

because I'm on chapter five of 10 or something like that anyway that's not what we're going to talk about today the idea is that I'm going to give you a historical context of how my sequel was born how Maria was born uh we're going to get a bit Technical and talk about storage engines which is pretty cool feature of mar especially of Marb uh Mar Enterprise of

course I have to mention that the present and future just a few words about it a bit of migration I'm going to give you a tip on that one and I always get asked this who is using Mar in fact you already asked me before uh we started the uh talk who is using uh Maria DB uh so I'm going to answer that for you later let's

start with the history of relational databases going to be super super brief I don't pretend to give you all the history of databases which is fantastic by the way uh it Al start in the early days in the um 60s with General Electric creating something called integrated data store ideas is the very first database that we know of is not relational yet that doesn't exist at this

point it led to the development of the Cal database model which basically is a cobal extensions where uh developers have to write nested loops and pointers it's pretty low level to create the database and that means they have to rewrite this code anytime the schema changes now how the data is stored in this so got C realizes okay these guys are wasting way too much time rewriting

these queries and he proposes their relational model which is tables right of columns and and rows uh he's a mathematician or was a mathematician so he formalizes this with the relational calculus and relational algebra which is what actually databases use and this allows to build query U optimizers right so databases have all these algorithms there like super uh uh specific uh algorithms for certain situations there are

many of them and they can pick the best algorithm given the circumstances of the data itself they collect for example uh statistics about the data so I'm I'm going to pick this algorithm when I have 10 rows but I'm going to pick that one when I have several millions or even billions of of rows and they're much better than any any um software developer really U all

this is Theory until around maybe even in in the 60s this started already but this exploded maybe during the' 70s uh it started to appear the first implementation this was all all theoretical before so one that's very very famous is system R which sometimes you find it in articles like if it was a database by IBM but it's not uh from what I have read I have

tried to read as much as I can about the history of databases I understand this as a project research project really they don't want it to develop a product that they were going to sell with this project incre in the University of uh California Berkeley uh the precursors of postgress Oracle wi know oracle mmer in the University of oops saline Sweden another academic project and the predominant

uh quer language was ql this means something like although there are different interpretations of this quering using the English language or people say just quering language or query language uh I think the most precise is uh quering using the English language so let's remember that one Co now later uh the scientists in especially at IBM the computer scientist they wanted to compete with these uh with these

language so they started to they wanted to compete with the University of California basically they wanted to have something better than than Co which I don't know if you have seen it it's not I don't think it's that great it has some cool things I have to say but I think equals much better and they started a game that they call Square I see it as a

game which is also kind of a but it was more like a game they have among them that's specifying queries using a relational environment they want to come up with that with a best solution for running queries against databases and uh it is the scientific notations it has like a super indices super indic it's really hard to to type on a on a keyboard so programmers are

not going to use this they redesign it and create something called SQL so they are kind of playing with the words the SQL to quill basically uh however this uh was a trademark in the UK I think it was an aircraft company or something like that they cannot use this they remove the vowels and you get SQL and that's the reason we say SQL right uh we

continue to say SQL especially uh Native uh English speakers they would say SQL and not SQL now the it got standardized after these databases adopted the language later in mid 80s and the standard does call it SQL but uh I like to say SQL because it kind of reminds me of this uh interesting uh story now to give you some context open source was well established uh

in the 80s with the G project the general public license GPL which uh well I guess some of many of you at least probably know that that's one the most popular data databases uh licenses for uh free software it guarantees that the the if I give if I distribute software I have to give access to the to the source code uh it protects protects the freedom of

the user they say at least the G new project like to think so the Linux uh uh operating system was under development it was released under the GPL POS is also under development vment it doesn't it's not GPL but it's under development uh with many many students contributing code to it but there are no free SQL databases because pogress wasn't designed originally to work SQL this changes

with the very first free cql database which was was called mini SQL or msql for short um this offer very performance then progress progress progress was I think implemented back then in like list or something like that so it wasn't really super performance now it is uh is still in use uh especially in the beted devices not heavily maintained that's the latest release 2021 but it's still

in use um however there is no open source database that supports SQL this one is free you can now at this point you can try databases and SQL for free you have to pay but you cannot see the code of this one so that changes with my SQL so Mel Venus uh created this datab um in Finland call it my squel my not because of the English

article my but the name of uh one of U his daughters I think in Swedish might be something like a I don't know if anybody uh speaks Swedish I have no idea uh some say Maya I think you would say Maya SQL or Maya SQL my SQL was born uh because of the need of creating a fast database he wanted a fast database for his customers uh

that was easy to use mini SQL wasn't that easy to use yeah it had some limitations no I'm talking about the 9s so this long ago it had limitations that's true but it turned out to be a great fit for websites so you can say that my SQL helped shape the internet as we know it today it was released under the GPL license now this database gained

popularity during the next decade almost and INB was introduced to solve the limitations that M have this is a storage engine we're going to see for about it later my SQL Finland AB is created a company to offer services around these open source database and then Oracle starts to make a series of strategic strategic decisions like let's call it that way they buy in aase which is

the developer of INB why I guess we know why Sun then some Microsystems buys my SQL some Microsystems had Java and these amazing hard word uh but analysts think that uh what happened next was due to my SQL I bet some of you remember what happened next Oracle by Sun micro system that was announced I believe in in December uh 2009 it was made effective pretty quickly

the next year now Oracle owns not only de most successful commercial database Oracle but the most popular open source database myquel of the moment so Mikel viel thinks there could be a conflict of interests maybe Oracle will kill my squel maybe they will stop Innovation I'm not going to be the judge but I'm going to show you this funny uh chat on a slack on the my

SQL slack somebody said I don't see much innovation in the 8.1 Innovation really these there are plenty of deprecations is that a new definition of innovation at Oracle they are joking for real but I let you decide whether there is truth in this or not mik VI thought so and the community around myal thought so and they copy the code it's GPL they can do it and

start a new project but not only that the developers of the my SQL project almost all of them they move to this new project that you you guys is mar Maria is another daughter of uh mik vienas and we're going to see something called Max scale Max you guessed it right so it was born as a fork of my SQL and in initially they wanted to have

this super high compatibility with my SQL to the point that you could just replace it so they called it drop in replacement it's not the case anymore the first version was 51 we are in 115 or something like that obviously it has to be GPL license and it gained popularity very very very quickly and like you said it's a default database in many uh Linux distribution to

point that when you when you type uh my SQL to connect supposedly to my SQL on a Linux distribution you're actually connecting to Marb in most of them not all of them um you can see it with the dbm popcorn popularity package that sends information of the packages that are installed in certain machines you see merb and blue are getting more and more installations and similar in

Windows I'm going to skip this so Marb gains popularity the Marb Foundation is uh created uh to avoid that what happened uh to my SQL with Oracle happen to Marb in the future and the Marb Corporation now it's called plc is also funded with M VI in both organizations this uh company creates faster connectors for Java python C++ Etc and additional storage engines as well as well

as some services so let's talk about that storage engines which is uh in my opinion one of the coolest features in marid and there are many many many many uh you see INB there already we already mentioned that that's for transactional workloads so operations it's call that way ironically you see column store in a horizontal way analytics so a some this kind of ad hoc maybe what

else my rocks created initially by Facebook that's right heavy workloads area which is like Maria without the DM that's where the name comes from uh that's for uh read heavy workloads memory you don't have to write to this you have S3 for the cloud CSV anyway there are many so how does this work so let's say we have this server this one instance of Marb there you

have a SQL API which is that what the applications are going to use right through something called connectors for again Java python C++ nodejs now that those queries or those uh statements they go through the uh storage engine API and then these layer picks one or more of these storage engines but notice that they are all in the same server they coexist at the same time in

fact when you install Marv and run show engines you get this so there already several engines installed there then these uh they Delegate for example to write to this or memory or whatever whatever it is so here's the cool thing about it you can do this so let's say we have an app and we're going to get tons of comments there so we say create table comments

some columns with the engine equals my rocks because it's going to be right heavy we're going to have many of those that that's actually the use case that Facebook had uh now we're going to save in on storage then we create another table categories some columns we say engine area because categories they don't change ever or they change every five or 10 years or more now to

be honest I would have used in be there but I just wanted to show you the example now you can run select something from comments join with categories and now you have the two storage engines in one single query both acting according to what they were uh developed for so that's a cool thing about it crust engine squel queres if you want to learn more about these

uh there's a video where I quickly go through all these uh um stor engines uh mentioning what exactly it is which one um feel fre each to navigate through those videos uh let's talk about analytics because I mentioned the colum store so you can do analytics with uh um Marb another another difference with my SQL we were talking before uh the talk uh so let's say I

have Marb now on a Docker container and you can try this very easily I'll show you how in a moment I create a schema or a database there are anonyms already and there I'm going to create two two tables doctors and appointments both use INB now I can create another schema or I could have created the table right there put another name but I think this more

organized for the example I call it the same it has the same structure or very similar to the appointments table with the same name in in the operations schema and we can copy this data so you know the basic data b line now in reality what you would do is maybe something like this you will have different servers maybe in the cloud and you could use something

like replication actually you wouldn't do it like that you would have maybe many servers there with column store because this this is for when you have billions and billions of rows and super fast uh you can escalate and then you you use something called CP import it doesn't matter where it is there's a tool for for copying this data from one database to the other back to

the example let's say we insert 4 million rows which is not even allowed for a relational database uh well I have these query so I'm selecting it's an analytical query so some average I'm all these details then from appointments and join doctors in the operations schema so both are in a DB this was slow it was a bit slow and it's not because you know DB is

slow if you run this with any other relational database you'll get the same results in fact INB is one of the fastest storage engines in the market now I'm going to join instead operations. doctors oper uh sorry analytics uh operations. appointments analytics. appointments which is column store this was fast much faster and this is in this machine that I'm presenting on these slides um so obviously I'm

not using indexes here an index would have made that as quick but this remember these are ad hoc queries we're creating these query writing it we get the results we forget about this query uh it's Analytics if you want to try that out it's a lot of fun to try it uh I created a Docker compos file just say Docker compose app you get all the infrastructure

the instructions to read uh to run some of these queries and then do compose down you you have had fun it's very very very cool to see it in action now let's mention Marb Enterprise which is I call it subscription really uh that's just a marketing thing uh but it's kind of true we don't want to be interrupted that the server was down or something like that

during the weekend right so it includes the Maria Enterprise server which uh is based on the community server but it has better maintenance non- blocking backups so you can have to stop the server Enterprise audit more things on the security side and it has pretty cool uh database proxy called Max scale let's talk about that one I call it intelligent database appro so approx database protion this

this case is something that sits between the U client and the server right in this case the web server and the database in a way that the web server in this case thinks it's talking to the database server and the database server thinks it's talking to the uh to the client to the to the web application in this case they don't they don't know there's something in

the middle and it's intelligent because it understands SQL so you can make decisions on what to do with the SQL or where to send it for example if this is not just one server but uh maybe a replicated topology now it understands also uh mql the mongodb query language so if you have a an application this case I put the Java logo there it has to use

this mongodb driver and then send the mql to mongodb right well you you instead of sending it to mongodb you can send it to Max scale it ers exps mql and not only that it transforms that into SQL and stores data in a relational database in Marb that's pretty cool because now you can join data that was created through SQL and no SQL in a single SQL

query so it gives you a 360 view of your data if you want to try that out that's also another way to uh to do it I think through a Docker compose as well I'm not so sure it's very easy uh you find this instructions on the demo there if you want to see it in action um did I said it was did didn't mention was intelligent

well it understands scfa as well now here I put the uh uh database on this side and you can use it for change data capture for example which is a uh change events in the database in terms of schema or data then from CFA to all the multitude of uh of connectors that there are for kfka and the opposite to ingest data back into a database read

right splitting for uh scaling so that's another very interesting use case let's say we have these two database servers and you enable replication if you want to learn how to do that there's another video I think at some point I stop showing these things just getting out of control I guess uh so the data that you WR here is available also on the replica almost in inally

it's not it's not right away there's a small lag and there ways to adjust these things you can put Max scale here and send the right to the primary and the reads to the replica so that's uh right right split then your application will send this equal to to the proxy remember it's a proxy so this doesn't this has no idea has this complexity there in fact

this is a single end point I think it's just one end point you can add more replicas or remove them when you don't need them to save money for example you need to change the application this happens without restarts or anything in fact you can change the whole thing you can evolve something like that now we have maybe even three cloud out if you want uh with

a mixture of INB and column store U the application doesn't know this is called body isolation let's talk about automatic fail over which is something that all these schema allows us to do and U let's say that one is the primary node so it's the one handling the the rights at least on that that cloud it could fail at any point Max scale detects this picks another

one one of the replicas and reconfigures it and promotes it as a new as a new uh primary so WR continue to happen you don't have to do anything this completely automatic eventually this one recovers maybe you maybe automatically maybe you are reboot the machine whatever it detects that it's now available reconfigures it as a new replica and all of a sudden you have the same capacity

as I mean those notes are are equal you can do also a manual switch over to our U UI or command line or configuration files now let's talk about the present future of midb So currently you can deploy midb anywhere on bare metal the cloud this one is a a Raspberry p cluster that I have in my office in in Finland uh I think I use Docker

there doer swarm when I took that photo uh basically you can't disconnect any cable there and the thing continues to uh to operate I have an application um heating these databases much so I can uh I don't see any any uh errors in the log of the application yeah when I disconnect something there's a tiny lag for sure while things uh reconfigure but it's pretty cool to

see that it survives uh I have I think I have two notes with Max scale which you can replicate the configuration of Max scale also one you change one the other one changes as well and then you have to to note so it's not a single point of failure right and then I have probably one primary node and a bunch of replicas uh I don't recommend the

setup for production databases with tiny SD SD cards probably not the best idea but it's a lot of fun to do it you can do it with a virtual machines but there is called multipass and if you know it to create virtual machine it works Windows Linux Mac single command you create a virtu machine you can emulate something like this a lot of fun now you can

deploy obviously on any Cloud the team is working right now on kubernetes uh deployments and orchestration so they are uh there was a guy I think he was in Spain he was de developing probably the best operator for Marb so we hired him now he's with the team and I think we are soon to release this uh with ton of feature we just need to uh complete

some more tasks around the project uh so stay tuned on that if interested and AI capabilities everybody's doing this in terms of database uh there is a upcoming product or feature that we're are calling for now Marb Vector for Vector search the strategy of the company where or or this is more to be honest this is more uh uh Guided by the uh Foundation uh they are

doing the the the most of the work there they didn't jump right away to you to providing one implementation of vector search they wanted to see what the industry is going to do what's the best strategy let's wait let's not get crazy so I presume there I haven't tried myself I have to say it but I presume they're doing a great job because of these precautions they

have had because they are taking the steps to make sure that they deliver a great great uh product around Maria DB so that's what's going to happen in the future now uh how to migrate to Marb so we have documented uh the main databases that our customers want to migrate from and to Marb so we have a lot of notes maybe for my sqls you could ver

pause resl Oracle there is a tool that we have to ease migration it's an assessment tool migration assessment tools that tells you uh where you could find maybe problems that you have to tweak manually uh but this is boring so I'm going to show you this one instead um the sequel compatibility modes but I I don't think any other database has this and it's basically a variable

that you set uh through a uh SQL statement or you can use a configuration uh file to set this parameter for example Oracle and now Oracle sorry MB understands uh subset of the dialect of Oracle yeah not all of it data being impossible but it really really helps with migration in this case from Oracle and you find the same from for poser SQL for SQL Server so

is that easy it understands a subset dialects now who uses all these stuff uh we ask me that I didn't really answer that that well when you were asking I apologize for that but maybe this will answer the question a bit better um here are some notable contributors so I I don't think I need to explain of those uh Brands they are huge huge Brands they have

tons of uh um instances of Maria DB running uh I put Maria DB there because so Maria DB this is the corporation that's not called Corporation anymore that they they hold the the rights on the on the brand and all that uh but they also are the main contributor in terms of in terms of commits to the to the community server in fact what you get in

Enterprise Edition most of it eventually makes it to community sometimes it doesn't even make it to makes it to the U Enterprise but directly to the community most of it so you see the the stats it's the company providing more code to the community server as such uh uh but well you have some other big companies here now some notable users uh Wikipedia when you read something

on Wikipedia you are reading information stored in Marb no redhe Samsung if you have Samsung devices um the uh authentication uh layer for the I think it's uh I don't remember the name it's a portal where you login with da services that goes through Maria DB when you pay with a credit card it's very very very likely that's going through through OMB fortunately I cannot mention the

name of the company but you have heard it uh Google WordPress DPS it is a huge Bank in Asia one of the largest they migrated from Oracle to Marb they're so happy they talk at our events about their experience with Marb uh and they have uh tons and tons of servers also running running um Marb in short 75% of the Fortune 500 use Marb so almost all

of them but not only big companies small companies too because Merv has more than one billion downloads on Docker and I'm not counting but I presume it's going to be similar with the number of installations from Linux so in conclusion we sell the Marb Evolution from even before from the context of databases where data was storing these kind of machines to today where one one click maybe

through kubernetes you install the full cluster or even I don't know Raspberry Pi if somebody wants to try that I'm I'm going to leave you with this message nobody says wuntu is a fork of deian or Microsoft SQL Server is a fork sbase yes that's true but they are historical remarks mostly they don't Define these Technologies so in the same way Mar DB is much more than

a fork of my SQL and I hope you saw a little bit of why and what's the difference with my SQL thank you these are my coordinates and if there is time and questions I can try try to answer some of those so Applause for the uh speaker first thank you and now it's the time for questions I think this one on slider actually was already answered

right but if you can elaborate once more is there something like Cloud native PG for Maria deis so operator yeah that's correct there's a an operator being under development right now that's a great uh question um it's under heavy development right now so I don't think it's ready for production but it's going to be ready very very soon do you know where we can or when we

can try it out no if I say something yeah I cannot don't don't quote me on these by the way all righty next one hi uh thanks for the uh detailed historical and giving some more context on Maria DB uh one question which always conflict um for users is that for example Maria DB has this called Galera cluster technology Y and uh then there is also something

another player which uses Pera extra DB can you maybe give a little detail or differentiate like which is the best case to use because some adopt gerera but uh in some cases maybe it it is good or bad so yeah so gallera is for high availability basically you cannot grow with Galera really at least not more than one server because each node is the same in a

gallera cluster so it depends on how powerful certain server is you cannot just grow like in terms of for example amount of data uh each server is the same that's the main thing with gor it's widely used is a amazing technology but it's Us only for that really if you want to go with uh um scaling horizontally scaling reads then you can use something like I I

saw there I show there which is a Max scale with uh read WR splitting but that's for reads now you cannot scale rights if you want to go um scaling both then you have to go with charting and there is a storage engine for that called spider for to mention that one it's very cool so it allows it eases uh charting data charting so now you don't

have the whole data in one server you have it over multiple servers and you can use replication so that one is one the data that died here is still there so those are kind of the the scenarios I'm not very familiar with uh oron though thank you great question questions hi thanks for great presentation so I have question regarding the different engines is there any recommendations when

you're using cross engine queries for example which one is fits better which one is not the best choice and any other hints thanks great question it it I mention this there as if it was super simple but like for example in the case of column store a great fit for analytics as I showed uh there are some twick there and there are some limitations and there are

things and the same I'm afraid would happen with all the storage engines you cannot get it all right in one single database that's the reason the ecosystem is so rich and we have so many databases andb tries its best to provide to extend its feature through stretch engines uh so it depends yeah obviously depends right but that that boring to say I'm going to elaborate so you

have to see the features of storage engine that that you want to use so if you have let's say right heavy a right heavy workload and it's it costs a lot to store thatt in the cloud right now we need to save costs or we want to you can move try and move to um my rocks and that that the compression is there you have to pay

somewhere right probably it's on RS speed this kind of thing and maybe there are some limitations so you have to go through the engine and see okay this one's good for this but bad for this so forth and then on Cross engine what happens is that it delegates to each engine each part so there's also a there could be a penalty there as well but for example

if you're doing like an analytics again which is maybe the EAS to understand an average and the column on which you are using the average or columns are all on column stor and we going to use only column store and going to be quick depending on configuration blah blah blah but if you if you include one that it's on say my rock or or INB then it's

going to use that performance because it obviously has to wait for that data uh so in short check I think there is a page I should have put it there where you see very very shortly or the video maybe I mentioned that which engine is good for what and then from there you can guess that okay if it's good for that it's not good for the other

thing it's always at that there's a trade of now inp is the the what you are going to 99.9% of the time uh but yeah great great question thank you thanks for that