Andrew Pruski: A Deep Dive into Docker

This talk provides an in-depth exploration of Docker, focusing on its fundamental concepts and practical applications. The speaker, Andrew Pruski, discusses container isolation, explaining how control groups and namespaces work to keep containerized processes isolated from each other and the host system. He emphasizes the importance of data persistence in containers, outlining methods like named volumes and custom images, particularly for database applications such as SQL Server. The session includes a detailed explanation of Docker networking, illustrating the benefits of user-defined bridge networks over the default network settings. Additionally, Andrew shares insights on leveraging Docker Compose for streamlining container management and concludes with a brief introduction to Docker Swarm for high availability.

Uh I'm really happy that you decided to join us to learn a bit about Docker. And with me you you can see Andrew Pruski. He is a Microsoft data platform MVP, Docker captain, a VM VMware V expert. He also or helps organize a lot of conferences. He's a passionate about all things related to databases, Kubernetes and containers, sharing his expertise by speaking at events across the globe.

And he is from Wales. So a huge round of applause and I hope you will enjoy learning about Docker. Thank you very much. Good morning everyone and welcome to this session, a deep dive into Docker. Just a another quick recap about myself. My name is Andrew Prusky. I'm a principal field solutions architect at Pure Storage, Microsoft data platform MVP, Docker captain, VMOV expert, Redgate ambassador, all round

basic database nerd. I'm originally from Swansea and Wales, but I've been living in Dublin for I think it's 12 years now. So if you're going, what on earth is this guy's accent all about? That would be why. My contact details are on the slide there at dblico.com. on blue sky. We don't talk about the other social media platform anymore. dbfin [email protected] is my email address. So if

you have any questions after today, please feel free to reach out. Always willing to talk about this stuff. My blog dbftheold.com posted multiple articles about running things like SQL Server uh in Kubernetes containers. It's basically anything I find interesting. You're more than welcome to check that out. Finally, my GitHub account. All the slides and the code from the demos we'll be doing today because this is a

very demoheavy session are available in a GitHub repo and I will post an exact link to that repo at the end of the session. So, on to the session. What are we going to be doing today? Well, we're going to be diving into the Docker platform to give you a little deeper knowledge of how things actually work. So, we'll be talking about things like container isolation. How

does container isolation actually work? We say containers are an isolated environment, but what does that actually mean? Then we'll have a look at doc some networking for docker. How does networking work? And there are some advantages over just using the defaults that we'll dive into. And then for us database people, we got any database people in. They didn't let them out. Great. Okay. So, oh, we got

some at the back. Yay. So probably the most important thing when it comes to working with containers and databases, stateful applications in general, is how do we persist our data from one container to another in Docker. Then we'll talk about building some custom images because we generally don't want to rely just on images from the Docker Hub or the Microsoft container repo. If you just pull down

images willy-nilly, you'll probably end up mining crypto. So we'll have a look at building our own Docker images. Then we'll have a look at Docker Compose and then round off by talking about some high availability for our Docker containers. But first things first, isolation. And the first thing I want to mention is there is no spoon. There is no such thing as a container. They aren't a

thing. They don't exist. Now it gets a little bit confusing because if you look in dock and say Docker container ls, it'll show you a load of containers, but there aren't actually a thing. All they are is processes running on a host. and they are isolated from their environments as it says here on the from the Docker website to ensure that it works the same manner regardless

of where it's deployed, staging, pre-production, production, QA. But how does isolation for containers actually work? Because if they're just running processes running on a host, how are they isolated? Well, they are isolated by three main constructs in Linux. The first one being control groups. So when we say hey I'm going to limit this container's memory limit this container CPU to negate the say noisy neighbor factor things

like SQL server bless for my sins I was a SQL server DBA for 15 years will take every single little bit of CPU and memory on a host if you let it so that's why we put limits on our containers but what's actually happening in the background when we put those limits on is control groups are created that actually enforce those limits for things like CPU memory

and those are the ones we are most interested in when we're working with containers. So if control groups control what a container can use, name spaces control what a container can see and they used to control a whole bunch of stuff. There's a load of them, things like mount, separating the file system mounts. So each container gets its own separate file system. The UTS namespace, the Unix

time sharing system namespace, which sounds a little bit scary, but all it really does is isolate host name and domain name, IPC namespace, interprocess communication namespace, which isolates interprocess communications, things like message cues, semaphors and shared memory. Then we have let's pick another one. Uh the network namespace isolates network resources. Processes running in a network namespace get their own IP address, network interfaces, routting tables, firewall rules.

So they control what a container can see. Control groups contain what a container can use. And the final one is the file system. Containers do not have access to the entire host's file system. they can only see a subset of it because their root is changed upon startup. So they only see a section of the actual host file system. We don't want our containers to see the

entire host file system. That would be an absolute disaster. So changing the root only gives them access to a subset of that file system. So we have control groups controlling what a container can use name spaces controlling what a container can see and changing the root of the container to limit the file Now, I said this is going to be a very demoheavy session. So, let's dive

into the demos. Can everyone see the code at the back? Give me a thumbs up. Yay. Cool. Alrighty. So, I'm running locally with a VM in HyperV. Forgive me. I'm running on Windows. And I'm going to run a container. So, I'm going to say docker container run-d run a container in the background. Publish some ports. So I'm going to pick port 15789 on the host to map

to port 1433 within the container. Port 1433 is the default port SQL listens on. I'm using SQL server images here as just a demo but this works Postgress, MySQL, Oracle, any other one. Any normal containers as well, stateless containers I should say, but I'm going to map port 15789. I'm just plucking that out of thin air because I just need a port that's not in use on

the host. Mapping it in. And there I'm going to specify a memory limit of 2 gig. Accepting the end user license agreement, which we have to do every time we run a SQL Server container. Talked to Microsoft about this. It's a legal thing. Set an SA password so we can connect in. Give it a name. And then pick in a container image. 3 2 1. Cool. All

righty. Let's have a look see if that container is up and running. And that's not very nice. So, I'm going to clean that up a little bit and use a format which I can totally type from memory and don't have to rely on code saved in files. So, we have our container. We have it up and running. Let's grab that container ID. And now we can see

the control groups created. And there's a whole bunch of them there. Loads of them. But the ones we are actually interested in is CPU and memory. So, let's grab those. Memory and CPU. So if I grab that now, if I grab that location because everything in Linux is file, divide it by 1024 a couple of times and boom, we have our memory limit. So when we specify

that memory flag in docker container run, what actually happened was a control group was created in the background that enforces that memory limit. Let's have a look at the CPU. We didn't specify a flag for CPU. So we get minus one. It's unlimited. This container can take all the CPU on the host if it wants to. We don't want that to happen. So, let's limit it to

two. Excellent. Okay, we can do this on the fly. And now, if we check it out in the background, okay, it's 200,000 200,000 cycles, essentially two CPUs. We've enforced that limit. So, all that's happening in the background when we put limits on our containers is these control groups are created to actually limits. Okay, so that's the control groups. Let's have a look at namespaces. Let's grap that.

And there we go. We got mount, UTS, IPC. I didn't mention PED, process ID. Containers, processes in a ped uh PID namespace can only see their own processes. We want this to happen. We don't want them to be able to see all the processes running on the host. There is another one called the user namespace, which allows you to mount different users in a container to different

users on the host. By default, this isn't enabled in Docker and that can cause issues and we'll see that in a second. But let's have a look at that namespace. So if you have a look at the host name, okay, naming stuff in tech is pretty difficult. So I've just named my host AP Andrew Preky. I'm running Docker on it. So Docker and this is the first

one I built. So 01 AP Docker 01. If I now execute that same command within the container using docker exec, I get a different host name. It's just a container ID because that process is in a UTS name space. If you have a look at the processes running and for SQL you always get two two top pit one pit 9. One of them is a watcher process.

One of them is the actual SQL server process. PID one pit 9 mapped to user MSSQL. If you have a look at the processes on the host, different pits, they are in a PID namespace. They can only see their own processes. Now, if I grab that PID, oh, one thing to mention here in the container, the user is MSQL, but on the host, it's mapped to user

10,01. That's the ID of the MSSQL user. It's mapped directly to it. Of course, the user doesn't exist on the host. That's why we get the UID and not the username. But it's a onetoone mapping. User name spaces are not enabled by default in Docker. But anyway, if we know all this, we can actually use different commands to jump into those namespaces. If I jump in, oh,

oh, I didn't grab the pit. And there we go. My command has changed. I've actually jumped into that name space. do a clear check the host name and check our processes. There we go. I'm in that namespace. All that's happening in the background is Docker is creating these constructs, control groups, namespaces to enforce that container isolation. One of the really cool things is that if we know

this, do we need Docker to spin up containers? No, of course we don't. If you go and check out the GitHub repo that I'm going to link to at the end, Liz Rice, I've got a YouTube video there for a session by someone called Liz Rice who actually builds a container out of Go code in 70 lines of Go code using control groups, name spaces, and changing

the route. It's really cool. Go ahead and check it out. Anyway, let's jump out of that. Let's spin up another container. And this time I'm using a custom image. And what this is doing is running SQL Server as root. And if I get the processes on the host, we can see that second process there is running as root. It's root is mapped to root. The username space

is not enabled by default. And this is why Microsoft changed their process their container images from 2017 to 2019 to run as the MSSQL user and not the root user because if someone gets into that container, breaks out of that container, they then have root access to your host. Not great. Okay. So let's go and let's create a database. Let's just create MSQL CLI connected in create

database. Excellent stuff. And there are my files within the container. They live at var opt msql data. Well, that's where the container thinks they live. If we try and do that on the host itself, lsvar opsql data doesn't doesn't exist. We sorry that container has its roots changed upon startup. So it thinks all its files are var opt msql data but in fact oh I've lost my

connection. Docker inspect that's not what I want. Copilot go away. Let's have a look, shall we? Merge directory. If we come up here, there we go. They're actually living here. So, if I grab that and slap it here, there's my files actually on the host. Oh, I was worried for a second there. Okay, cool. But that's where they actually live because the container's root has been changed.

it can only see a subset of the host file system. So by those three constructs that we can see in action when we spin up a container that is how container isolation is achieved. Control groups controlling what the controlling what the container can see and then changing the route as well. Okay, let's jump back into the slides. So the next one, let's get rid of that networking.

Okay, when you go on a Docker host and have a look at the default networks, you say Docker network ls. You will see three networks by default. None. Okay, we're doing exactly what it says on the tin here. No networking whatsoever. Really useful for highly isolated environments. You just want a remote instant container, do a load of work, and then blow it away. host effectially move remove

the partition between the host networking stack and the containers networking stack aka let the container use the host networking stack and then bridge this is the default network that we use when we spin up a container and we don't specify a network you can see it as the docker zero network if you do IPA on your host supports port mapping kind of useful we saw this in

the first demo where we mapped 15789 to 1433 within the container. Kind of useful because typically we'll spin up more than one container on a host and if they're all trying to listen on a default port of the app, we're not going to be able to do it. But the key thing here is that containers on the default bridge network only communicate by IP address, their internal

IP address. You can get them talking to each other. Say if you have an app container or a database container or if you're like me and you're testing things like postcast replication SQL server availability groups you need two containers talking to each other. You can only do that via IP address. So we kind of want to spin up something different maybe a userdefined network. We have multiple

drivers here. We have things like we have the three defaults I've already talked about host bridge none. There's MAC VLAN where you can actually assign a MAC address to your container if you want to. IP VLAN gives you complete control over IPv4 and IPv6 addressing. Uh there's also overlay which is used for Docker Swarm connecting multiple Docker demons together. And then of course there's userdefined bridge networks.

Now userdefined bridge networks have a couple of advantages over the default bridge network. One, containers can be connected to more than one of the networks. You can add a container and disconnect from multiple networks without having to restart the container, which you can't do with a with a bridge network. But the key one is that userdefined bridge networks support DNS resolution of container names to IP addresses.

So you don't have to work out what IP address your container has if you want to get more than one talking to each other. So let's have a look at some networking. Get rid of Okay, let's jump on and let's list our networks. So, we've got bridge, host, and none. The defaults. If we inspect that bridge network, we can see here, no containers running. So, let's go

and spin up a couple of containers. Docker container run accepting the end user license agreement. Yada yada yada. Using a custom network using a custom image here is basically just to install some network tools with a SQL server image if I want to jump in and do things pinging. Confirm the containers are running. We should have two. Yep. Up 9 seconds. And now if we inspect for

that bridge network, we have our two containers with their IP addresses 7172.17.0.3 and 02. So if we grab those excellent and try and ping one from the other. So I'm execing into container one. I'm going to try and ping two. It's not going to work. Default bridge network does not support DNS resolution from IP address to container name. So we can use the IP address. No problem.

And that'll work. But we have to go in and we have to write that horrible long command to pass out that JSON and get at that IP address. So let's blow those away. Now a little trick you can do is you can actually add in an IP address the host file of a container 1703 02 and spin those up. Now okay there's a couple of problems with

this. Firstly you need to know what those IP addresses are going to be in advance. And if you spin up a lot of containers on your host up and down it's not guaranteed. I'm quite happy to do it here because the container host is only ever running one or two containers and I know those IP addresses. But now I can I can ping a container via name.

Okay, not great. I let's get rid those. So let's go and let's just create a default bridge network. So if we say docker create and we're going to call it a name SQL server not specing any driver. So by default it will use the driver. And there we go. Nice and quick. Have a look at our networks. And then we now have our SQL server bridge network.

All right. So let's create two new containers on that custom host. And nice and easy. We just say docker container run. Specifying the network publishing the ports environment variables. Yeah. Yeah. Yeah. Yeah. go. And now if we go in, we can ping by name our containers from one from the other. Really, really handy when you're spec when I started working with containers, I was getting things like

SQL server availability groups up. Then the IP address is good, but what you really want is the container name for the actual instance that you're going to be targeting. You don't because of course it can change. So when you're scripting this out, you don't want to have to go off, grab the IP address, drop it into your scripts, redeploy. You can just have the container name in

there. And it's all done via creating a custom bridge network. And that's because Docker has an internal DNS server. And you can see it there 127.0.0.11. But for some reason, don't ask me why, the default bridge network can't use it. We have to use these networks. Excellent stuff. Okay. Where did my I know it's done. Come here. I love Windows. I love Windows. I love Windows. All

right. Okay. So, probably the most, as I said, important topic for us database folks. Very glad to see you here. I'm glad they let you out. Is persisting our data from one container to another. I remember when I first started working with containers and Kubernetes and I spun up a container, did a load of work, blew the container away and I went, "Oh, actually I need to

get that data back. How do I get the t how do I get that data back?" Haha. That container was gone. There is no way of getting that data back. So we need options for persisting our data. But before we actually talk about the options for persisting our data, I just want to mention one thing about working with containers, Kubernetes, Docker Swarm, all this. I am a

by trade SQL server DVA. Up until the container start coming out, I was only working with say virtual machines or physical machines with my SQL instances running on them. And I'm used to looking after my VMs, my physical machines, my instances of SQL. I'm used to patching them, whispering them to them, stroking them, whispering sweet nothings. You know, it's the cattle versus pets things that containers brought

here. Do we care about the compute really? No, we don't care about the compute. What we care about is the data. As long as something can access that data, we don't care about what it actually is. Could be anything. Physical machine, virtual machine, container, whatever. And that is what containers has brought. You don't fix a container. If there's something wrong with your container, you bye surely gone.

Pets versus cattle. It's a really cool way of thinking and it really changed my mindset of how to work in with SQL Server. I talk about a disaster recovery a lot and I work for a storage company. We have replication technologies. We replicate data up to the cloud for some of our customers. Do we want instances running there the entire time? Probably not. So what if we

could spin up a custom image of SQL Server on demand to that replicated data in the cloud instead of having VMs running all the time? Some really cool stuff like that. Anyway, I digress. Let's talk about the options we have for persisting our data. The first one, bind mounts, mounting volumes from the host. I don't like bind mounts. They are the antithesis of actually working with containers

because what you are doing is putting a dependency on the file system of the host to that container. Containers are supposed to be able to run the same way anywhere. But if you deploy a container that's relying on a bind mount that bind mount has to be present on every single host you run on. Don't like it. Now I see why people do this because hey I've

got a had a customer said that I've got a 290 terabyte database and I want to mount it into a container. I like let's roll that statement back a few times shall we? If you are running really really large databases containers may may permanently may not be the ideal solution. Now that's why people use binds mounting from the host. Then there's data volume containers. Now these are

a bit weird. What you do is you create not run a container. You then add a load of volumes to that container when you create it. And in the background, Docker creates what's known as anonymous volumes. And then when you come to spin up, say your stateful app, you don't individually mount your volumes. You just say volumes from your data volume container and it automatically mounts them

in. Really, really handy. I've seen it used very occasionally for when you have a lot of volumes that you want to mount in. But to be honest, things like Docker Compose make that moot point anyway, but it can be done. But the recommended way of persisting your data from one container to another is what we call named volumes. And again, exactly what it says on the tin,

it is a volume with a name. Let's have a look at persisting our data for SQL Server. Bear with me from one container another. All righty. How we doing for time? 20 minutes. Cool. Loads of time. Right. Let's create two named volumes. We're going to call them MSQL system for our system databases. SQL Server has 15 minutes. All right. Okay. you. Right. So we have one name

volume for MSQL system and one volume for our user databases where we actually create databases. Confirm our volumes. Cool. There they are. They are two volumes with names. Yeah. Okay. So let's create a database. Well, let's let's roll that back a bit. Let's get not get ahead of ourselves. Andrew getting all excited. Let's go and create a container. We're mounting a volume at var opt msql mounting

a volume at var ops sql server setting some environment variables for default data paths and default log paths set a container name and the image nice and quick. By the way, when I first saw SQL Server spun up in a container in less than a second, it blew my mind. I'm just like, hang on a second. I'm used to sitting there waiting for SQL Server to chug

through its install process and going it'll be finished at any point now. I swear. But having the ability to spin up an instance of SQL that quickly. Okay, there we go. Now, one thing I do have to do here is SQL runs as MSQL user within the container. It will not have access to that location I've just mounted. So, I have to change permissions. And now I

can create my Excellent stuff. Let's have a look. Let's see where it is. Hopefully, it's there. Test database two. Even check the file system. There they are. Var opt msql test database mdf mass data file ldf log file. And we can see them in the container as well. Cool. All right. There's no messing around here. Something's gone wrong. I'm going to blow that container away. Confirm that

container is gone. stuff. But we still have our volumes. This is the key thing with containers. The compute now is separate from the data. Its life cycle is completely independent. So we can blow the computer away, spin it up, blow it away, spin it up. But we still have our data files separately. So I can spin up another container through boom. Confirm that container is running. SQL

takes about 9 seconds. cuz I know we have to sit here and wait to come up within a container. And there we are. We have our database. So we have persisted our data from one container to another using volumes. Excellent stuff. And let's blow away. Okie dokie. So custom images. So Microsoft release images for SQL Server and they release them to the Microsoft container registry which we

can pull down and use no problem. However, I would always recommend you build your own image and share them in your own container registry. Now installing SQL Server in a container image is nice and simple. It's literally the same process as installing on Linux. Why? Because containers are just processes running on a host. It's all they are. But I would highly recommend you don't rely on images

for Docker Hub or Microsoft container registry or anywhere anywhere they publish releases. Build your own images because you get control over things like bring keeping them up to date, security patching, customization. And we build our images via what's known as a file on a host that contains a bunch of instructions known as a Docker file. And when we execute a command, Docker steps through each one of

those commands in a Docker file, building us our custom images. that gives us the ability to do things grant permissions to file directories, add databases to SQL Server. And so here is an example. We have building from I know I know I know said don't rely on the the images from the MCR, but just bear with me. It's a simple demo. We're going to start off with

the MCR. We're going to use a SQL Server 2019 CU5 based on Ubuntu 18.04. I know it's a little bit out of date. We're going to switch to the root user. make some directories, grant permissions to those directories, switch back to the SQL user, and then spin up SQL Server. Very simple Docker file, but it gets us over the problem we just had where we had to

spin up a container, then grant permissions, and then we could create our databases. So, let's go ahead and let's build a custom image. Okay. So, it's going to navigate to where my files are. And we have the Docker file. Exactly the same as it was on the slides. So, we're going to jump that. I can just say Docker, build me an image- tag it with a name.

We're going to call it, I don't know, custom image one. And then dot look in the current location for a file called Docker file. And we can see I've got a warning there about the legacy thing, but we can see it stepping through each one of the commands in our Docker file and building us our custom image. If we go ahead now and have a look, there's

our custom image. Anyone here work with Docker files quite a lot? An image of 1.51 gig is gigantic. Uh it is a SQL server images are pretty big. However, I will say for me it's not that big cuz when we first started we're working with SQL in Docker containers, we were working with Windows containers and the Windows container images were once I think they were anywhere between

10 and 15 gigs. Absolute nightmare to work with. Although you are talking to a man who once built a SQL Server 2014 image that was 22 gigs and then pushed it to the Docker Hub and got a very angry email about anyway. Um, okay. So, let's create a load of volumes. There we go. Local driver just running locally. We've got one for our backups, our data, our

log, our system. But now I can run a container from a custom image. Docker container run. Publish. Mapping our name volumes to locations within our container. setting a load of environment variables, a default data directory, log directory to those locations that supported by our volumes. Set an SA password, given a container name, and a custom image. Okay, check that container is running. Check the permissions on those

locations. And we can see that the MSQL user now has access to it because we set that within our Docker file, which means we can now go and just create a confirm that database is there. Excellent stuff. And to have a look at those files. So, we've got them on the data and the log files. Cool. Now, if I want to blow that away, I'm very conscious

of time here. Container is gone. Let's spin up another container. Exactly the same. Can check that it's running. And if we give it about 9 seconds, I know we wait. We can check that out if our there. Excellent stuff. Okay. So, by using a Docker file, we've gotten around that issue where we had to go in and set permissions to any custom directories. Nice and simple. Of

course, that is a very, very simple Docker file. But if we have a look here and have a quick look at a larger Docker file, go away. So if I was actually going to build a SQL server container image, what I would do is start off with say from yubunt from a base image through buntu. It' be 2204 now. 2404 probably. No, it's not supported. 2204. Uh

basically you have a label in there maintainer basically who to blame when things go wrong. adding a SQL user and then running through the installer SQL, installing the dependencies, adding GPG keys, adding a repository, installing SQL. We can install SQL server tools if we wanted to. Probably not. It's a SQL command if we wanted to go into the container and connect into SQL creating our directories and

then cop we could copy some database files in. Now, this is kind of antiattern. You shouldn't really do this. Now I say you can do this as long as they are schema only files. Don't copy massive database files into your container images. It's a bad thing to do. But schema only yeah. Okay. We can set some permissions on our directories. MSQL v setting the permissions even as

we did before. Attach DB script setting some variables and then starting SQL server. Now this is a bit weird bit of a quirk. We're running the attach DB script before starting SQL server. Now the reason for that is what happens when we run that script is the script starts up and waits 20 seconds and then attaches the database. The reason being is that a container always needs

a process running. If we had it the other way round, started the SQL server and then attached the database, the attached database script would become the main process, PID one in the container. It would start up, attach the database, stop the script, and then the container would shut down. So by switching it around this way, waiting 20 seconds, SQL starts up, becomes PID one in the container,

the main running process, and then the database is attached and the container stays running. And I'm not going to tell you how long it took me to work that out. Many, many times of why is my container stopped? Everything's fine. Okay, five minutes. Okie dokie. But anyway, that's our custom images. Pretty much went to the bottom of that. So, jump back into the slides. We've only got

a little bit left. So, Docker Compose. So, way back in 2018, I went to Docker Con. I was speaking, but I wasn't a Docker captain at the time. And I went to the speaker meal and sort of ran up to a load of captains and went, "Hello, my name's Andrew and I have questions." And when I talked about Compose, I always thought it was for multicontainer images.

And I never used to use it because I was only ever really spinning up one SQL container at a time. I didn't think it was really anything uh pertinent to me. But one of the captains said, "No, no, no. I use compose for everything." And the reason being is look at this container run statement to get a SQL container up. Docker container publish setting some environment variables

network volumes name image 15 lines of code. Am I really going to be typing that out every single time I want a container? Not really. Okay, yes I could save it as a script and execute the script, but there has to be a better way. And that way is compost. the ability, kind of like Kubernetes, to use a declarative method to spin up containers. We put everything

we want in a YAML file and execute that YAML file instead of typing out all that nonsense. So last demo, everything's gone well so far, I feel. How is it going to break now? So there's the container run statement. Let's have a look at Docker Compose. And we can just navigate to the location. We've got a whole bunch of files in here. We've got a Docker compose

file, a Docker file, which is the same as the one we saw in the previous demos, an environment variables file. Okay, I've got my MSQL password in there. Not great if I'm going to push this up to a repo, but we could have that. Eenv file say in a git ignore file and not push it up. And our we've got our compos file where everything is a

service. Specify the Docker file that we're going to build, some ports, our environment variables, and our volumes. So let's go ahead and let's just spin that up. How easy is this? Docker up-d and that's going to create a network, our volumes, build us an image, and then spin up our container. And we can see that now we now have a custom bridge network compose default, a load

of name images, and containers. Nice and simple. So I would 100% recommend that if you are working with Docker, no more Docker container run statements. Do everything in compose. You can have it in YAML files. They can be versions. You can push them to GitHub repos. You can source control them, work with them. It makes things a lot easier. Covers everything that we've talked about today. Networking

data uh networking volumes, building your own custom images. Nice and simple oneline command. And again, of course, to spin it down, docker compose down. And of course, it'll delete your container, your network, but your volumes won't be deleted unless you specify a -v flag because it keeps again the life cycle of your volumes separate from your compute. All righty. So, only got a couple minutes left. So,

let's talk availability. So, there is Docker Swarm. Now, when I first started working with containers, I was working in a company in Dublin and my QA department came to me and said, "Look, we're having a real problem here. Every month, we refresh our VMs and we rebuild all of them. There's about 40 of them and we reinstall SQL each time. And it's taken anywhere between 45 60

minutes per VM purely because of the host they were running on their Citrix Zen servers, which were old back then. And of course, this process wasn't exactly reliable either. it did fail and then they're left left trailing through error logs trying to work out what went wrong redeploying. So we rearchitected everything got rid of the local SQL installs and had containers spinning up on a host and

after we built all this. So what would happen is you'd spin up the VM script would pop off to the container host and it would spin up a container for for that VM. So 45 to 60 minutes it used to be. We got it down fully deployed with this custom SQL image with databases ready to go and have their data pumped into them. 2 minutes huge saving.

It actually saved us about a month's worth of dev time per year. We worked it out as so that's how I got involved. However, one day I came into work and that host was down. All the containers were down. Nobody could do any work. And I was the most popular person in that office until I got everything back up and running and people could do their work.

I didn't have any high availability for that to host. What I needed was something simple, nice and easy, just to provide some basic high availability. And that's what Docker Swarm gives you. A swarm of Docker engines that requires no additional orchestration software. Gives you nice built-in resilience. And it uses a declarative model just like Compose. Everything is a service. So if you need something simple and you

need high availability for your Docker containers, definitely check out Docker Swarm. It is definitely still a thing. However, let's talk about the elephant in the room when we talk about container orchestration and high availability and that of course is Kubernetes. Kubernetes definitely did win the container wars if you or the orchestration wars I should say between them. But Docker Swarm does have a simpler setup obviously tighter

Docker integration. It's nice and lightweight, but of course it has a limited ecosystem, lower community adoption, weaker multicloud, hybrid support purely for the fact that everyone is everyone's and their uncles is talking about Kubernetes at the moment. Huge community out there, really uh really loads of community plugins being built. It's endlessly customizable and it provides better HA for. So if you're looking at real production high availability,

definitely look at Kubernetes over Docker Swarm. But if you were like me in that situation and you just needed something simple and easy, that's where Docker Swarm comes in. How am I doing for time? I'm over zero. Right. Okay. Thank you. Thank you so much. Uh this is the link to the repo that contains all the code and the slides that we've been talking about today. Have

a good rest of the day. So, thank you Andrew. A really great talk. But there are questions. Oh yes, there are a lot of questions, but we will start with uh can a bind mount handle multiple writes? Uh yes, depending um if they're talking about multiple containers talking down to the same bind mount, yes. However, for SQL server databases, no, because SQL will lock the database files.

Uh and that's probably true for most database systems purely for the fact that it's to prevent that split brain functionality. Okay. So another one could you please explain again how creating custom bridge network allowed us to ping uh via namespace. Why is the custom bridge network will allow uh allows you to use the built-in docker DNS server whereas default network doesn't and that's why you can it

supports the DNS resolution from container name to IP address. Okay. So, another one, and this is also a question that I'm very interested in. Which plug-in do you use in VS Code to run selected code in the terminal? I don't I have a custom um shortcuts built that allows me to either select text or just run the text that's on the custom line by hitting control and

semicolon. Actually, if you check out my blog, I've got a I've got the plug-in for you there. Okay. Thank you. It's not a plugin. Sorry. I've got the script uh what are the advantages of using containers with named volumes rather than a database on virtual machine on permiss data storage. Well, that's the flexibility that containers gives you. So if you are running say a VM or um

a physical machine that's pretty much you've built that it's running it's there. Containers give you the ability to spin up quickly do a lot of work blown away. Ideal for things like CI/CD pipelines and whatnot. Building from custom images allows you to deploy a customized in instance of SQL Server in seconds. You don't have to do any further configuration. So you're not running through the install process.

So things like CI/CD containers are perfect for that. Of course, how can I back up my database with containers using uh without using cloud storage in the production environment? Oh, this is a great question. Pretty much because we get a little bit sidetracked when we start talking about Kubernetes and containers because I I'm the same thing. We start thinking about, oh, I'll do volume snapshots or I'll

use this plugin or that plugin. It is just another platform to run your databases on. You should be running your backups as you would where say if they were an inst physical machine, virtual machine, wherever. So for SQL Server, if you're running for a long time in a container, you're still taking your FOS, your diffs, your transaction logs, 100%. Don't get carried away with, oh, I can

do this snapshot or that snapshot or this that the other. You still need to roll through your DR strategy for that and we are near the end so don't worry. As Kubernetes on the market, uh, does it has sense to stick to swarm? Well, as I said, if you just need a simple environment that provides high availability, yeah, you can use Docker Swarm. But if you really

do need all the customizability that Kubernetes gives you or you are running, say in a high um highly adaptive fluid production environment where you need what Kubernetes gives you, you have to go with Kubernetes. It just did win the orchestration. And the last one, what if I need to run my Docker container differently sometimes, like adding changing a CLI argument to the command in a Docker compose?

Uh, blow the container away, you redeploy. That's the best thing I can say. If you need to, you don't try and fix a container, you blow it away, spin up a new one. Okay. So, anyone has a question from the audience? One quick one. How to find your blog? Uh, it's dba fromthe cold.com. Thank you. I have to run to the airport because of my niec's communion

is tomorrow morning which I totally forgot about. [Music]