Posted on 8 June 2020, updated on 6 December 2022.
The container orchestrator Kubernetes is a powerful tool in the cloud-based world. Allowing for high scalability, reliability, and availability, it is available on all cloud providers. But its use is not restricted to a cloud setup. Here is how you can set up a k8s cluster on-premise, using kubeadm.
Why would you do this?
- You own your bare metal infrastructure
- You want to deploy Kubernetes to take advantage of it
- You do not wish to migrate your application to the cloud
Why it will not always suit your needs?
- Setup is longer than using cloud providers like GCP or AWS
- Updates to Kubernetes or kubeadm have to be done manually
- Less resilient: if your building has an outage, your service will be down
- Scaling horizontally will require you to buy more servers: Kubernetes cannot auto-scale as much as in a cloud setup
How to do this?
Step 1: The container runtime
First up, you will need to install a runtime for your containers to run in. The most commonly used with Kubernetes is Docker, but you could also use
CRI-O for example. We will go with
docker in this tutorial.
If you haven't installed it already, the docker installation is as easy as running the following command on any Linux distro. (Please refer to the docker docs for installation otherwise).
Step 2: Installing kubeadm
What is kubeadm?
is a Kubernetes installer. This means it is a tool built for the sole purpose of making it easy to install a Kubernetes cluster on any machine in a matter of minutes. Other installers exist, like
kops (repository link) or
kubicorn (repository link), but
kubeadm is the most commonly used as it is the easiest to use, and is officially supported by all popular cloud providers.
The first thing to do when installing kubeadm, is to disable swap on all machines (both master nodes and worker nodes). The
kubelet, as it functions on the day of writing this article, does not support swap memory. There is a discussion around this feature actually, so if your use case requires the use of swap, check this feature request on GitHub out!
To disable swap, you will need to:
- Remove any swap entry for the file
Now that we've got that out of the way, we have a couple of things to install on each machine that is part of the cluster. We first need to add the Kubernetes repository to our package manager.
Then, we can install all the components we will need to configure our cluster:
kubeadm, which will bootstrap the creation of the cluster
kubectlas a client to interact with the cluster
Note that we
hold the versions for all three of these components. This is key because you do not want to accidentally update your Kubernetes cluster. Or worse, have different nodes running on different versions! Updates can be tricky and have to be planned carefully.
Step 3: Starting the Kubernetes cluster
Launching the cluster is “as easy” as running
kubeadm init. However, this single kubeadm command encapsulates a lot of complexity, and there are a few points to watch out for when running it.
- Starting with a simple one, it is good to know that the default CRI Container Runtime Interface is
docker. It is possible to change it with the
- There are many options for configuring the Pod Network. The default one is Calico, but many others are available. You can pick the one that best suits your needs! Upon setup,
kubeadmwill add the
[node-role.kubernetes.io/master:NoSchedule]taint to the master node.
This means your master node will never be scheduled to work, and your worker nodes will do all the work. In most cases, this is normal behavior, but if you are setting up a single-node cluster, this will definitely be an issue. You can remove it after initiating kubeadm by using:
kubectl taint nodes master key:NoSchedule-.
- By default,
kubeadmuses the default network interface of the machine it runs on to set the advertise address for its API server. However, you can choose to use a different one, that matches your setup with the
Once all that is done, you will have an up-and-running master node! The init will take a while, but at the end, kubeadm should provide you with an output looking like this, including instructions on how to set up
kubectl to connect to the cluster, and how to have a node join the cluster as a worker:
Step 4: Joining a node to the Kubernetes cluster
To join the Kubernetes cluster as a worker node, you will need to start with step two: Installing kubeadm. Then use the command you got from Step 3: Starting the Kubernetes cluster:
Your node has now joined the cluster, and you can see it when running
kubectl gets nodes! There you go, your own bare metal Kubernetes cluster is ready to roll!
If you're looking for tips on how to use Kubernetes, check out our blog, it is filled with them!