Posted on 25 April 2024.
What is a hypervisor?
To understand the role of a hypervisor, one must have at least some knowledge of virtual machines (VMs) and core virtualization concepts.
Virtualization is the reservation of some of your computer’s resources to separate the processes running on the computer completely and some other processes you want to run in a different environment.
Resources allocation in a computer with and without VMs
Once these resources are separated or allocated, as this separation is called technically, an operating system (OS) can be installed on these specific resources to create what is commonly called a virtual machine.
In less technical terms, it’s a computer running inside your computer while taking some of its resources to function.
Software that allows resource separation is called a hypervisor, and the most widespread of them is KVM (Kernel-based Virtual Machine).
KVM is a hypervisor implemented directly in the Linux kernel (the core of an OS, which links the physical resources to the software). This makes it highly efficient because it allows the reservation of physical resources.
Architecture differences between a type 1 and a type 2 hypervisor
KVM is straightforward to use if you want to deploy a couple of virtual machines. However, it rapidly becomes a nightmare if you want to manage a whole infrastructure with it, which is when Proxmox VE enters the picture.
What problems does Proxmox VE solve?
Let’s imagine for a moment that you manage a company's infrastructure that hosts 10 web apps. Each needs at least a web server with a remote database, which amounts to 20 virtual machines needed.
Imagine managing all of them via CLI, making sure the resource allocation is correct when you boot the machine, and needing multiple commands to know the resources each machine uses.
Oh, did I mention that you also need to set up the network if you want to isolate your workloads?
Proxmox VE eases this process by offering an interface between you, the sysadmin, and KVM. To be more specific, Proxmox VE presents itself to you as an interface (web or CLI) that you can use to set up everything you need to configure in a private cloud.
The first thing you’ll probably use with this tool is to boot a VM. With Proxmox VE, it’s as simple as clicking “Create VM,” choosing which OS you want to use, what resources the machine needs, and which network you want the machine to go in, and that’s it!
Of course, you have some more advanced options, like disk partitioning and network interface configurations, if you need them.
Summary interface of a running VM
The way that the network works in Proxmox is quite simple. You first have Zones, which can be as simple as a local zone, to allow your VMs to communicate only inside a node. You can also set up VLAN or VXLAN zones if you need communication between nodes.
Inside these zones, you can create VNets (Virtual Networks), which allow you to manage subnets, for example. Once you have these basic bricks, you can easily use Proxmox to set up DHCP, DNS, and IPAM inside your cluster.
While I use the words nodes and cluster, I should probably explain what they mean in our context. A node is simply a server on which Proxmox is installed. In a basic setup, only one node contains all the necessary resources.
However, you may want to use more computing power as your infrastructure grows. To achieve this, you can link multiple nodes to create a cluster, for example, where the nodes can share network configuration.
You can also use this cluster configuration to store your VMs on NAS storage. With this NAS storage, you can have some of your nodes be absolute computing beasts, with a ton of CPUs and RAM and some nodes dedicated to storage, which can be a lot cheaper than having every node be a massive server.
When should you use it in the public cloud era?
Now that we’ve covered what Proxmox VE is great for let’s examine when to use it and how it compares with the current public cloud services.
Cost of infrastructure: Private cloud wins
If you are well versed in FinOps or look at your cloud provider invoices every month, there is one thing we will all agree on public cloud resources are expensive. On AWS, for example, an 8 CPU and 32 GiB of RAM EC2 instance will cost about $250 monthly, whereas a physical machine with the same capacities will cost about $75 on OVH (both prices are estimates, without reservation discounts)
Installing Proxmox VE on some of these OVH machines will allow you to drop your computing costs drastically. If you already own some resources, you can easily integrate them into your cluster.
Ease of usage: Public cloud wins
That one is a no-brainer; if you want to boot a Virtual Machine using Proxmox, you first have to buy a (physical or virtual) server, install Proxmox VE on it, and then use the interface to launch an instance with the desired configuration.
If you want to boot a Virtual Machine on public cloud providers, click “launch instance,” and… that’s it! Your machine is available to use almost instantly, and you’ll be billed at the end of the month for its usage.
Sovereignty: Private cloud wins
One point that may affect you is the sovereignty of data. When you store something on a public cloud provider, you don’t really know where it lands. Of course, their infrastructures are regularly audited and secure, but you can’t know which disks hold which data. If you setup Proxmox VE on machines you own, you know at every moment where your bytes are hosted, which helps protect your data from leaking.
This sovereignty also helps with the portability of your data if you want to migrate your infrastructure. Each public cloud provider has a different way of storing VM disks, and moving between providers can be quite a hassle. With Proxmox VE, you can export a raw image of your disk, boot it wherever you like, and you’re done!
Cost of operation: Public cloud wins
This point is difficult to settle on. On both sides, you need quite an extensive knowledge of the technologies if you want to operate on them. Yes, launching an EC2 on AWS only takes 2 clicks, but managing your VPC, your NAT gateways, your load balancers, and so on, forces you to understand the provider well.
The same can be said about Proxmox VE, the basic setup is pretty simple, but if you want to go a bit further, you better be an expert in layer 3 networking.
What really settles it for me is data storage management. On private cloud, you need to manage your RAID array configuration, your disks’ partitions, and eventually your Ceph cluster above this. On public cloud, you just say “hey, I need 2 TiB on this instance”, and some cloud magic happens to give you the storage.
Ability to scale: Public cloud wins
And this is where the public cloud providers take the lead by a landslide.
Do you want to add 10 TiB of storage to your Proxmox VE cluster? Let’s order disks, install them in your NAS storage, and configure your partitions to use this new space.
Do you want to add 10 TiB of storage attached to an EC2 instance? You already attached some elastic file storage, so it will auto-scale to petabytes if needed. The same can be said about every resource you think about.
Yes, public cloud providers are expensive. But that’s because they allow you to scale every part of your infrastructure to a degree you would never have imagined needing, all in a matter of minutes.
Relevant xkcd about the difficulty of managing infrastructure. Source: xkcd
Conclusion
Proxmox is a very good tool if you want to host your private cloud. It allows you to manage your resources with ease and have a single access point to use them. This is especially useful if you already own some servers that are becoming increasingly difficult to manage.
However, in a world where public cloud providers become increasingly more efficient, one can argue that the time you will need to operate and scale your infrastructure is well worth the extra bucks given to providers to ease your work.