Posted on 8 December 2022.
The History of GitOps
Before we go into more detail about what exactly GitOps is, it's interesting to look at its origin and the problems it tries to solve.
The GitOps story begins in 2016 at Weaveworks, a company that had just released a Software As A Service (SaaS) offering - Weave Cloud - powered by Kubernetes and EC2.
One day, an employee said he planned to make a change to their infrastructure that could potentially wipe out their entire system if something went wrong. The change was made and, as Murphy's Law would have it, causing a series of malfunctions that rendered almost all of their services inoperable, including Weave Cloud and all hosted software.
However, the Weaveworks team managed to restore their entire infrastructure in less than an hour, without any loss of data or significant customer impact. This performance was made possible by one key factor: their entire configuration was declared and versioned on Git repositories.
Indeed, the life cycle of their infrastructure was simple: never allow an irreversible change and always be able to roll back in case of a problem.
After this incident, satisfied with their high resilience capacity, the Weaveworks teams wrote down the first major rules of what they would later call GitOps:
- The entire system must be described in a declarative manner
- The desired state of the system is versioned in Git
- Approve only those changes that can be automatically applied to the system
- Use software to verify and ensure system compliance and alert in case of divergence.
This operational framework addresses the need for good Infrastructure as Code (IaC) management in a Cloud Native framework (an approach to software development that uses cloud computing to build and run scalable cloud applications).
As a result, it has been rapidly adopted by other companies as well as large Cloud Providers (AWS, Google Cloud, and Azure) due to their dominant market position.
With simple and clear principles, everyone can propose their own concrete application of the GitOps method via numerous technologies. Even if the framework was originally designed for tools like Kubernetes, it is possible to use it with other tools like Terraform, Ansible, or any other system using IaC. The important thing is that the tools are compatible with the GitOps philosophy.
What is GitOps?
Weaveworks defines GitOps today as "A path to a developer-centric application management experience. " The idea is to manage infrastructure provisioning and deployment in an automated way via tools like Git, all in line with Infrastructure as Code.
As a reminder, Infrastructure as Code (IaC) is about managing and provisioning infrastructure using lines of code rather than manual processes.
By using adapted technical tools, it becomes possible to build infrastructure via simple configuration files. This allows great flexibility of use and the guarantee of providing similar environments at each execution.
It is in the continuity of the infrastructure as Code that GitOps is registered.
The infrastructure of a project is no longer managed manually according to the needs and updates, but processed and described in the form of code defining the desired state of the latter.
The desired state of infrastructure is then "declared", it is not a set of instructions allowing the implementation of a system on a particular type of machine but a set of facts, and expected final results.
This reference state will be used as the only source of truth when it comes to the infrastructure, whether by the teams or for the various IT tools used on the project. So if a change is made to its declaration, the infrastructure should be updated to match the latest version of the desired state.
The idea of automation is very important in GitOps. Once a change is approved, it should be implemented as quickly as possible with minimal human intervention.
This desire for automation translates into a continuous integration and continuous deployment (CI/CD) approach that is usually found on the software development side. Indeed, the GitOps workflow is similar since code is sent to remote repositories but this time if to manage the infrastructure.
When a new change is proposed, CI/CD pipelines can be launched to check that it is compliant with the project standard. If it is, the change can be validated and merged to give a new desired state.
This new state is then detected by a software operator. The operator will then compare this state with the actual state of the deployed infrastructure and update it if there is a difference.
With a GitOps approach, there's no need to manage clusters by applying your new configurations by hand: if the CI tests pass, it will be automatically deployed for you. The use of CI/C techniques avoids many careless mistakes and saves a lot of time for your teams.
Finally, the last essential point of GitOps is the idea of monitoring.
Once the infrastructure is deployed by the CI/CD, we want to know if the current state of the infrastructure corresponds to what we asked.
For this, there are many software solutions that will perform this monitoring and automatically fix problems. This way, an infrastructure will constantly repair itself, guaranteeing a high level of service and freeing up the time of maintenance teams.
In the case of a problem that is difficult to control automatically (a program that crashes in a loop, a misconfiguration, etc.), the infrastructure managers will be notified quickly and precisely about the nature of the problem, thus making interventions more efficient.
The benefits of GitOps
Once implemented, GitOps will greatly increase the deployment efficiency and resiliency of your infrastructure. It will be much easier to roll back in case of a problem and you will always know what resources you need to deploy your infrastructure.
In addition, all your systems will be more stable thanks to the automation via CI/CD of the deployments while guaranteeing greater readability on the actions which are made there. Indeed, each change in your IaC is versioned and if it leads to unexpected changes it will be easily detected by the monitoring.
Tracing a problem thus becomes easier, which is a significant advantage for the restoration of service and the consideration of security issues while saving considerable time for teams.
At the same time, GitOps allows you to have a global and easily readable vision of your infrastructures via the IaC. This makes it easier for you to control them and adapt the resources needed for their operation. This better control of resources also leads to better control of the costs associated with them.
For example, managing the scaling of a set of Kubernetes nodes hosted by a cloud provider is now done in a few lines of code and can easily save you money without much effort.
By using simple and unified tools, the GitOps approach also facilitates collaboration between developers and operators by standardizing the overall development approach of your projects. The use of techniques that are already well known to developers, such as CI/CD pipelines, allows them to better understand operator issues. This makes it possible to be much more efficient in solving problems and to considerably reduce their number during development.
A GitOps practice
GitOps is not a single, ready-made product or technical solution. There are a thousand ways to use it, and each project must find a way to implement it that fits its needs. Here is a concrete example of the GitOps approach used on several projects at Padok.
Code management
For all the code repository parts we use Gitlab on which we manage the arrival of all new features/modifications via the Merge Request system. This is a tool that allows us to organize the code review and validate the modifications before merging them with our common repository used as a single source of truth within the project. We use Gitlab in conjunction with Gitlab-CI, an integrated CI/CD management tool that allows us to test code, and do application builds and deployment simulations automatically. Another important point, this CI/CD tool is manageable via IaC.
Application deployment
For the deployment of applications on the infrastructure, we use Kubernetes and ArgoCD.
Kubernetes is a container orchestrator, it aims to simplify and automate the deployment and maintenance of distributed systems. It declares in the form of code (still the IaC) the applications and other desired services, and Kubernetes automatically ensures their proper functioning.
ArgoCD is a tool that connects to a Git repository and will automatically detect the validation of a Merge Request and that of the new code. When this happens, ArgoCD will synchronize the Kubernetes cluster to match the new desired state. Thus, all applications are always up to date and operational, while limiting the number of direct human interventions.
Infrastructure management
For the broader management of the infrastructure (network management, internal resources, security policies, etc.) we use Terraform. This is a resource provisioning tool that will automatically deploy the resources/infrastructure declared in a configuration file. Once again, Terraform takes on all the concepts of Infrastructure as Code.
Monitoring
Finally, for the monitoring tools, we use Prometheus and Grafana to have a graphical view of the different metrics concerning our infrastructures. Prometheus is the tool that collects information on our various machines and Grafana is the tool that will format them (graphs, tables, etc.) to manage any alerts.
Conclusion
In the end, GitOps stands out as a reliable and efficient framework when it comes to Cloud Native projects. These simple and affordable concepts make the method easily accessible. If you are already working with IaC tools, the productivity gains and the daily management comfort will be major assets for you and your teams.
However, be careful to keep in mind that this is not a miracle method adapted to all projects without any adjustment. Certain flexibility with regard to this framework is necessary to reveal its full potential. Indeed, the "limitation of human intervention" part is an objective that can quickly turn out to be a constraint if it is taken at face value.
It's important to think ahead about your project challenges to determine how to use the GitOps framework to maximize the way you work without making your life more difficult.