The automation of server management is essential for monitoring changes and reproducibility of environments. This is why Camptocamp manages its park with Puppet.
With its three divisions (Geospatial, Business, Infrastructure), the Camptocamp group is inherently organized around the DevOps methodologies. The main objective of this approach is to align our employees (specialized engineers, business analysts, integrators, testers) with common objectives and values, work together to produce the value of the company and give satisfaction to our customers. Promoting agility and organizing our teams around automated and transversal processes is at the heart of our concerns.
In the late 1990s, the advent of the Internet for the general public saw the emergence of large computer parks with new challenges. This multiplication of configurations has become even more pronounced with the advent of virtualization, making it possible to provision machines very quickly.
In this new paradigm, machines have become short-lived and new needs have arisen:
- standardisation: installed machines must comply with a common basic base (compliance)
- reproducibility: the machines must be able to be reinstalled at the same time at all times
- parallelization: it must be possible to instantiate several machines in parallel without multiplying the human working time proportionally
- change management: any change must be auditable and logged (quality/rollbacks)
- inventory: machinery must be known at all times and their condition must be known
The adaptation of the operational processes (ITSM, IT service management) makes it possible to address some of these needs. However, only the automation of machines management allows significant efficiency gains.
As early as the 1990s, configuration management tools were developed, such as Cfengine (1992). In 2005, Luke Kanies created Puppet, a configuration management tool combining Cfengine principles with simple language and modular architecture.
Camptocamp took an early interest in Puppet. As early as 2007, Camptocamp was using Puppet in production and was part of the small number of companies distributing open source Puppet modules.
Like most Puppet users, we started with a stack Puppet Master based on Apache and mod_passenger. From its 0.2.0 release in 2014, we adopted Puppetserver (based on the JVM) as a replacement and were able to assess the gains in performance and ease of implementation. We then published a module to manage the deployment and configuration of the Puppetserver with Puppet (still used in the community to date).
Over the past 10 years, Camptocamp has continuously modified its Puppet infrastructure to keep pace with technological advances.
Our teams have been deeply involved in the community through many contributions:
- Forge Puppet module (most important community contributor)
- developers Augeas and Augeasproviders
- active members of Voxpupuli, the module steering group in shared management of the Puppet community
Since 2014, Camptocamp has been a partner of Puppet Inc. As such, we regularly teach courses in the official curriculum at all levels. The vast majority of our engineers are Puppet Professional certified.
Puppet is implemented around a central component: the Puppet Master, which provides a source of truth for machine configuration. Based on the Puppet code (manifests), data (Hiera), and possibly external sources (e. g. Foreman), the Puppet Master allows each machine (node) to retrieve at regular intervals the target state it is in charge of maintaining.
The entire Puppet code of our teams is managed in git and deployed on the Puppet Master using the r10k tool. This allows us to do collaborative code review, as well as integrate our Puppet code into continuous integration processes that test the code before it goes into production. We implement unit tests („pure“ Puppet code testing) and acceptance tests (simulation of deployment in disposable virtual environments) to ensure the quality and tracking of our code.
We also have a difference calculation system that allows us to evaluate the impact of a code release in detail.
Our nodes are classified by Puppet certificate extensions. This gives us a great flexibility of classification at the time of provisioning, since Terraform can easily generate a Puppet certificate pointing to an environment and a precise role for the node being created.
We use Hiera with the eyaml-gpg plugin to manage the secret parameters of our Puppet code. This allows us to have a fine-tuned management of administrator rights over secrets by using individual PGP keys.
The automation of provisioning requires automatic management of Puppet certificates. We use the method of automatic certificate signing policy based on shared challenges. This method allows us to generate Puppet certificates through Terraform, and thus easily install new machines without human intervention to sign the certificates.
We have also set up a system for the automatic renewal of Puppet certificates when they are about to expire. This system uses the Puppet code itself to recreate the certificate of the nodes if necessary.
The management of the park in the form of code allows to set up an agile management of the production, by the use of processes of continuous integration and peer reviewing.
The Puppet Master provides reports for each managed node, which can then be processed and viewed in web interfaces, such as Foreman or Puppetboard.
Puppet’s machines management can be easily integrated with provisioning processes, for example using Foreman, VMWare Orchestrator or Terraform.