Infrastruktur – News

Cleaning up Puppet code

22 April 2020

After months and years of using Puppet, the code base becomes increasingly complex and cluttered. How can you ensure its quality, as well as clean up unused code?

puppet-lint

In the Puppet world, puppet-lint is the reference for code quality. It is used as a standard to check that modules follow the style guide, ensuring consistency in coding style and practices.

puppet-lint can also be used in your control repository, to check your private modules (such as roles & profiles).

There’s at least three ways of achieving this: using PDK, a Rakefile, or onceover along with its code quality plugin.

PDK

PDK, the standard tool to manage Puppet modules in a standard way, also works with control repositories. Once installed, you can convert your control repository and run validation tests with:


$ pdk convert
$ pdk validate

Rakefile method

The Rakefile method is a easy way to automate puppet-lint:

Add a Gemfile in your repository to install puppet-lint:

You can then run the lint with:


$ bundle install --path vendor/bundle
$ bundle exec rake lint

# And if you want to autofix the detected mistakes


$ PUPPETLINT_FIX=yes bundle exec rake lint

Onceover Code Quality

Onceover is a toolbox to automate tasks for Puppet control repositories. Among other things, its code quality plugin allows to run syntax checks and invoke puppet-lint.

In order to use it, update your Gemfile, e.g.:

Refresh your bundle and run onceover:


$ bundle update
$ bundle exec onceover run codequality --no-docs

Ideally, run this on every commit in a Continuous Integration/Continuous Deployment setup. At Camptocamp, we use a GitLab CI pipeline to check our control repo using Onceover before deploying it with r10k (also running a GitLab CI runner).

Getting rid of dead code

You’ve checked the quality of your existing code. Good!

But what if you’re actually maintaining and cleaning code that you don’t use anymore? This would be quite the waste of time…

At Camptocamp, we’ve built on puppet-lint to provide a system to detect unused code and help us clean it up. This is what the puppet-ghostbuster project is for.

Under the hood, puppet-ghostbuster is a collection of puppet-lint plugins, distributed in a single puppet-ghostbuster gem. These plugins analyze your Puppet code and then connect to your PuppetDB to check if that code is actually used for any known node. It can also check Hiera data for unused keys.

Just as previously, you can set it up as a Rake task, but our current setup requires a patch to puppet-lint in order to whitelist the puppet-lint checks activated (the current release of puppet-lint only supports blacklisting checks in Rake tasks).

You can then set up a Rake task such as this one:

puppet-ghostbuster requires info to connect to your PuppetDB, so you need to provide the following environment variables:

  • PUPPETDB_URL: URLs to PuppetDB
  • PUPPETDB_CERT_FILE: path to the certificate to use to connect to PuppetDB
  • PUPPETDB_KEY_FILE: path to the key to use to connect to PuppetDB
  • PUPPETDB_CACERT_FILE: path to the Puppet CA certificate
  • HIERA_YAML_PATH: path to hiera.yaml to use

If you don’t want to provide certificates and keys, you can connect to the PuppetDB through the unencrypted port 8080, for example by forwarding it through SSH.

At Camptocamp, we’re automating this setup by using summon as a wrapper to launch the command. We store the certificates and keys to connect to PuppetDB in gopass, then provide a secrets.yml file like so:

Which allows to run:


$ summon bundle exec rake ghostbuster

This returns a list of classes, defines, files, templates, etc. that are unused in our code. We can then check these results and clean up our code!

Do you have ideas to contribute to puppet-ghostbuster? Pull requests are welcome!

You can also contact us for quotes on Puppet consulting or training!