Camptocamp – News

Puppetconf 2014

13 Oktober 2014

After a busy year of collaboration with Puppet Labs, our Infrastructure Training Leader, Raphaël Pinson, had the opportunity to attend the Puppetconf 2014 in San Francisco (USA).

This 4th edition of the conference gathered 1’700 people on site (and many more online), including both local professionals and international speakers and contributors.

New Puppet features and improvements were presented. It was also a great occasion to exchange ideas with Puppet developers and contributors.

TRAINING

Various trainings were organized before the conference since Puppet Labs has recently changed its training curriculum and introduced two new courses: Puppet Practitioner and Puppet Architect. Raphaël followed the new Puppet Architect training, so Camptocamp is now able to teach this course in Europe!

meet the authors

The Puppet world has been growing continuously over the years and more books on the subject are now written, as well as books about the Devops movement. Alessandro Franceschi, author of the new Extending Puppet book, was there to present his book, along with other authors: Nan Liu & Dan Bode (Puppet Types and Providers), James Turnbull (Pro Puppet, The Docker Book) and Gene Kim  (The Phoenix Project).

Luke Kanies Keynote

Luke Kanies started the conference with a keynote on the state of Puppet after nearly a decade of existence. Puppet is still rapidly changing and improving, and Luke outlined the new features and improvements to be expected in Puppet 4, most of which are centered on performance: the port of Facter to C++ (aka cFacter) and of the Puppet Master to the JVM (aka Puppet Server), as well as the introduction of a new dynamic node classifier for Puppet Enterprise, and the possibility to easily access Puppet Master metrics in Puppet Enterprise using Puppet Server.

To the Future ­— Goals for Puppet 4

The first talk Raphaël attended was a general presentation of Puppet 4’s goals, by its principal engineers: Andrew Parker and Kylo Ginsberg. They presented the various rewritings that will make Puppet faster (mainly cFacter and Puppet Server), as well as the new language features, detailed in greater length in Henrik Lindberg’s talk.

Performance Tuning Your Puppet Infrastructure

While it is clear that Puppet 4 will provide great performance improvements, most of us are looking to improve the performance of Puppet 3. This was the subject of Nic Benders’ talk. He explained how the New Relic team instrumentalized the Puppet Master and the PuppetDB in order to get useful profiling metrics, which helped them to understand how to tune their Puppet infrastructure.

Not surprisingly, their analysis showed that the Puppet Master spent most of its time calling the PuppetDB, and the most time-consuming calls in the PuppetDB were to the PostgreSQL database. It is interesting to note that they improved the performance of the system by adding indexes to their PostgreSQL database.

Puppet Language 4.0

One of the most expected features in Puppet 4 (and already available with the `parser=future` setting in Puppet 3) is the update of the Puppet’s DSL. Henrik Lindberg, the main author of this language refactoring, presented these new features: logical loops (Ruby style), strict object typing and clarification of object types (a number is a number, the empty string is false, etc.).

It was a great talk for anyone who hasn’t followed Henrik’s blog posts on the subject. A must to prepare for the upcoming Puppet 4 transition.

Killer R10K Workflow

Puppet is a great tool, but a great tool is not much without a proper workflow. In this talk, Phil Zimmerman presented the adventure that led the engineers at Time Warner Cable from a monolithic Puppet modules repository to a flexible workflow of individual repositories, articulated around R10K, git, a CI system (Jenkins for example) and automatic modules updates using a git post-receive hook and git tags.

A very good example of Puppet workflow, allowing a very efficient collaboration in the operations team while validating all the code with unit and acceptance tests. Their post-receive hook does not support Travis CI yet, but it is a planned feature.

The Puppet Master on the JVM

For years now, porting the Puppet Master to the JVM (and JRuby) has been an envisioned solution to Ruby’s proverbial slowness. The Puppet Server project uses Clojure to achieve this port, leveraging the technology that already made PuppetDB the powerful replacement we know to ActiveRecord and MySQL for storeconfig.

Chris Price presented Puppet Server and the actual gains that were measured with the current—still beta—version of it. He demonstrated its architecture, based on Jetty, JRuby and Clojure.

The results in terms of performance are impressive. Catalogs compile faster (even without compiling the Puppet Master code as bytecode), but their application too. By improving the response time on the Master, the Agent (still in pure Ruby) is able to apply catalogs faster, since file requests have less latency.

Using the JVM also makes it easier to share the memory between the threads. Various functionalities of the Puppet Master will gradually be rewritten in Java, making the whole architecture much more modular.

In addition to better performance, Puppet Server provides useful metrics in Puppet Enterprise, which was demonstrated by graphing Puppet Master functionalities in Graphite.

Puppet Server is already packaged and can used as a drop-in replacement of the Puppet Master without a change in configuration.

Test Driven Development with Puppet

Gareth Rushgrove presented ideas to improve and automate functional tests on Puppet. Most notably, he started generating functional tests (using Serverspec) using the resources stored in the PuppetDB.

This presentation was very interesting since the ideas overlapped and complemented the Orchestrated Functional Testing talk that Raphaël gave.

Orchestrated Functional Testing with Puppet-spec and Mspectator

It came as a surprise to Raphaël that Gareth talk’s ideas overlapped his on functional testing. In addition to this, he found out that Hunter Haugen also had a similar project, started almost at the same time as Puppet-spec.

Combining Gareth’s ideas of generating functional tests with Puppet-spec and Mspectator could lead to Puppet automatically checking its work after catalogs are applied, and Mspectator working as a kind of monitoring replacement for these functional tests.

Conclusion

Puppetconf 2014 was very interesting. It led to exchanges of great ideas for the future of Puppet. See you soon at a next Puppetcamp!