How to monitor the impact of Puppet Runs using Hosted Graphite

Introduction

In this article, we will look at what Puppet is and why it is important to monitor Puppet server metrics. We will also analyze the tools that help us monitor Puppet’s performance. Ultimately we will learn about the benefits of using Hosted Graphite by MetricFire to monitor Puppet server metrics.

Sign up for MetricFire free trial today or book a demo with the MetricFire team to understand how you can take advantage of its monitoring solutions.

What are software configuration management tools? 

Configuration management is the process of keeping servers, systems, and software functioning within a set of established rules and parameters. This process ensures that the system and its resources function correctly despite updates, additions, and deletions. Configuration management makes it possible to ensure that all devices in your network infrastructure operate in a consistent manner.

Configuration management tools perform various tasks to track and control changes to the software. They track all configuration items and document functional dependencies, ensuring consistency of physical and logical operations.

The use of configuration management tools allows you to automate a lot of tasks, which reduces the possibility of human error and the amount of labor involved. With configuration management tools, you can keep accurate and detailed records of computer system information and standards and update them as needed.

Let’s list the most common configuration management tool features:

• Application deployment.
• Infrastructure automation.
• Role-based access control.
• Task management.
• Visualization and reporting.
• Parallel control.
• Regulatory Compliance.
• Integration with other development tools.
• Fast reliable recovery.
• Quick fix and feature release.

What is Puppet? 

Puppet is a popular configuration management tool that allows you to centrally manage the configuration of operating systems and programs installed on multiple computers. It simplifies the process of defining, managing, and deploying changes across your entire infrastructure. Puppet is available as both open-source and commercial software.

Puppet code is declarative. This means that you are describing the desired state of the systems in your infrastructure, not a list of steps to achieve that state. Puppet then automates the process of getting systems into the right state and keeping them there. To do this, it uses the main Puppet server and the Puppet agent. The main Puppet server is the server that stores the code that describes the desired system state. The Puppet agent translates this code into commands and then executes it on the systems you specify.

Why monitor your Puppet Servers? 

Puppet monitoring is the process of obtaining and analyzing various performance metrics. Constant monitoring of Puppet servers allows you to get detailed data on their performance, track system failures, identify the causes of failures and performance problems and take timely measures to eliminate them.

By monitoring Puppet servers, you can get data about when Puppet runs occur, how long each run takes, how often resources change, correlate Puppet runs with metrics and events from other parts of your infrastructure, and more. Puppet can export a lot of metrics. In order not to overload the monitoring tools’ server, you can use only a part of the available metrics.

For ease of use, all Puppet metrics are divided into three groups.

• Statistical: These metrics must have all of the statistical analysis fields listed: maximum, average, and minimum measured value, standard deviation from the mean, incremental count, 50th, 75th, and 95th percentile value.
• Only counters: Metrics that only consider a value or have only a counter field.
• Other: Metrics with a unique set of available fields.

How we monitor the impact of Puppet Runs using Hosted Graphite

MetricFire is a powerful monitoring tool that provides hosted Graphite solution for storing, processing, and visualizing metrics. Hosted Graphite allows you to collect data from different sources, process, and store them in a time-series format in real-time.

MetricFire uses Puppet as a configuration management tool that makes it easy to define, manage, and deploy changes across your entire infrastructure. Information from Puppet is collected and used to get data on any outstanding issues that have occurred on the nodes.

MetricFire tracks the average execution time and the maximum average (95th percentile) to help us understand what happens to Puppet runs that don’t run properly. For example, if there are too many changes, the cluster is not responding, or there is a problem with network traffic between servers. We track how many failures there are on Puppet launches both overall and per node.

Sending Puppet metrics vis the Hosted Graphite or Telegraf agent

MetricFire uses our Hosted Graphite agent to monitor its Puppet servers. This is a customized 'diamond' agent that sends data about the state of the server (whether Puppet running on the nodes was successful or failed). Another option for easily sending Puppet metrics is the Telegraf Puppet module. 

Grafana Dashboards 

Grafana provides a lot of great dashboards that you can use to visualize different types of metrics. Grafana dashboards are highly customizable, you can choose the styles and formats of the charts, resize them and move them around. Grafana can connect to a lot of different data sources, including Graphite. You can use different data sources even in one dashboard.

Let’s look at some examples of Grafana dashboards that you can use to view different Puppet metrics.

• Active requests: This graph shows a flat line that represents the number of processors in your system. This metric indicates the total number of HTTP requests actively processed by the server at any given time, and a running average of the number of active requests.
• Request duration: This is the average response time for various types of requests made by Puppet agents.
• Request Rate: How many requests of each type were processed by Puppet Server.
• Communication with PuppetD: This graph tracks the amount of time it takes Puppet Server to send data and queries to perform common operations on PuppetDB and receive responses from it.
• JRubies: How many JRubies are in use, how many are free, the average number of free JRubies, and the average number of JRubies requested.
• Memory usage: How much heap and non-heap memory does Puppet Server use.
• Compilation: This graph allows you to understand how much each stage costs on the primary server.

Get a free trial version of MetricFire or book a demo with our team and learn more about hosted Graphite and Grafana features provided by MetricFire.

Events and annotations 

Events and annotations are used in order to identify recent changes that have been made to the configuration or group host. This is a key part of troubleshooting changes that have had a bad effect on the servers or other problems that occur as we need to confirm if the code we submitted had any unintended consequences for our Puppet cluster for debugging purposes.

The annotations are set to show up in the toolbar when we push new code to GitHub. The annotations describe in detail the commit and what happened so that we can directly understand the effect of it and how groups of nodes are faring on our Grafana graphs. This is a useful and clear indication of whether or not there is a problem in code that has been pushed recently. You can see what changes are affecting the system and when they happened by configuring these within your own Puppet Grafana dashboards. 

Having a solution to visualize what happens to clusters when new code is executed or a configuration is updated can make debugging easier as you can fully understand what changes to your configurations are good for your cluster or bad for your services at a glance.

The advantages of a Hosted Graphite solution from MetricFire

MetricFire provides hosted Graphite and Grafana as a single software product. It does all the necessary work to install, configure, and maintain these monitoring tools. With MetricFire you can use hosted Graphite and Grafana as web applications, which saves you a lot of time and money. There is also an option to install other customized plugins. MetricFire works transparently and allows you to view its internal system metrics on an open public page. It gives you control over your data. You are able to access and export your data at any time.

Conclusion

We talked about the importance of monitoring Puppet Server metrics and how this process helps in detecting and fixing problems with Puppet’s performance. We also reviewed the monitoring tools that are used for this purpose. Also, we have learned about the possibilities Graphite and Grafana provide for monitoring your metrics, and why it is important to use hosted solutions from MetricFire.

Try MetricFire for free and enjoy all the benefits of working with it. Book a demo with the MetricFire team and ask any questions about the operation of our system.