Monitor IoT devices using MQTT, Hosted Graphite & Grafana

Introduction

IoT devices and technologies are very popular now. Therefore, an important task is to ensure the smooth operation of IoT infrastructures. In this article, we will learn why and how to monitor MQTT metrics and also look at the tools that you should use for monitoring.

In order to start visualizing your metrics, get a MetricFire free trial. If you want to know more about the features provided by MetricFire, book a demo with our team and we will answer all your questions.

Key Takeaways

1. MQTT (Message Queuing Telemetry Transport) is a lightweight publish/subscribe messaging protocol ideal for connecting IoT devices, offering minimal code and network bandwidth requirements.
2. MQTT metrics are crucial for assessing the health and performance of IoT systems, including metrics like the percentage of connected devices, message load levels, and active users.
3. Grafana and Graphite are open-source monitoring tools that provide powerful capabilities for visualizing, processing, and analyzing time-series data, making them suitable for monitoring MQTT in IoT.
4. Implementing MQTT monitoring with Graphite and Grafana can be achieved through various brokers like Eclipse Mosquitto and HiveMQ, which can be integrated to send MQTT metrics data to Graphite.

What is MQTT in IoT? 

MQTT is a lightweight publish/subscribe messaging protocol for the Internet of Things (IoT). It is ideal for connecting remote devices with little code and minimal network bandwidth. MQTT supports two-way communication and allows you to exchange messages between the device and the cloud and the cloud on the device. MQTT supports persistent sessions, which reduces the time it takes for a client to reconnect to a broker when the connection is interrupted.

In the MQTT protocol, there are concepts of “clients” and “broker”. Clients are devices that interact with a server (broker). The broker is engaged in the transfer of data between clients. When a client wants to distribute information, it publishes this information on a certain topic, after which the broker sends this information to all clients who have subscribed to this topic. The MQTT protocol is widely used in various industries such as automotive, manufacturing, telecommunications, etc. You can scale it to connect to millions of IoT devices.

MQTT metrics

Metrics are important indicators that allow you to assess the current state of the system, identify and correct errors in its operation, and predict the behavior of the system in the future.

Let’s list the main MQTT metrics.

1. Percentage of connected devices. This metric provides you with information about the number of IoT devices that are typically connected. With this data, you can diagnose the general health of your IoT devices.
2. Loading a message. Monitoring the message load level allows you to track suspicious activity from one of the connected IoT devices. Using this data, you can also decide to expand the capabilities of message processing.
3. Active users. Monitoring the number of active users allows you to understand how many customers you have and how popular your products are.

Benefits of Grafana and Graphite for MQTT/IoT monitoring

You need special systems to monitor MQTT metrics. Grafana and Graphite are flexible open-source monitoring tools that allow you to visualize, process, and analyze time-series data. Graphite allows you to collect time-series data from different sources, store them and process them. It also has a scalable real-time charting solution. You can use Graphite as the data source for Grafana.

Grafana has advanced data visualization tools. It allows you to create amazing and easily customizable dynamic dashboards. Grafana dashboard consists of a group of widgets with which you can create different types of graphs. Grafana allows you to explore metrics with ad hoc queries and dynamic drill-downs. You can split the view and compare different time ranges, queries, and data sources side by side. It provides you with the ability to use different data sources on the same chart. It also allows you to create alerts and send notifications to systems like Slack, PagerDuty, VictorOps, and OpsGenie.

To learn more about how to use Graphite and Grafana to monitor IoT devices, sign up for a Metricfire free trial or book a demo with our team.

Implementing MQTT, Graphite & Grafana using a Broker

MQTT brokers are used for transferring data between clients (publishers and subscribers). There are different types of brokers. For example, Eclipse Mosquitto, HiveMQ, and others. In this section, we will look at how to send MQTT metrics to Graphite using different ways. In order to send MQTT data to hosted Graphite, it must be formatted in Graphite.

Eclipse Mosquitto

Eclipse Mosquitto is an open-source message broker that implements MQTT protocol versions 5.0, 3.1.1, and 3.1. Mosquitto is lightweight and suitable for use on all devices, from low-power single-board computers to full servers.

To install Eclipse Mosquitto, read the detailed instructions for different platforms. After installation, the broker should start automatically. If it hasn't been launched, run the command:

mosquitto

If you want to subscribe to a topic, use the command:

mosquitto_sub -t 'test/topic' -v

Run the following command to post a message:

mosquitto_pub -t 'test/topic' -m 'hello world'

To export data from Mosquitto to Graphite, you can use special programs. One such program is Mosquitto Exporter. You can run the exporter docker image using the following command:

docker run \
  -p 9234:9234 sapcc/mosquitto-exporter \
  --endpoint tcp://mosquitto:1883

mqtt2graphite

Mqtt2graphite is a program that allows you to subscribe to any number of MQTT topics, extract payloads from messages, and send them to Graphite. The data can be in the form of simple numbers or JSON strings.

To install and configure the program, run the following commands:

pip install git+https://github.com/jpmens/mqtt2graphite.git
./init_server.sh mqtt.hostname

In order to start mqtt2graphite run the following command or add it to your file /etc/rc.local.

supervisord -c /etc/supervisord.conf

HiveMQ

HiveMQ is an MQTT messaging broker. It allows you to quickly, efficiently, and reliably move data to and from connected IoT devices and enterprise systems. HiveMQ has built-in integration with the Graphite monitoring tool. When working with HiveMQ, it is useful to use Hosted Graphite instead of setting up your own Graphite server.

To do this, first, create a hosted Graphite account and get your API key. Then make the following settings:

graphite.enabled=true
graphite.interval=180
graphite.url=carbon.hostedgraphite.com:2003
graphite.metricPrefix=YOUR_API_KEY

Some hosted Graphite solutions require an API key as a special prefix for metrics. For example, API_KEY.ActiveClients should be used instead of ActiveClients. After making the above settings, you can run HiveMQ and see that it publishes metrics to the hosted Graphite instance. Create dashboards in your Graphite instance and experiment with statistics published by HiveMQ.

The advantages of a Hosted Graphite solution from MetricFire

MetricFire is an open-source monitoring platform. It provides you with hosted Graphite and Grafana solutions. This means that you can use Graphite and Grafana as web applications without having to install and configure them.

To monitor IoT devices using MQTT, Hosted Graphite, and Grafana, you only need to follow a few simple steps.

1. Configure your IoT devices to send MQTT data to an MQTT broker using a QTT channel.
2. Set up Grafana to get IoT data from Graphite for display.

These steps will only take 5 minutes to complete. For comparison, if you decide to install and configure the monitoring system yourself, it can take more than 10 hours.

Let’s consider other important advantages of using solutions from MetricFire.

1. Saving time and resources. There is no need to install and maintain a monitoring system, which can significantly save time and resources.
2. Data availability. You can access your data at any time.
3. A flexible pricing system allows you to choose a plan that suits all your needs and fits your budget.
4. Reliable technical support. The MetricFire team will answer any questions you have about our products.

Conclusion

In this article, we have explained what MQTT is and what main metrics characterize its state. Also, we have looked at the main advantages of using Graphite and Grafana for monitoring MQTT metrics.

Choose hosted solutions from MetricFire and create dynamic information graphics easily and quickly. Get MetricFire free trial or book the demo with our experts and learn more about the possibilities provided by MetricFire.