[Howto] Icinga2 with Graphite and Grafana on Debian
Written by Alexander Wirt, credativ GmbH
Tranlsated by Irenie White, credativ UK
In this post we describe how to integrate Icinga2 with Graphite and Grafana on Debian stable (jessie).
What is Graphite?
Graphite stores performance data on a configurable period. Via a defined interface, services can send metrics to Graphite, which are then stored for the designated period. Possible examples of the type of metrics that it might be useful for are CPU utilization or traffic of a web server. Graphs can be generated from the various metrics via the integrated web interface in Graphite. This allows changes in values to be observed across different time periods. A good example of where this type of trend analysis could be useful is monitoring levels on a hard drive. By using a Trend Graph, it's clear to see the growth rate of the footprint and anticipate when an exchange of storage will become necessary.
What is Grafana?
Graphite does have a private web interface but it is neither attractive nor flexible. This is where Grafana comes in. Grafana is a frontend for various storage metrics, supporting Graphite, InfluxDB and OpenTSDB, among others. Grafana offers an intuitive interface through which you can create graphs to represent the metrics and a variety of functions to optimise their appearance and presentation. The graphs can then be summarized in dashboards; you can also opt to display one from a specific host only, through parameterization.
Installation of Icinga2
Here we describe the only installation which is essential for Graphite. Icinga2 packages in the current version of Debian can be obtained directly from Debmon project.
At the Debmon project up to date versions of the different monitoring tools software for the different Debian releases are provided by the official Debian package maintainers.
To include these packages, use the following commands:
# add debmon
cat <<EOF >/etc/apt/sources.list.d/debmon.list
deb http://debmon.org/debmon debmon-jessie main
# add debmon key
wget -O - http://debmon.org/debmon/repo.key 2>/dev/null | apt-key add -
# update repos
Finally we can install Icinga2:
apt-get install icinga2
Installation of Graphite and Graphite-Web
Once Icinga2 is installed, Graphite and Graphite-Web can also be installed.
# install packages for icinga2 and graphite-web and carbon
apt-get install icinga2 graphite-web graphite-carbon libapache2-mod-wsgi apache2<br><br><br><br><br><br><br>
Configuration of Icinga2 with Graphite
Icinga2 must be configured so that all defined metrics are exported to Graphite. The Graphite component receives this data called "Carbon". In our sample installation Carbon runs on the same host as Icinga2 and uses the default port, meaning no further configuration of Icinga2 is necessary - it is enough to export on.
To do this, simply enter the command:
Icinga2 feature enable graphite
Subsequently Icinga2 must be restarted:
service Icinga2 restart
If the Carbon Server is running on a different host or a different port, the configuration of the file
Icinga2 should be adjusted
Details can be found in the
<a href:“Icinga2 documentation”
If the configuration was successful, then after a short time a number of files should appear in "/ var / lib / graphite / whisper / icinga". If they don't, then you should take a look in the log file of Icinga2 (located in "/var/log/icinga2/icinga2.log")
Configuration of Graphite – Web
Grafana uses the frontend of Graphite as an interface for the stored metrics of Graphite so correct configuration of Graphite-web is very important. For performance reasons, we operate Graphite-web as a WSGI module. A number of configuration steps are necessary:
1. First, we create a user database for Graphite-web. Since we will not have many users, at this point we use SQLite as the backend for our user data.
We do this using the following commands which initialize the user database and transfer ownership to the user under which the Web front-end runs:
<span style="font-family: Courier, monospace;">gr</span><span style="font-family: Courier, monospace;">aphite-manage syncdb</span>
<span style="font-family: Courier, monospace;"> chown _graphite:_graphite /var/lib/graphite/graphite.db</span>
2. Then we activate the WSGI module in Apache:
3. For simplicity the web interface should run in a separate virtual host and on its own port. So that Apache listens to this port, we add the line
"Listen 8000" to the file "/etc/apache2/ports.conf"
4. The Graphite Debian package already provides a configuration file for Apache, which is fine to use with slight adaptations.
<span style="font-family: Courier, monospace;">cp /usr/share/graphite-web/apache2- graphite.conf /etc/apache2/sites-available/graphite.conf </span>
So that the virtual host also uses port 8000, we replace the line:
<span style="color: #222222;"><span style="font-family: monospace;"><VirtualHost *:80></span></span>
<span style="font-family: Courier, monospace;"><VirtualHost *:8000></span>
5. We can now activate the new virtual host via a2ensite graphite and restart Apache:
# a2ensite graphite
And restart Apache using:
# systemctl restart apache2
6. You should now be able to reach Graphite-web at http://YOURIP:8000/ If you cannot, the Apache log files in
"/ var / log / apache2 /"
can provide valuable information.
Configuration of Grafana
Grafana is not currently included in Debian. However, the author offers an Apt repository from which you can install Grafana.
Even if the repository points to Wheezy, the packages should still function under Debian Jessie.
The repository is only accessible via https - so first, you need to install https support for apt:
<span style="font-family: Courier, monospace;">apt-get install apt-transport-https</span>
Next, the repository can be integrated.
<span style="font-family: Courier, monospace;"># add repo (package for wheezy works on jessie)</span>
<span style="font-family: Courier, monospace;">cat <<EOF >/etc/apt/sources.list.d/grafana.list</span>
<span style="font-family: Courier, monospace;">deb https://packagecloud.io/grafana/stable/debian/ wheezy main</span>
<span style="font-family: Courier, monospace;">EOF</span>
<span style="font-family: Courier, monospace;"># add key</span>
<span style="font-family: Courier, monospace;">curl -s https://packagecloud.io/gpg.key | sudo apt-key add -</span>
<span style="font-family: Courier, monospace;"># update repos</span>
<span style="font-family: Courier, monospace;">apt-get update</span>
After this, the package can be installed:
<span style="font-family: 'Liberation Sans', sans-serif;">apt-get install grafana</span>
For Grafana to run we still need to activate the service
<span style="font-family: 'Liberation Sans', sans-serif;"><span style="font-family: Courier, monospace;">systemctl enable grafana-server.service </span> and start <span style="font-family: Courier, monospace;">systemctl start grafana server</span></span>
Grafana is now accessible at http://YOURIP:3000/. The default user name and password in our example is 'admin'. This password should, of course, be replaced by a secure password at the earliest opportunity.
Grafana must then be configured so that it uses Graphite as a data source. For simplicity, the configuration is explained in the following screencast:
After setting up Graphite as the data source, we can create our first graph. Here is another short screencast to illustrate this:
Congratulations! You have successfully installed and configured Icinga2, Graphite and Grafana. For subsequent steps, please refer to the documentation for the specific projects:
To read more about credativ's work with Debian, please read our Debian blog posts.