Graphite and Graphite Web on Debian – Part 2

graphiteI really left you hanging last post, but for me getting Graphite running was a real challenge as well, because of a lot of, to me, unknown software and mechanisms.

But you have to learn someday right? Well, I did and am now running some beautiful graphs alongside my Icinga configuration. This post we’re wrapping up the gunicorn installation, carbon and graphite configuration and Icinga integration.

Here we go.

As said in the last post, surfing to¬†https://graphite.yourdomain.com will result in a “502 Bad Gateway” error. This is because Gunicorn is not running and serving anything.

We’d want to start and enable the gunicorn service and keep it running at all times.

Create /etc/systemd/system/gunicorn.service:

[Unit]
Description=gunicorn daemon
After=network.target

[Service]
User=root
Group=www-data
WorkingDirectory=/opt/graphite/webapp
ExecStart=/usr/bin/gunicorn wsgi --workers=4 --bind=0.0.0.0:8080 --log-file=/var/log/gunicorn.log --preload --pythonpath=/opt/graphite/webapp/graphite

[Install]
WantedBy=multi-user.target

Start and enable it:

# systemctl start gunicorn
# systemctl enable gunicorn

Restart your server and it should be running:

# systemctl status gunicorn

Test with your browser.

Carbon needs a lot of configuration. To start out, luckily we can do with the default example files. We only need to copy them to the correct location.

# cd /opt/graphite/conf/
# cp carbon.conf.example carbon.conf
# cp storage-schemas.conf.example storage-schemas.conf
# cp storage-aggregation.conf.example storage-aggregation.conf
# cp relay-rules.conf.example relay-rules.conf
# cp aggregation-rules.conf.example aggregation-rules.conf
# cp rewrite-rules.conf.example rewrite-rules.conf
# cp whitelist.conf.example whitelist.conf
# cp blacklist.conf.example blacklist.conf

As with Gunicorn, we need to enable this as a service, start carbon and make sure it starts at boot.

Create /etc/systemd/system/carbon-cache.service:

[Unit]
Description=Graphite Carbon Cache
After=network.target

[Service]
Type=forking
StandardOutput=syslog
StandardError=syslog
ExecStart=/opt/graphite/bin/carbon-cache.py --config=/opt/graphite/conf/carbon.conf --pidfile=/var/run/carbon-cache.pid start
ExecReload=/bin/kill -USR1 $MAINPID
PIDFile=/var/run/carbon-cache.pid

[Install]
WantedBy=multi-user.target

Start and enable it:

# systemctl start carbon-cache
# systemctl enable carbon-cache

Restart your server and it should be running:

# systemctl status carbon-cache

We’re good to go! Let’s get our data into Icinga.

# icinga2 feature enable graphite
# vi /etc/icinga2/features-enabled/graphite.conf

The contents of this configuration file:

library "perfdata"

object GraphiteWriter "graphite" {
host = "127.0.0.1"
port = 2003
enable_send_thresholds = true
}

Restart Icinga:

# systemctl restart icinga2

Get your hands on the proper module:

# cd /usr/share/icingaweb2/modules
# git clone https://github.com/Icinga/icingaweb2-module-graphite.git
# mv icingaweb2-module-graphite graphite

  • Now login at the Icinga Web 2 portal and finish the configuration:
  • Go to Configuration – Modules
  • Click Graphite – Enable
  • Choose Backend and fill in the proper URL
  • Graphite Web user and Graphite Web password are emtpy
  • Save changes
  • Go to Graphite – Hosts or Services and integrate the graphs in your dashboard

Congratulation with your full functional monitoring system. This really was a great adventure.

My next step will probably be getting even more data in the graphs and when I do, I’ll be sure to post my findings.

Complete series: Intro, LEMP preparation, Installation, Clients part 1, Clients part 2, Third Party Plugins part 1, Third Party Plugins part 2, Schedule downtime, Telegram push part 1, Telegram push part 2, Graphite part 1 and Graphite part 2.

Update: Check out how we’ve added Grafana in the mix.

2 comments Add yours
  1. a lot of my graphs appear to have bars and gaprs instead of a smooth fill. Any idea how to fix that? Some graphs are OK.. like ping4 rta/pl.. but load,memory,disk (for windows) all have gaps and bars with a line across the top.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.