This post is a brief introduction of how I setup a system to add, update and remove Nagios configurations as servers are created and destroyed. Future posts detailing each step will be coming soon. Until then you can go to hashicorp.com or nagios.org for help.

If you use the Nagios Open Source edition and manage more than a couple dozen machines and services then you know how challenging it can be to keep all the different configuration files up-to-date. There are contacts, contact groups, services, service groups, hosts, host groups, object inheritance and escalations to keep track of. Consul and Consul-Template can completely automate the creation and updating of all your Nagios settings.

The Pieces

The features I use from Consul and Consul-Template are:

  • Consul (service, nodes list, key/value store)
  • Consul-Template (definition file, template file)

Setup

The idea is for all nodes to be running Consul and have an entry in the key/value store. Additionally, the Nagios server is running Consul-Template and has the template definition and template files.

There needs to be a mechanism to create an association in the key/value store for each server and its related contacts, host and service groups. There are many ways to get this information and it can be as simple as a server bootstrap to make a CURL call to the Consul API and issue a PUT command. I use a Consul Service Health Check file, which acts like a cronjob, to grab specific information about a server and update the appropriate keys.

Here’s an example hierarchy of the key store containing paths and some sample values:

Note that the values are JSON. This takes advantage of Consul-Templates JSON parsing methods on the Nagios server.

The Template

Consul-Template would then have a definition file and a template to read the top key and loop through each host and hostgroup, write the nagios config file and reload nagios.

Hopefully this gets you interested in trying to automate Nagios for your infrastructure. Now let’s get into the details.

Tagged on: