Consul and Consul-Template are two powerful tools for updating server files and executing a command after the file has been updated. This makes it a great and lightweight tool for managing configuration files for applications that need to reload or restart after a configuration change.

In this post I will describe how to setup a Logstash configuration file for Nginx. I describe writing Nginx Access logs in JSON here. Installing Consul, Consul-Template and Logstash is beyond the scope of this post.


Logstash is a data processing pipeline. It has Inputs, Filters and Outputs. Popular use is for log processing where the Input is a log line from a server, the Filter formats the line (and much more) and the Output sends the formatted line to a receiver like Elasticsearch. Configuration is done via file(s) that is read by Logstash at runtime. The management of these files can be done manually, using configuration management like Saltstack or my favorite, using Consul and Consul-Template.

For this example I will assume logstash-forwarder on the remote server has been configured to tail the JSON formatted nginx access log and shipped the log line(s) to the logstash server.

Formatting Nginx logs as JSON (/etc/nginx/nginx.conf)

Add or modify your log format settings to something similar to below.

log_format main '{"port": "$server_port", "visit": "$cookie_visit", "visitor": "$cookie_visitor", "remote_addr": "$remote_addr", "remote_user": "$remote_user", "time_local": "$time_local", "request": "$request", "status": $status, "bytes_sent": $bytes_sent, "body_bytes_sent": $body_bytes_sent, "http_referer": "$http_referer", "http_user_agent": "$http_user_agent", "request_time": $request_time, "http_host": "$host", "cf_connecting_ip": "$http_cf_connecting_ip", "cf_ipcountry": "$http_cf_ipcountry", "x_forwarded_for": "$http_x_forwarded_for", "x_request_id":"$http_x_request_id", "true_client_ip": "$http_true_client_ip", "x_requested_with": "$http_x_requested_with"}';

access_log /var/log/nginx/access.log main;

Add Keys to Consul

Input (Consul Key: services/logging/etc/logstash/conf.d/inputs)


input {
  lumberjack {
    port => 5043
    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
    type => "default"

Filter (Consul Key: services/logging/etc/logstash/conf.d/filters)


filter {
  if [type] == "nginx" {
    json {
      source => "message"

    date {
      match => [ "time_local",  "dd/MMM/YYYY:HH:mm:ss Z" ]
      locale => "en"

    if "_jsonparsefailure" not in [tags] {
      mutate {
        remove_field => [ "message" ]

Output (Consul Key: services/logging/etc/logstash/conf.d/outputs)


output {
  elasticsearch {
    hosts => ["localhost"]
    index => "logstash-nginx-%{+YYYY.MM.dd}"
    document_type => doc

Create Consul-Template Files


That’s it. Just restart Consul-Template and the logstash configuration file will be created and logstash restarted. If there were no errors you’ll start processing Nginx log lines. Cheers!

Tagged on: